brucoder commited on
Commit
612ebbf
·
verified ·
1 Parent(s): e764835

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +84 -24
README.md CHANGED
@@ -13,27 +13,9 @@ tags:
13
 
14
  # winter-frost-2
15
 
16
- This is a fine-tune of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct), created by INEZA AIME BRUNO (brucoder). It is **not** an original model trained from scratch — it is Qwen2.5-7B-Instruct with a LoRA adapter merged into the base weights.
17
 
18
- ## What this is
19
-
20
- - **Base model:** Qwen/Qwen2.5-7B-Instruct (all core knowledge, reasoning, and instruction-following ability comes from this base model)
21
- - **Fine-tuning method:** QLoRA (4-bit NF4 quantized base, fp16 compute, LoRA r=16 alpha=32) on a single T4 GPU
22
- - **Target modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (all attention and MLP projections)
23
- - **Training data:** ~690 examples total
24
- - 300 general instruction examples from [HuggingFaceH4/no_robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots)
25
- - 300 Python coding examples from [iamtarun/python_code_instructions_18k_alpaca](https://huggingface.co/datasets/iamtarun/python_code_instructions_18k_alpaca)
26
- - 90 custom persona/identity examples teaching the model to identify its creator
27
- - **Training length:** 3 epochs, 132 steps, ~45 minutes on a free-tier T4 GPU
28
- - The LoRA adapter was merged directly into the base model's weights (not loaded separately at inference time)
29
-
30
- ## What this is not
31
-
32
- This is **not** a new model architecture, not trained from scratch, and not a significant capability upgrade over the base model. With ~690 training examples, this fine-tune reliably shifts small, well-defined behaviors (like stating who created it) but should not be expected to meaningfully change general reasoning, coding ability, or knowledge compared to the base Qwen2.5-7B-Instruct model.
33
-
34
- ## Usage
35
-
36
- This model is a standalone, merged checkpoint — no PEFT/LoRA adapter loading required:
37
 
38
  ```python
39
  from transformers import AutoModelForCausalLM, AutoTokenizer
@@ -54,15 +36,93 @@ inputs = tokenizer.apply_chat_template(
54
  messages, tokenize=True, add_generation_prompt=True,
55
  return_tensors="pt", return_dict=True
56
  ).to(model.device)
57
- output = model.generate(**inputs, max_new_tokens=100, do_sample=False, pad_token_id=tokenizer.eos_token_id)
 
58
  print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
59
  ```
60
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
61
  ## Adapter-only version
62
 
63
- If you'd prefer to load the LoRA adapter separately on top of the base model instead of this merged version, it's available at [brucoder/winter-frost-2-adapter](https://huggingface.co/brucoder/winter-frost-2-adapter).
64
 
65
  ## Creator
66
 
67
- Created by INEZA AIME BRUNO (brucoder).
68
- Instagram: [iabru.ceo](https://instagram.com/iabru.ceo)
 
13
 
14
  # winter-frost-2
15
 
16
+ A fine-tune of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct), created by INEZA AIME BRUNO (brucoder). This is a standalone, merged checkpoint — the LoRA adapter has already been folded into the base model's weights, so it loads and runs like any other model, no adapter-loading step required.
17
 
18
+ ## Quickstart
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
  ```python
21
  from transformers import AutoModelForCausalLM, AutoTokenizer
 
36
  messages, tokenize=True, add_generation_prompt=True,
37
  return_tensors="pt", return_dict=True
38
  ).to(model.device)
39
+
40
+ output = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7, top_p=0.9, pad_token_id=tokenizer.eos_token_id)
41
  print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
42
  ```
43
 
44
+ ## Run it
45
+
46
+ ### Google Colab (free GPU)
47
+
48
+ Paste into one cell:
49
+
50
+ ```python
51
+ !pip install -q -U transformers accelerate bitsandbytes
52
+
53
+ from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
54
+ import torch
55
+
56
+ bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
57
+ bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True)
58
+
59
+ tokenizer = AutoTokenizer.from_pretrained("brucoder/winter-frost-2")
60
+ model = AutoModelForCausalLM.from_pretrained("brucoder/winter-frost-2",
61
+ quantization_config=bnb_config, device_map="auto")
62
+
63
+ conversation = [{"role": "system", "content": "You are a helpful assistant."}]
64
+ print("Model loaded. Chat with winter-frost-2 (type 'quit' to stop).\n")
65
+
66
+ while True:
67
+ user_input = input("You: ")
68
+ if user_input.strip().lower() in ("quit", "exit"):
69
+ break
70
+ conversation.append({"role": "user", "content": user_input})
71
+ inputs = tokenizer.apply_chat_template(conversation, tokenize=True, add_generation_prompt=True,
72
+ return_tensors="pt", return_dict=True).to(model.device)
73
+ output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.7,
74
+ top_p=0.9, pad_token_id=tokenizer.eos_token_id)
75
+ response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
76
+ print(f"winter-frost-2: {response}\n")
77
+ conversation.append({"role": "assistant", "content": response})
78
+ ```
79
+
80
+ ### Local machine
81
+
82
+ Requires Python 3.9+. A GPU with 16GB+ VRAM keeps this fast; it will also run on CPU, just slowly. First run downloads ~15GB of weights (cached after that):
83
+
84
+ ```bash
85
+ pip install -q -U transformers accelerate torch
86
+ python3 -c "
87
+ from transformers import AutoModelForCausalLM, AutoTokenizer
88
+ import torch
89
+
90
+ tokenizer = AutoTokenizer.from_pretrained('brucoder/winter-frost-2')
91
+ model = AutoModelForCausalLM.from_pretrained('brucoder/winter-frost-2', torch_dtype=torch.float16, device_map='auto')
92
+
93
+ conversation = [{'role': 'system', 'content': 'You are a helpful assistant.'}]
94
+ print('Model loaded. Chat with winter-frost-2 (type quit to stop).')
95
+
96
+ while True:
97
+ user_input = input('You: ')
98
+ if user_input.strip().lower() in ('quit', 'exit'):
99
+ break
100
+ conversation.append({'role': 'user', 'content': user_input})
101
+ inputs = tokenizer.apply_chat_template(conversation, tokenize=True, add_generation_prompt=True, return_tensors='pt', return_dict=True).to(model.device)
102
+ output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.7, top_p=0.9, pad_token_id=tokenizer.eos_token_id)
103
+ response = tokenizer.decode(output[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
104
+ print('winter-frost-2:', response)
105
+ conversation.append({'role': 'assistant', 'content': response})
106
+ "
107
+ ```
108
+
109
+ ## What this is
110
+
111
+ - **Base model:** Qwen/Qwen2.5-7B-Instruct
112
+ - **Fine-tuning method:** QLoRA (4-bit NF4 quantized base, fp16 compute, LoRA r=16, alpha=32) on a single T4 GPU
113
+ - **Target modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
114
+ - **Training data:** ~690 examples — 300 general instructions from [HuggingFaceH4/no_robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots), 300 Python coding examples from [iamtarun/python_code_instructions_18k_alpaca](https://huggingface.co/datasets/iamtarun/python_code_instructions_18k_alpaca), and 90 custom identity examples
115
+ - **Training length:** 3 epochs, 132 steps, ~45 minutes
116
+
117
+ ## What this is not
118
+
119
+ Not a new architecture and not trained from scratch. With ~690 training examples, this reliably shifts small, well-defined behaviors (like stating who created it) but should not be expected to meaningfully change general reasoning, coding ability, or knowledge compared to the base model.
120
+
121
  ## Adapter-only version
122
 
123
+ If you'd rather load the LoRA adapter separately on top of the base model instead of this merged version, it's available at [brucoder/winter-frost-2-adapter](https://huggingface.co/brucoder/winter-frost-2-adapter).
124
 
125
  ## Creator
126
 
127
+ INEZA AIME BRUNO (brucoder)
128
+ Instagram: [iabru.ceo](https://instagram.com/iabru.ceo)