Text Generation
Transformers
Safetensors
English
qwen2
qlora
lora
fine-tuned
qwen2.5
conversational
text-generation-inference
Instructions to use brucoder/winter-frost-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use brucoder/winter-frost-2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="brucoder/winter-frost-2") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("brucoder/winter-frost-2") model = AutoModelForCausalLM.from_pretrained("brucoder/winter-frost-2", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use brucoder/winter-frost-2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "brucoder/winter-frost-2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "brucoder/winter-frost-2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/brucoder/winter-frost-2
- SGLang
How to use brucoder/winter-frost-2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "brucoder/winter-frost-2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "brucoder/winter-frost-2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "brucoder/winter-frost-2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "brucoder/winter-frost-2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use brucoder/winter-frost-2 with Docker Model Runner:
docker model run hf.co/brucoder/winter-frost-2
Update README.md
Browse files
README.md
CHANGED
|
@@ -13,27 +13,9 @@ tags:
|
|
| 13 |
|
| 14 |
# winter-frost-2
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
-
##
|
| 19 |
-
|
| 20 |
-
- **Base model:** Qwen/Qwen2.5-7B-Instruct (all core knowledge, reasoning, and instruction-following ability comes from this base model)
|
| 21 |
-
- **Fine-tuning method:** QLoRA (4-bit NF4 quantized base, fp16 compute, LoRA r=16 alpha=32) on a single T4 GPU
|
| 22 |
-
- **Target modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (all attention and MLP projections)
|
| 23 |
-
- **Training data:** ~690 examples total
|
| 24 |
-
- 300 general instruction examples from [HuggingFaceH4/no_robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots)
|
| 25 |
-
- 300 Python coding examples from [iamtarun/python_code_instructions_18k_alpaca](https://huggingface.co/datasets/iamtarun/python_code_instructions_18k_alpaca)
|
| 26 |
-
- 90 custom persona/identity examples teaching the model to identify its creator
|
| 27 |
-
- **Training length:** 3 epochs, 132 steps, ~45 minutes on a free-tier T4 GPU
|
| 28 |
-
- The LoRA adapter was merged directly into the base model's weights (not loaded separately at inference time)
|
| 29 |
-
|
| 30 |
-
## What this is not
|
| 31 |
-
|
| 32 |
-
This is **not** a new model architecture, not trained from scratch, and not a significant capability upgrade over the base model. With ~690 training examples, this fine-tune reliably shifts small, well-defined behaviors (like stating who created it) but should not be expected to meaningfully change general reasoning, coding ability, or knowledge compared to the base Qwen2.5-7B-Instruct model.
|
| 33 |
-
|
| 34 |
-
## Usage
|
| 35 |
-
|
| 36 |
-
This model is a standalone, merged checkpoint — no PEFT/LoRA adapter loading required:
|
| 37 |
|
| 38 |
```python
|
| 39 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
@@ -54,15 +36,93 @@ inputs = tokenizer.apply_chat_template(
|
|
| 54 |
messages, tokenize=True, add_generation_prompt=True,
|
| 55 |
return_tensors="pt", return_dict=True
|
| 56 |
).to(model.device)
|
| 57 |
-
|
|
|
|
| 58 |
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
|
| 59 |
```
|
| 60 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
## Adapter-only version
|
| 62 |
|
| 63 |
-
If you'd
|
| 64 |
|
| 65 |
## Creator
|
| 66 |
|
| 67 |
-
|
| 68 |
-
Instagram: [iabru.ceo](https://instagram.com/iabru.ceo)
|
|
|
|
| 13 |
|
| 14 |
# winter-frost-2
|
| 15 |
|
| 16 |
+
A fine-tune of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct), created by INEZA AIME BRUNO (brucoder). This is a standalone, merged checkpoint — the LoRA adapter has already been folded into the base model's weights, so it loads and runs like any other model, no adapter-loading step required.
|
| 17 |
|
| 18 |
+
## Quickstart
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
```python
|
| 21 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
| 36 |
messages, tokenize=True, add_generation_prompt=True,
|
| 37 |
return_tensors="pt", return_dict=True
|
| 38 |
).to(model.device)
|
| 39 |
+
|
| 40 |
+
output = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=0.7, top_p=0.9, pad_token_id=tokenizer.eos_token_id)
|
| 41 |
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
|
| 42 |
```
|
| 43 |
|
| 44 |
+
## Run it
|
| 45 |
+
|
| 46 |
+
### Google Colab (free GPU)
|
| 47 |
+
|
| 48 |
+
Paste into one cell:
|
| 49 |
+
|
| 50 |
+
```python
|
| 51 |
+
!pip install -q -U transformers accelerate bitsandbytes
|
| 52 |
+
|
| 53 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
|
| 54 |
+
import torch
|
| 55 |
+
|
| 56 |
+
bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
|
| 57 |
+
bnb_4bit_compute_dtype=torch.float16, bnb_4bit_use_double_quant=True)
|
| 58 |
+
|
| 59 |
+
tokenizer = AutoTokenizer.from_pretrained("brucoder/winter-frost-2")
|
| 60 |
+
model = AutoModelForCausalLM.from_pretrained("brucoder/winter-frost-2",
|
| 61 |
+
quantization_config=bnb_config, device_map="auto")
|
| 62 |
+
|
| 63 |
+
conversation = [{"role": "system", "content": "You are a helpful assistant."}]
|
| 64 |
+
print("Model loaded. Chat with winter-frost-2 (type 'quit' to stop).\n")
|
| 65 |
+
|
| 66 |
+
while True:
|
| 67 |
+
user_input = input("You: ")
|
| 68 |
+
if user_input.strip().lower() in ("quit", "exit"):
|
| 69 |
+
break
|
| 70 |
+
conversation.append({"role": "user", "content": user_input})
|
| 71 |
+
inputs = tokenizer.apply_chat_template(conversation, tokenize=True, add_generation_prompt=True,
|
| 72 |
+
return_tensors="pt", return_dict=True).to(model.device)
|
| 73 |
+
output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.7,
|
| 74 |
+
top_p=0.9, pad_token_id=tokenizer.eos_token_id)
|
| 75 |
+
response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
|
| 76 |
+
print(f"winter-frost-2: {response}\n")
|
| 77 |
+
conversation.append({"role": "assistant", "content": response})
|
| 78 |
+
```
|
| 79 |
+
|
| 80 |
+
### Local machine
|
| 81 |
+
|
| 82 |
+
Requires Python 3.9+. A GPU with 16GB+ VRAM keeps this fast; it will also run on CPU, just slowly. First run downloads ~15GB of weights (cached after that):
|
| 83 |
+
|
| 84 |
+
```bash
|
| 85 |
+
pip install -q -U transformers accelerate torch
|
| 86 |
+
python3 -c "
|
| 87 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 88 |
+
import torch
|
| 89 |
+
|
| 90 |
+
tokenizer = AutoTokenizer.from_pretrained('brucoder/winter-frost-2')
|
| 91 |
+
model = AutoModelForCausalLM.from_pretrained('brucoder/winter-frost-2', torch_dtype=torch.float16, device_map='auto')
|
| 92 |
+
|
| 93 |
+
conversation = [{'role': 'system', 'content': 'You are a helpful assistant.'}]
|
| 94 |
+
print('Model loaded. Chat with winter-frost-2 (type quit to stop).')
|
| 95 |
+
|
| 96 |
+
while True:
|
| 97 |
+
user_input = input('You: ')
|
| 98 |
+
if user_input.strip().lower() in ('quit', 'exit'):
|
| 99 |
+
break
|
| 100 |
+
conversation.append({'role': 'user', 'content': user_input})
|
| 101 |
+
inputs = tokenizer.apply_chat_template(conversation, tokenize=True, add_generation_prompt=True, return_tensors='pt', return_dict=True).to(model.device)
|
| 102 |
+
output = model.generate(**inputs, max_new_tokens=300, do_sample=True, temperature=0.7, top_p=0.9, pad_token_id=tokenizer.eos_token_id)
|
| 103 |
+
response = tokenizer.decode(output[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
|
| 104 |
+
print('winter-frost-2:', response)
|
| 105 |
+
conversation.append({'role': 'assistant', 'content': response})
|
| 106 |
+
"
|
| 107 |
+
```
|
| 108 |
+
|
| 109 |
+
## What this is
|
| 110 |
+
|
| 111 |
+
- **Base model:** Qwen/Qwen2.5-7B-Instruct
|
| 112 |
+
- **Fine-tuning method:** QLoRA (4-bit NF4 quantized base, fp16 compute, LoRA r=16, alpha=32) on a single T4 GPU
|
| 113 |
+
- **Target modules:** q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
|
| 114 |
+
- **Training data:** ~690 examples — 300 general instructions from [HuggingFaceH4/no_robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots), 300 Python coding examples from [iamtarun/python_code_instructions_18k_alpaca](https://huggingface.co/datasets/iamtarun/python_code_instructions_18k_alpaca), and 90 custom identity examples
|
| 115 |
+
- **Training length:** 3 epochs, 132 steps, ~45 minutes
|
| 116 |
+
|
| 117 |
+
## What this is not
|
| 118 |
+
|
| 119 |
+
Not a new architecture and not trained from scratch. With ~690 training examples, this reliably shifts small, well-defined behaviors (like stating who created it) but should not be expected to meaningfully change general reasoning, coding ability, or knowledge compared to the base model.
|
| 120 |
+
|
| 121 |
## Adapter-only version
|
| 122 |
|
| 123 |
+
If you'd rather load the LoRA adapter separately on top of the base model instead of this merged version, it's available at [brucoder/winter-frost-2-adapter](https://huggingface.co/brucoder/winter-frost-2-adapter).
|
| 124 |
|
| 125 |
## Creator
|
| 126 |
|
| 127 |
+
INEZA AIME BRUNO (brucoder)
|
| 128 |
+
Instagram: [iabru.ceo](https://instagram.com/iabru.ceo)
|