Text Generation
PEFT
Safetensors
lora
qwen3.5
instruction-tuning
File size: 3,378 Bytes
c6e610d
f264d84
c6e610d
 
f264d84
 
 
 
 
c6e610d
 
f264d84
c6e610d
f264d84
c6e610d
f264d84
c6e610d
f264d84
c6e610d
f264d84
 
c6e610d
f264d84
 
 
 
 
 
 
 
5174afa
 
f264d84
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
c6e610d
 
f264d84
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
---
license: apache-2.0
base_model: Qwen/Qwen3.5-2B-Base
library_name: peft
pipeline_tag: text-generation
datasets:
  - allenai/tulu-3-sft-personas-instruction-following
  - databricks/databricks-dolly-15k
tags: [lora, peft, qwen3.5, instruction-tuning]
---

# instruct

Instruction-tuned Qwen3.5-2B-Base: a rank-8 LoRA trained with memorize.sft_torch (the sft-run2 recipe) on the memorize.sft_data mix. Start point for the label-split experiment.

From [Memorize](https://github.com/AriSpiesberger/Memorize), a small lab for poking at Qwen3.5-2B.

## use

```python
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-2B-Base")
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-2B-Base", dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "Arisp/memorize-instruct")

ids = tok.apply_chat_template(
    [{"role": "user", "content": "Name three fruits."}],
    add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True,
)["input_ids"]
out = model.generate(ids, max_new_tokens=100, eos_token_id=tok.convert_tokens_to_ids("<|im_end|>"))
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
```

Use the chat template with `enable_thinking=False` (how it was trained), and stop
on `<|im_end|>`: the base model's own config only stops on `<|endoftext|>`.

## results

| | base model | this adapter |
| --- | --- | --- |
| validation loss (200 held-out chats) | 1.361 | 1.250 |
| instruction probes passed (greedy) | 15/19 | 18/19 |
| MMLU-Pro, direct letter, n=500 | 37.4% | 38.0% |

Probes are short prompts with checkable output constraints
([memorize/probes.py](https://github.com/AriSpiesberger/Memorize/blob/main/memorize/probes.py)). MMLU-Pro is
scored from the letter right after `ANSWER:` with no reasoning, on label-split C, split seed 0
([memorize/label_split.py](https://github.com/AriSpiesberger/Memorize/blob/main/memorize/label_split.py)).

## how it was made

- **Data**: 1,800 constraint-following chats from tulu-3-sft-personas-instruction-following
  plus 600 Dolly-15k chats, 200 held out for validation, each at most 512 tokens;
  multiple-choice and step-by-step items dropped. Seed 0.
- **LoRA**: rank 8, alpha 160 (mlx-lm's scale of 20), no dropout, on every linear
  layer of every block (attention, linear attention, MLP); 8.4M parameters.
- **Training**: 1 epoch, 300 steps of 8 chats, Adam at lr 1e-5 (20 warmup steps
  from 1%, then cosine to 10%), gradient norm clipped at 20, bf16, loss on the
  assistant reply only. Seed 0.
- **Environment**: Python 3.12.7, torch 2.11.0+cu128, transformers 5.18.0, peft 0.21.2, datasets 5.1.0, one RTX 4090 (Windows 11).

`sft_torch.json` (in this repo) has every argument and the validation curve;
`train.log` is the full training log.

## reproduce

```bash
git clone https://github.com/AriSpiesberger/Memorize && cd Memorize
git checkout aad35a9        # the code that trained it
python -m venv .venv && .venv/bin/pip install -r requirements.txt   # Windows: .venv\Scripts\pip
python -m memorize.sft_data           # data/sft/train.jsonl, valid.jsonl
python -m memorize.sft_torch --out adapters/instruct
```

The data files should hash to sha256 `92234522d2e41621…` (train) and
`fdcbf7bca4450523…` (valid) if the upstream datasets are unchanged.