Text Generation
PEFT
Safetensors
lora
qwen3.5
instruction-tuning
Arisp commited on
Commit
f264d84
·
verified ·
1 Parent(s): 0d413dc

Update model card

Browse files
Files changed (1) hide show
  1. README.md +67 -15
README.md CHANGED
@@ -1,27 +1,79 @@
1
  ---
2
- license: mit
3
  base_model: Qwen/Qwen3.5-2B-Base
4
  library_name: peft
5
- tags: [lora, qwen3.5, mmlu-pro, memorization]
 
 
 
 
6
  ---
7
 
8
- # memorize adapters
9
 
10
- LoRA adapters from [Memorize](https://github.com/AriSpiesberger/Memorize), a small
11
- lab for poking at Qwen3.5-2B. Every adapter sits on `Qwen/Qwen3.5-2B-Base`, in its
12
- own folder of this repo.
13
 
14
- | adapter | format | base | what it is |
15
- | --- | --- | --- | --- |
16
- | `sft-run2-torch` | peft | Qwen/Qwen3.5-2B-Base | Instruction SFT: the sft-run2 recipe (memorize.sft_torch) on the data from memorize.sft_data. 18/19 instruction probes, 38.0% direct-answer MMLU-Pro (500 held-out). Default start for run_split.py. |
17
 
18
- ```python
19
- from huggingface_hub import snapshot_download
20
- snapshot_download("Arisp/memorize-adapters", allow_patterns=["sft-run2-torch/*"], local_dir="adapters")
21
 
22
- # or, for a peft adapter, straight from the hub:
 
23
  from peft import PeftModel
24
- model = PeftModel.from_pretrained(base_model, "Arisp/memorize-adapters", subfolder="sft-run2-torch")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
  ```
26
 
27
- In the Memorize repo: `python -m memorize.hub pull <name>` (or `--all`).
 
 
1
  ---
2
+ license: apache-2.0
3
  base_model: Qwen/Qwen3.5-2B-Base
4
  library_name: peft
5
+ pipeline_tag: text-generation
6
+ datasets:
7
+ - allenai/tulu-3-sft-personas-instruction-following
8
+ - databricks/databricks-dolly-15k
9
+ tags: [lora, peft, qwen3.5, instruction-tuning]
10
  ---
11
 
12
+ # instruct
13
 
14
+ Instruction-tuned Qwen3.5-2B-Base: a rank-8 LoRA trained with memorize.sft_torch (the sft-run2 recipe) on the memorize.sft_data mix. Start point for the label-split experiment.
 
 
15
 
16
+ From [Memorize](https://github.com/AriSpiesberger/Memorize), a small lab for poking at Qwen3.5-2B.
 
 
17
 
18
+ ## use
 
 
19
 
20
+ ```python
21
+ import torch
22
  from peft import PeftModel
23
+ from transformers import AutoModelForCausalLM, AutoTokenizer
24
+
25
+ tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-2B-Base")
26
+ base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-2B-Base", dtype=torch.bfloat16)
27
+ model = PeftModel.from_pretrained(base, "Arisp/memorize-instruct")
28
+
29
+ ids = tok.apply_chat_template(
30
+ [{"role": "user", "content": "Name three fruits."}],
31
+ add_generation_prompt=True, enable_thinking=False, return_tensors="pt",
32
+ )
33
+ out = model.generate(ids, max_new_tokens=100, eos_token_id=tok.convert_tokens_to_ids("<|im_end|>"))
34
+ print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
35
+ ```
36
+
37
+ Use the chat template with `enable_thinking=False` (how it was trained), and stop
38
+ on `<|im_end|>`: the base model's own config only stops on `<|endoftext|>`.
39
+
40
+ ## results
41
+
42
+ | | base model | this adapter |
43
+ | --- | --- | --- |
44
+ | validation loss (200 held-out chats) | 1.361 | 1.250 |
45
+ | instruction probes passed (greedy) | 15/19 | 18/19 |
46
+ | MMLU-Pro, direct letter, n=500 | 37.4% | 38.0% |
47
+
48
+ Probes are short prompts with checkable output constraints
49
+ ([memorize/probes.py](https://github.com/AriSpiesberger/Memorize/blob/main/memorize/probes.py)). MMLU-Pro is
50
+ scored from the letter right after `ANSWER:` with no reasoning, on label-split C, split seed 0
51
+ ([memorize/label_split.py](https://github.com/AriSpiesberger/Memorize/blob/main/memorize/label_split.py)).
52
+
53
+ ## how it was made
54
+
55
+ - **Data**: 1,800 constraint-following chats from tulu-3-sft-personas-instruction-following
56
+ plus 600 Dolly-15k chats, 200 held out for validation, each at most 512 tokens;
57
+ multiple-choice and step-by-step items dropped. Seed 0.
58
+ - **LoRA**: rank 8, alpha 160 (mlx-lm's scale of 20), no dropout, on every linear
59
+ layer of every block (attention, linear attention, MLP); 8.4M parameters.
60
+ - **Training**: 1 epoch, 300 steps of 8 chats, Adam at lr 1e-5 (20 warmup steps
61
+ from 1%, then cosine to 10%), gradient norm clipped at 20, bf16, loss on the
62
+ assistant reply only. Seed 0.
63
+ - **Environment**: Python 3.12.7, torch 2.11.0+cu128, transformers 5.18.0, peft 0.21.2, datasets 5.1.0, one RTX 4090 (Windows 11).
64
+
65
+ `sft_torch.json` (in this repo) has every argument and the validation curve;
66
+ `train.log` is the full training log.
67
+
68
+ ## reproduce
69
+
70
+ ```bash
71
+ git clone https://github.com/AriSpiesberger/Memorize && cd Memorize
72
+ git checkout aad35a9 # the code that trained it
73
+ python -m venv .venv && .venv/bin/pip install -r requirements.txt # Windows: .venv\Scripts\pip
74
+ python -m memorize.sft_data # data/sft/train.jsonl, valid.jsonl
75
+ python -m memorize.sft_torch --out adapters/instruct
76
  ```
77
 
78
+ The data files should hash to sha256 `92234522d2e41621…` (train) and
79
+ `fdcbf7bca4450523…` (valid) if the upstream datasets are unchanged.