Instructions to use Arisp/memorize-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Arisp/memorize-instruct with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-2B-Base") model = PeftModel.from_pretrained(base_model, "Arisp/memorize-instruct") - Notebooks
- Google Colab
- Kaggle
Update model card
Browse files
README.md
CHANGED
|
@@ -1,27 +1,79 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
| 3 |
base_model: Qwen/Qwen3.5-2B-Base
|
| 4 |
library_name: peft
|
| 5 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
---
|
| 7 |
|
| 8 |
-
#
|
| 9 |
|
| 10 |
-
LoRA
|
| 11 |
-
lab for poking at Qwen3.5-2B. Every adapter sits on `Qwen/Qwen3.5-2B-Base`, in its
|
| 12 |
-
own folder of this repo.
|
| 13 |
|
| 14 |
-
|
| 15 |
-
| --- | --- | --- | --- |
|
| 16 |
-
| `sft-run2-torch` | peft | Qwen/Qwen3.5-2B-Base | Instruction SFT: the sft-run2 recipe (memorize.sft_torch) on the data from memorize.sft_data. 18/19 instruction probes, 38.0% direct-answer MMLU-Pro (500 held-out). Default start for run_split.py. |
|
| 17 |
|
| 18 |
-
|
| 19 |
-
from huggingface_hub import snapshot_download
|
| 20 |
-
snapshot_download("Arisp/memorize-adapters", allow_patterns=["sft-run2-torch/*"], local_dir="adapters")
|
| 21 |
|
| 22 |
-
|
|
|
|
| 23 |
from peft import PeftModel
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
```
|
| 26 |
|
| 27 |
-
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
base_model: Qwen/Qwen3.5-2B-Base
|
| 4 |
library_name: peft
|
| 5 |
+
pipeline_tag: text-generation
|
| 6 |
+
datasets:
|
| 7 |
+
- allenai/tulu-3-sft-personas-instruction-following
|
| 8 |
+
- databricks/databricks-dolly-15k
|
| 9 |
+
tags: [lora, peft, qwen3.5, instruction-tuning]
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# instruct
|
| 13 |
|
| 14 |
+
Instruction-tuned Qwen3.5-2B-Base: a rank-8 LoRA trained with memorize.sft_torch (the sft-run2 recipe) on the memorize.sft_data mix. Start point for the label-split experiment.
|
|
|
|
|
|
|
| 15 |
|
| 16 |
+
From [Memorize](https://github.com/AriSpiesberger/Memorize), a small lab for poking at Qwen3.5-2B.
|
|
|
|
|
|
|
| 17 |
|
| 18 |
+
## use
|
|
|
|
|
|
|
| 19 |
|
| 20 |
+
```python
|
| 21 |
+
import torch
|
| 22 |
from peft import PeftModel
|
| 23 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 24 |
+
|
| 25 |
+
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-2B-Base")
|
| 26 |
+
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-2B-Base", dtype=torch.bfloat16)
|
| 27 |
+
model = PeftModel.from_pretrained(base, "Arisp/memorize-instruct")
|
| 28 |
+
|
| 29 |
+
ids = tok.apply_chat_template(
|
| 30 |
+
[{"role": "user", "content": "Name three fruits."}],
|
| 31 |
+
add_generation_prompt=True, enable_thinking=False, return_tensors="pt",
|
| 32 |
+
)
|
| 33 |
+
out = model.generate(ids, max_new_tokens=100, eos_token_id=tok.convert_tokens_to_ids("<|im_end|>"))
|
| 34 |
+
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
Use the chat template with `enable_thinking=False` (how it was trained), and stop
|
| 38 |
+
on `<|im_end|>`: the base model's own config only stops on `<|endoftext|>`.
|
| 39 |
+
|
| 40 |
+
## results
|
| 41 |
+
|
| 42 |
+
| | base model | this adapter |
|
| 43 |
+
| --- | --- | --- |
|
| 44 |
+
| validation loss (200 held-out chats) | 1.361 | 1.250 |
|
| 45 |
+
| instruction probes passed (greedy) | 15/19 | 18/19 |
|
| 46 |
+
| MMLU-Pro, direct letter, n=500 | 37.4% | 38.0% |
|
| 47 |
+
|
| 48 |
+
Probes are short prompts with checkable output constraints
|
| 49 |
+
([memorize/probes.py](https://github.com/AriSpiesberger/Memorize/blob/main/memorize/probes.py)). MMLU-Pro is
|
| 50 |
+
scored from the letter right after `ANSWER:` with no reasoning, on label-split C, split seed 0
|
| 51 |
+
([memorize/label_split.py](https://github.com/AriSpiesberger/Memorize/blob/main/memorize/label_split.py)).
|
| 52 |
+
|
| 53 |
+
## how it was made
|
| 54 |
+
|
| 55 |
+
- **Data**: 1,800 constraint-following chats from tulu-3-sft-personas-instruction-following
|
| 56 |
+
plus 600 Dolly-15k chats, 200 held out for validation, each at most 512 tokens;
|
| 57 |
+
multiple-choice and step-by-step items dropped. Seed 0.
|
| 58 |
+
- **LoRA**: rank 8, alpha 160 (mlx-lm's scale of 20), no dropout, on every linear
|
| 59 |
+
layer of every block (attention, linear attention, MLP); 8.4M parameters.
|
| 60 |
+
- **Training**: 1 epoch, 300 steps of 8 chats, Adam at lr 1e-5 (20 warmup steps
|
| 61 |
+
from 1%, then cosine to 10%), gradient norm clipped at 20, bf16, loss on the
|
| 62 |
+
assistant reply only. Seed 0.
|
| 63 |
+
- **Environment**: Python 3.12.7, torch 2.11.0+cu128, transformers 5.18.0, peft 0.21.2, datasets 5.1.0, one RTX 4090 (Windows 11).
|
| 64 |
+
|
| 65 |
+
`sft_torch.json` (in this repo) has every argument and the validation curve;
|
| 66 |
+
`train.log` is the full training log.
|
| 67 |
+
|
| 68 |
+
## reproduce
|
| 69 |
+
|
| 70 |
+
```bash
|
| 71 |
+
git clone https://github.com/AriSpiesberger/Memorize && cd Memorize
|
| 72 |
+
git checkout aad35a9 # the code that trained it
|
| 73 |
+
python -m venv .venv && .venv/bin/pip install -r requirements.txt # Windows: .venv\Scripts\pip
|
| 74 |
+
python -m memorize.sft_data # data/sft/train.jsonl, valid.jsonl
|
| 75 |
+
python -m memorize.sft_torch --out adapters/instruct
|
| 76 |
```
|
| 77 |
|
| 78 |
+
The data files should hash to sha256 `92234522d2e41621…` (train) and
|
| 79 |
+
`fdcbf7bca4450523…` (valid) if the upstream datasets are unchanged.
|