Instructions to use Arisp/memorize-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Arisp/memorize-instruct with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-2B-Base") model = PeftModel.from_pretrained(base_model, "Arisp/memorize-instruct") - Notebooks
- Google Colab
- Kaggle
Update model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
base_model: Qwen/Qwen3.5-2B-Base
|
| 4 |
+
library_name: peft
|
| 5 |
+
tags: [lora, qwen3.5, mmlu-pro, memorization]
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# memorize adapters
|
| 9 |
+
|
| 10 |
+
LoRA adapters from [Memorize](https://github.com/AriSpiesberger/Memorize), a small
|
| 11 |
+
lab for poking at Qwen3.5-2B. Every adapter sits on `Qwen/Qwen3.5-2B-Base`, in its
|
| 12 |
+
own folder of this repo. `mlx` adapters load with mlx-lm on Apple Silicon (and convert
|
| 13 |
+
to PyTorch with `python -m memorize.mlx_to_peft`); `peft` adapters load with PEFT.
|
| 14 |
+
|
| 15 |
+
| adapter | format | base | what it is |
|
| 16 |
+
| --- | --- | --- | --- |
|
| 17 |
+
| `sft-run2-torch` | peft | Qwen/Qwen3.5-2B-Base | Instruction SFT: the sft-run2 recipe (memorize.sft_torch) on the data from memorize.sft_data. 18/19 instruction probes, 38.0% direct-answer MMLU-Pro (500 held-out). Default start for run_split.py. |
|
| 18 |
+
| `sft-run2` | mlx | Qwen/Qwen3.5-2B-Base | Instruction SFT with memorize.sft, 1 epoch, batch 8, lr 1e-5. The starting point for the GRPO experiment. |
|
| 19 |
+
| `sft-run1` | mlx | Qwen/Qwen3.5-2B-Base | First instruction SFT with memorize.sft, 2 epochs, batch 1 x 4 accumulation, lr 1e-5. |
|
| 20 |
+
| `sft-run1-epoch1` | mlx | Qwen/Qwen3.5-2B-Base | sft-run1 after its first epoch: 48.7% zero-shot chat-CoT MMLU-Pro, 91.3% strict answer format (results/sft-run1-epoch1-chat-cot). |
|
| 21 |
+
| `base-if-lora` | mlx | Qwen/Qwen3.5-2B-Base | Early experiment: mlx_lm.lora on instruction-following chats, continued from base-dolly-lora (2400 iters, lr 1e-4). |
|
| 22 |
+
| `base-dolly-lora` | mlx | Qwen/Qwen3.5-2B-Base | Early experiment: mlx_lm.lora on Dolly-15k chats (3000 iters, lr 1e-4). Chats fine, but direct-answer MMLU-Pro drops to chance (9%), so it is not used as a starting point. |
|
| 23 |
+
|
| 24 |
+
```python
|
| 25 |
+
from huggingface_hub import snapshot_download
|
| 26 |
+
snapshot_download("Arisp/memorize-adapters", allow_patterns=["sft-run2-torch/*"], local_dir="adapters")
|
| 27 |
+
|
| 28 |
+
# or, for a peft adapter, straight from the hub:
|
| 29 |
+
from peft import PeftModel
|
| 30 |
+
model = PeftModel.from_pretrained(base_model, "Arisp/memorize-adapters", subfolder="sft-run2-torch")
|
| 31 |
+
```
|
| 32 |
+
|
| 33 |
+
In the Memorize repo: `python -m memorize.hub pull <name>` (or `--all`).
|