Text Generation
PEFT
Safetensors
lora
qwen3.5
instruction-tuning

instruct

Instruction-tuned Qwen3.5-2B-Base: a rank-8 LoRA trained with memorize.sft_torch (the sft-run2 recipe) on the memorize.sft_data mix. Start point for the label-split experiment.

From Memorize, a small lab for poking at Qwen3.5-2B.

use

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-2B-Base")
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-2B-Base", dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "Arisp/memorize-instruct")

ids = tok.apply_chat_template(
    [{"role": "user", "content": "Name three fruits."}],
    add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True,
)["input_ids"]
out = model.generate(ids, max_new_tokens=100, eos_token_id=tok.convert_tokens_to_ids("<|im_end|>"))
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Use the chat template with enable_thinking=False (how it was trained), and stop on <|im_end|>: the base model's own config only stops on <|endoftext|>.

results

base model this adapter
validation loss (200 held-out chats) 1.361 1.250
instruction probes passed (greedy) 15/19 18/19
MMLU-Pro, direct letter, n=500 37.4% 38.0%

Probes are short prompts with checkable output constraints (memorize/probes.py). MMLU-Pro is scored from the letter right after ANSWER: with no reasoning, on label-split C, split seed 0 (memorize/label_split.py).

how it was made

  • Data: 1,800 constraint-following chats from tulu-3-sft-personas-instruction-following plus 600 Dolly-15k chats, 200 held out for validation, each at most 512 tokens; multiple-choice and step-by-step items dropped. Seed 0.
  • LoRA: rank 8, alpha 160 (mlx-lm's scale of 20), no dropout, on every linear layer of every block (attention, linear attention, MLP); 8.4M parameters.
  • Training: 1 epoch, 300 steps of 8 chats, Adam at lr 1e-5 (20 warmup steps from 1%, then cosine to 10%), gradient norm clipped at 20, bf16, loss on the assistant reply only. Seed 0.
  • Environment: Python 3.12.7, torch 2.11.0+cu128, transformers 5.18.0, peft 0.21.2, datasets 5.1.0, one RTX 4090 (Windows 11).

sft_torch.json (in this repo) has every argument and the validation curve; train.log is the full training log.

reproduce

git clone https://github.com/AriSpiesberger/Memorize && cd Memorize
git checkout aad35a9        # the code that trained it
python -m venv .venv && .venv/bin/pip install -r requirements.txt   # Windows: .venv\Scripts\pip
python -m memorize.sft_data           # data/sft/train.jsonl, valid.jsonl
python -m memorize.sft_torch --out adapters/instruct

The data files should hash to sha256 92234522d2e41621… (train) and fdcbf7bca4450523… (valid) if the upstream datasets are unchanged.

Downloads last month
11
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Arisp/memorize-instruct

Adapter
(47)
this model

Datasets used to train Arisp/memorize-instruct