| --- |
| language: |
| - en |
| license: apache-2.0 |
| base_model: Qwen/Qwen3-4B |
| base_model_relation: adapter |
| library_name: peft |
| pipeline_tag: text-generation |
| tags: |
| - collabllm |
| - qwen3 |
| - lora |
| - sft |
| - conversational |
| --- |
| |
| # CollabLLM Qwen3-4B SFT — Math |
|
|
| Task-specific supervised fine-tuning **LoRA adapter** for [Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B), trained on the original CollabLLM interactive math-hard data. This is an independent reproduction artifact, not an official checkpoint from the CollabLLM authors. Base weights are downloaded separately when loading the adapter. |
|
|
| ## Training |
|
|
| - Base revision: `1cfa9a7208912126459214e8b04321603b3df60c`. |
| - Data: [interactive-math-hard](https://github.com/Wuyxin/collabllm/tree/d573e4949abdc301cb2b630525c593cc432e94b8/data/interactive-math-hard), from the original authors' archived repository. |
| - One SFT conversation per dialogue ID: select the longest prompt and append its chosen assistant response. Split: 450 training / 50 validation conversations, seed 42. |
| - Three epochs, 24 optimizer steps, effective batch size 64; learning rate 1e-5, warmup ratio 0.1. |
| - Assistant-only loss. Maximum sequence length: 1536 tokens. |
| - LoRA rank 32, alpha 16, dropout 0.1; attention and MLP projection layers. |
| - Non-thinking mode. The included tokenizer chat template fixes `enable_thinking = false`. |
|
|
| Exact hyperparameters, training/validation metrics and the adapter SHA-256 are in `training_metadata.json`. The original assistant system prompt is included as `system_prompt.txt` (attributed to CollabLLM under the accompanying `COLLABLLM_LICENSE`). |
|
|
| ## Load |
|
|
| Install compatible releases of `torch`, `transformers`, `peft`, `accelerate` and `huggingface_hub` with Qwen3 support. |
|
|
| ```python |
| import torch |
| from pathlib import Path |
| from huggingface_hub import hf_hub_download |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| |
| adapter_id = "ZachW/CollabLLM-Qwen3-4B-SFT-Math" |
| tokenizer = AutoTokenizer.from_pretrained(adapter_id) |
| base = AutoModelForCausalLM.from_pretrained( |
| "Qwen/Qwen3-4B", |
| revision="1cfa9a7208912126459214e8b04321603b3df60c", |
| torch_dtype=torch.bfloat16, |
| device_map="auto", |
| ) |
| model = PeftModel.from_pretrained(base, adapter_id).eval() |
| system_prompt = Path(hf_hub_download(adapter_id, "system_prompt.txt")).read_text() |
| messages = [ |
| {"role": "system", "content": system_prompt}, |
| {"role": "user", "content": "Can you help me with a task?"}, |
| ] |
| text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
| inputs = tokenizer(text, return_tensors="pt").to(model.device) |
| with torch.inference_mode(): |
| output = model.generate(**inputs, max_new_tokens=256, do_sample=False) |
| print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| ## Scope and attribution |
|
|
| This release contains the final SFT adapter and tokenizer, not GRPO, PPO, or DPO weights. It uses Qwen3-4B, so it does not establish reproduction of the original paper's Table 1 results. No benchmark score is claimed by this model card. The model can produce incorrect responses and should be evaluated for its intended application. |
|
|
| Method and source data: [CollabLLM paper](https://arxiv.org/abs/2502.00640) and [original code](https://github.com/Wuyxin/collabllm/tree/d573e4949abdc301cb2b630525c593cc432e94b8). Base model: [Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B), Apache 2.0. The source data remain subject to their original terms. The included CollabLLM prompt retains its original MIT notice. |
|
|
