---
language:
- en
license: apache-2.0
base_model: Qwen/Qwen3-4B
base_model_relation: adapter
library_name: peft
pipeline_tag: text-generation
tags:
- collabllm
- qwen3
- lora
- sft
- conversational
---

# CollabLLM Qwen3-4B SFT — Math

Task-specific supervised fine-tuning **LoRA adapter** for [Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B), trained on the original CollabLLM interactive math-hard data. This is an independent reproduction artifact, not an official checkpoint from the CollabLLM authors. Base weights are downloaded separately when loading the adapter.

## Training

- Base revision: `1cfa9a7208912126459214e8b04321603b3df60c`.
- Data: [interactive-math-hard](https://github.com/Wuyxin/collabllm/tree/d573e4949abdc301cb2b630525c593cc432e94b8/data/interactive-math-hard), from the original authors' archived repository.
- One SFT conversation per dialogue ID: select the longest prompt and append its chosen assistant response. Split: 450 training / 50 validation conversations, seed 42.
- Three epochs, 24 optimizer steps, effective batch size 64; learning rate 1e-5, warmup ratio 0.1.
- Assistant-only loss. Maximum sequence length: 1536 tokens.
- LoRA rank 32, alpha 16, dropout 0.1; attention and MLP projection layers.
- Non-thinking mode. The included tokenizer chat template fixes `enable_thinking = false`.

Exact hyperparameters, training/validation metrics and the adapter SHA-256 are in `training_metadata.json`. The original assistant system prompt is included as `system_prompt.txt` (attributed to CollabLLM under the accompanying `COLLABLLM_LICENSE`).

## Load

Install compatible releases of `torch`, `transformers`, `peft`, `accelerate` and `huggingface_hub` with Qwen3 support.

```python
import torch
from pathlib import Path
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

adapter_id = "ZachW/CollabLLM-Qwen3-4B-SFT-Math"
tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B",
    revision="1cfa9a7208912126459214e8b04321603b3df60c",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id).eval()
system_prompt = Path(hf_hub_download(adapter_id, "system_prompt.txt")).read_text()
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Can you help me with a task?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

## Scope and attribution

This release contains the final SFT adapter and tokenizer, not GRPO, PPO, or DPO weights. It uses Qwen3-4B, so it does not establish reproduction of the original paper's Table 1 results. No benchmark score is claimed by this model card. The model can produce incorrect responses and should be evaluated for its intended application.

Method and source data: [CollabLLM paper](https://arxiv.org/abs/2502.00640) and [original code](https://github.com/Wuyxin/collabllm/tree/d573e4949abdc301cb2b630525c593cc432e94b8). Base model: [Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B), Apache 2.0. The source data remain subject to their original terms. The included CollabLLM prompt retains its original MIT notice.