---
language:
- en
license: apache-2.0
base_model: Qwen/Qwen3-4B
base_model_relation: adapter
library_name: peft
pipeline_tag: text-generation
tags:
- collabllm
- qwen3
- lora
- sft
- conversational
---

# CollabLLM Qwen3-4B SFT — Code

Task-specific supervised fine-tuning **LoRA adapter** for [Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B), trained on the original CollabLLM interactive bigcodebench data. This is an independent reproduction artifact, not an official checkpoint from the CollabLLM authors. Base weights are downloaded separately when loading the adapter.

## Training

- Base revision: `1cfa9a7208912126459214e8b04321603b3df60c`.
- Data: [interactive-bigcodebench](https://github.com/Wuyxin/collabllm/tree/d573e4949abdc301cb2b630525c593cc432e94b8/data/interactive-bigcodebench), from the original authors' archived repository.
- One SFT conversation per dialogue ID: select the longest prompt and append its chosen assistant response. Split: 450 training / 50 validation conversations, seed 42.
- Three epochs, 24 optimizer steps, effective batch size 64; learning rate 1e-5, warmup ratio 0.1.
- Assistant-only loss. Maximum sequence length: 1536 tokens.
- LoRA rank 32, alpha 16, dropout 0.1; attention and MLP projection layers.
- Non-thinking mode. The included tokenizer chat template fixes `enable_thinking = false`.

Exact hyperparameters, training/validation metrics and the adapter SHA-256 are in `training_metadata.json`. The original assistant system prompt is included as `system_prompt.txt` (attributed to CollabLLM under the accompanying `COLLABLLM_LICENSE`).

## Load

Install compatible releases of `torch`, `transformers`, `peft`, `accelerate` and `huggingface_hub` with Qwen3 support.

```python
import torch
from pathlib import Path
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

adapter_id = "ZachW/CollabLLM-Qwen3-4B-SFT-Code"
tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B",
    revision="1cfa9a7208912126459214e8b04321603b3df60c",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id).eval()
system_prompt = Path(hf_hub_download(adapter_id, "system_prompt.txt")).read_text()
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Can you help me with a task?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

## Scope and attribution

This release contains the final SFT adapter and tokenizer, not GRPO, PPO, or DPO weights. It uses Qwen3-4B, so it does not establish reproduction of the original paper's Table 1 results. No benchmark score is claimed by this model card. The model can produce incorrect responses and should be evaluated for its intended application.

Method and source data: [CollabLLM paper](https://arxiv.org/abs/2502.00640) and [original code](https://github.com/Wuyxin/collabllm/tree/d573e4949abdc301cb2b630525c593cc432e94b8). Base model: [Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B), Apache 2.0. The source data remain subject to their original terms. The included CollabLLM prompt retains its original MIT notice.