Blame for README.md

---
base_model: alan13367/Qwen3.5-4B-RustCoder
base_model_relation: quantized
library_name: mlx
license: apache-2.0
pipeline_tag: text-generation
language:
  - en
tags:
  - mlx
  - qwen3.5
  - rust
  - code
  - lora
  - 8bit
---

# Qwen3.5-4B-RustCoder-MLX-8bit

[Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) fine-tuned to write Rust that compiles, passes tests and is clippy-clean. It keeps its thinking mode. This is the MLX checkpoint for Apple Silicon, 8-bit (group size 64), with the LoRA merged into the weights of [mlx-community/Qwen3.5-4B-MLX-8bit](https://huggingface.co/mlx-community/Qwen3.5-4B-MLX-8bit). It is the checkpoint that was trained and evaluated.

For transformers and other non-MLX tools, use the bf16 safetensors version: [alan13367/Qwen3.5-4B-RustCoder](https://huggingface.co/alan13367/Qwen3.5-4B-RustCoder).

Every training answer was checked by a gate that builds the code, runs hidden tests and lints it. Answers that failed it were not used.

## Results

Pass rate: the share of answers whose code passes the task's hidden tests. Each task was sampled 4 times, with thinking on and no system prompt. The test tiers were never trained on.

| Pass rate | Easy (29 tasks) | Hard (40) | Quality (29) | Dev (82) |
|---|---|---|---|---|
| Qwen3.5-4B (8-bit) | 41.4% | 7.5% | 8.6% | 20.7% |
| **Qwen3.5-4B-RustCoder** | **51.7%** | **8.8%** | **17.2%** | **22.9%** |

- Pooled over all 180 tasks: +4.3 points over the base (95% CI +0.6 to +8.2) and +0.046 graded reward (CI +0.012 to +0.081).
- The tiers are small. A single tier only detects a gain of about 9 points (hard, quality) or 14 points (easy), so only the pooled gain is established.
- Quality covers fixing bugs, refactoring, writing tests and new domains. A refactor counts only if it is also lint-clean. Dev is the tier used to pick checkpoints.
- At least 1 of the 4 attempts passes on 82.8% of easy tasks and 22.5% of hard tasks.

General ability, against the base (subsets of each benchmark):

| | MMLU (200) | ARC-Challenge (150) | GSM8K (100) | HellaSwag (150) |
|---|---|---|---|---|
| Qwen3.5-4B (8-bit) | 69.5% | 89.3% | 92.0% | 80.7% |
| Qwen3.5-4B-RustCoder | 69.0% | 90.0% | 92.0% | 82.0% |

On general text, its token distribution stays close to the base: KL 0.026 nats/token and 94.2% top-1 agreement.

## Usage

With [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) (the checkpoint keeps the base's Qwen3.5 layout, vision tower included):

```bash
pip install -U mlx-vlm
python -m mlx_vlm.generate --model alan13367/Qwen3.5-4B-RustCoder-MLX-8bit --enable-thinking \
  --max-tokens 4096 --temperature 0.6 --top-p 0.95 --top-k 20 \
  --prompt "Write in Rust: pub fn median(xs: &mut [i64]) -> Option<f64>"
```

```python
from mlx_vlm import load, generate

model, processor = load("alan13367/Qwen3.5-4B-RustCoder-MLX-8bit")
messages = [{"role": "user", "content": "Write in Rust: pub fn median(xs: &mut [i64]) -> Option<f64>"}]
prompt = processor.tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
out = generate(model, processor, prompt, max_tokens=4096, temperature=0.6, top_p=0.95, top_k=20)
print(out.text)
```

Recommended:

- **Use no system prompt.** Training and evaluation used none. A long style prompt sent the base model into runaway reasoning.
- **Leave thinking on and cap the output at about 4096 tokens.** Answers that finish take a median of about 1.2k tokens. An attempt that runs to the cap is usually stuck in a reasoning loop, so resampling helps more than raising the cap.
- **Use temperature 0.6, top-p 0.95 and top-k 20.** These are the settings behind the numbers above.

## Training

LoRA SFT, one epoch. Rank 16, alpha 32, dropout 0.05, on the last 16 decoder layers. Learning rate 1e-4, batches of up to 4096 tokens. In total 739 samples, 784k trained tokens, 402 steps. Validation loss was 0.411.

The data is mostly the model's own work, filtered by the gate:

| Source | Records | What it is |
|---|---|---|
| Own passing attempts | 238 | The base model's own attempts at the training tasks, kept only if they pass the hidden tests and the test lints |
| Self-repairs | 277 | The base model fixing its own failed attempts, given the compiler or test output as the next user turn, kept only if the fix passes |
| Teacher traces | 175 | Passing traces from DeepSeek V4.1 Flash. 159 of them repair the 4B's own failed attempts. Only traces within the 4B's own length and style were kept (completion of at most 2048 tokens, little second-guessing) |
| Replay | 88 | The base model's own answers to general, non-Rust prompts, to keep its general ability |

What the experiments behind this checkpoint showed:

- **The model learns best from its own style.** Reference solutions with model-written reasoning made it worse (easy 41.4% → 26.7%). It imitated their dense style and compiled less often.
- **Unfiltered teacher traces also made it worse** (easy 34.5%). Their long, second-guessing reasoning taught it to never leave thinking: 177 of 720 answers ran to the cap.
- **Filtering the teacher traces fixed that.** This checkpoint has 36 of 720 answers that run to the cap, against 82 for the base.
- **The loss covers only the final reply.** In repair records, the failed attempt and the feedback are context, not training targets.

## Limitations

- It is a 4B model and Rust is hard for it. Many of its answers still fail to compile (about half did for the base model and the previous fine-tune), mostly with type, trait-bound and method errors in its own code (E0308, E0277, E0599). Harder multi-part tasks rarely pass (hard tier 8.8%). It does much better with a compiler in the loop: given the error output it fixes a share of its failures, and the best of 4 attempts passes far more often than a single one.
- It still over-thinks sometimes. Cap the output, and resample when an answer gets stuck.
- Only the text side was trained. The vision tower is the base's, unchanged and untested here.
- The evaluation sets are small (29 to 82 tasks per tier), so each per-tier number has a wide margin. Treat the gains as modest.
- It was trained and evaluated on standalone tasks: a function, a type or a small module checked by hidden tests, plus bug fixes, refactors and test-writing. It has not been measured on multi-file projects or long-context editing.

## License

Apache 2.0, the license of the base model [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B).