---
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.5-0.8B-Base/blob/main/LICENSE
base_model: Qwen/Qwen3.5-0.8B-Base
library_name: transformers
pipeline_tag: text-generation
language:
- en
tags:
- weight-space-modeling
- zero-gradient
- math
- pedagogy
- tutor
- qwen3_5
model-index:
- name: Qwen3.5-0.8B-Math-Tutor
results:
- task:
type: text-generation
name: Zero-Shot Math Tutoring (Plain + ChatML Held-Out)
dataset:
name: Carbon-MathTutor-HeldOut-180
type: custom
split: test
metrics:
- type: accuracy
value: 82.78
name: Overall Math Accuracy (%)
- type: accuracy
value: 80.00
name: Plain Q&A Math Accuracy (%)
- type: accuracy
value: 85.56
name: ChatML Math Accuracy (%)
- type: accuracy
value: 54.44
name: Verified Pedagogical Teaching (%)
- type: accuracy
value: 96.23
name: Intermediate Equation Soundness (%)
---
**`Qwen3.5-0.8B-Math-Tutor`** is a drop-in 24-layer `Qwen3.5-0.8B` pedagogical math tutor created strictly by editing **`292,600` (`0.039%`) static weights** of [`Qwen/Qwen3.5-0.8B-Base`](https://huggingface.co/Qwen/Qwen3.5-0.8B-Base) — with **100% zero gradients (`torch.set_grad_enabled(False)`), zero optimizer steps, and zero fitted activations**.
Whereas untouched `Qwen3.5-0.8B-Base` collapses into bare 1-line wrong guesses (`13.33%` accuracy, `1.11%` verified teaching) on zero-shot `"Question: ...\nAnswer:"` prompts, `Qwen3.5-0.8B-Math-Tutor` derives solutions step by step across **both** plain Q&A (`80.00%` accuracy = **`6.0×`** Base) and native `ChatML` (`85.56%`), while shifting general `FineWeb-Edu` causal NLL by just **`+0.0030` nats** (`2.6202 -> 2.6232`).
> Need exact 256-bit bignum arithmetic, CORDIC scientific functions, definite integrals, and function graphing compiled directly into the model's layers? See companion checkpoint **[`DJLougen/Qwen3.5-0.8B-Math-GraphCalc`](https://huggingface.co/DJLougen/Qwen3.5-0.8B-Math-GraphCalc)**.
---
## Example Output — Untouched Base vs. Weight-Space Math Tutor
> **Zero-Shot Plain Prompt:**
> ```text
> Question: Solve for x: 7x + 9 = 3x + 41.
> Answer:
> ```
- **Untouched `Qwen/Qwen3.5-0.8B-Base`** (`0` weights edited — bare 1-line wrong guess):
```text
x = 10
```
- **`Qwen3.5-0.8B-Math-Tutor`** (`292,600` static weights edited, `0` gradients — verbatim greedy output):
```text
We need to solve the linear equation: 7x + 9 = 3x + 41.
We can solve by isolating x.
Subtract 3x from both sides: 7x - 3x + 9 = 41 => 4x + 9 = 41.
Subtract 9 from both sides: 4x = 41 - 9 = 32.
Divide both sides by 4: x = 32/4 = 8.
Thus x = 8.
We can also check: 7*8 + 9 = 56 + 9 = 65; 3*8 + 41 = 24 + 41 = 65. Works.
```
---
## Held-Out Benchmark (`N = 180` Unseen Problems × 2 Formats = `360` Generations)
Evaluated on `180` strictly disjoint problems (`30` each in `arithmetic`, `fractions_pct`, `algebra`, `geometry`, `gsm8k`, and `math_comp`, zero overlap with probe pool) plus `256` `FineWeb-Edu` 512-token blocks:
| Arm | Changed Weights | Math Acc (`plain` / `chatml`) | Verified Teaching (`plain` / `chatml`) | Eq Soundness | Teach Score | FineWeb-Edu NLL |
|---|---:|---:|---:|---:|---:|---:|
| `Qwen/Qwen3.5-0.8B-Base` (`A0`) | `0` | `50.83%` (`13.33%` / `88.33%`) | `35.83%` (`1.11%` / `70.56%`) | `97.04%` | `0.6039` | `2.6202` |
| `E1` Turn Bridge Only (`A1`) | `3,068` (`0.0004%`) | `80.83%` (`80.00%` / `81.67%`) | `51.11%` (`43.33%` / `58.89%`) | `95.27%` | `0.8833` | `2.6204` |
| Zero-Leak Pareto (`A2`) | `623,927` (`0.083%`) | `78.06%` (`71.67%` / `84.44%`) | **`58.61%`** (**`53.89%`** / `63.33%`) | **`97.19%`** | **`0.8978`** | `2.6232` |
| **`Qwen3.5-0.8B-Math-Tutor` (`A3`)** | **`292,600` (`0.039%`)** | **`82.78%` (`80.00%` / `85.56%`)** | `54.44%` (`42.22%` / **`66.67%`**) | `96.23%` | `0.8888` | `2.6232` |
---
## How It Works (Zero Gradients)
Every edit is a closed-form algebraic function of the pretrained static weight tensors and BPE token IDs:
1. **`E1` — Closed-Form Turn-Embedding Bridge (`3,068` scalars)**:
- In Qwen's BPE vocabulary, digits `'0'..'9'` carry no leading space while English openers (`' We'`, `' Let'`, `' Step'`) do — causing bare `' '` + 1-line guesses to win right after `Answer:`.
- Re-anchoring `'Question'` (`14162`) and `'Answer'` (`15666`) in `embed_tokens.weight` toward Qwen3.5's latent mid-trained tutor circuit (`<|im_start|>`, `user`, `assistant`, ``, ``, `Step`/`Solution`) at `plain_norm_scale = 0.72` respects the Layer-0 un-normalized residual skip connection vs. tied `lm_head` self-echo threshold, jumping `plain` math accuracy **`6.0×`** (`13.33% -> 80.00%`) with `+0.0002` nats LM shift.
2. **`E2` — Anti-Pedagogical Late-MLP Suppression (`L14..L23`)**:
- Damps specialist worksheet-continuation neurons (`"Question 2:"`), multi-newline collapse neurons, bare multiple-choice guess neurons, and a 6D CJK exam-leakage subspace orthogonalized against English math vocabulary.
3. **`E3` — Pedagogical Step-Starter & Equation-Writer Boost (`L13..L23`)**:
- Boosts 32 specialist MLP neurons writing English step initiators, deductive connectors (`"therefore"`, `"thus"`, `"since"`), and explicit intermediate equations, plus turn-boundary `gate_proj`/`up_proj` key routing on 9 core step starters.
---
## Quick Start (`transformers`)
`Qwen3.5-0.8B-Math-Tutor` preserves exact architectural parity with `Qwen/Qwen3.5-0.8B-Base` (`tie_word_embeddings = True`, standard 24-layer `Qwen3_5ForConditionalGeneration`, compatible with HuggingFace `transformers`, `vLLM`, and `SGLang`):
```python
import torch
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
ckpt = "DJLougen/Qwen3.5-0.8B-Math-Tutor"
tok = AutoTokenizer.from_pretrained(ckpt)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
ckpt,
dtype=torch.bfloat16,
device_map="auto",
).eval()
# Works zero-shot with plain "Question: ...\nAnswer:" OR ChatML
prompt = "Question: Solve for x: 7x + 9 = 3x + 41.\nAnswer:"
inputs = tok(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```
---
## Honest Limitations
1. **Mental-Math Digit Ceiling**: Without compiled tool layers, a `0.8B` model still fails 8–10 digit long multiplication (`20%` accuracy) and cannot plot coordinate graphs. Use **[`DJLougen/Qwen3.5-0.8B-Math-GraphCalc`](https://huggingface.co/DJLougen/Qwen3.5-0.8B-Math-GraphCalc)** when exact bignum/rational/CORDIC/graphing execution is required.
2. **Tied-Embedding Scratchpad vs. Clean-Stop Tradeoff**: Preserving `tie_word_embeddings = True` creates a slight Pareto tradeoff between maximum scratchpad self-verification (`82.78%` `math_acc`) and occasional literal `` emissions when `skip_special_tokens=False`.
---
## License
Apache-2.0 (inherited from [`Qwen/Qwen3.5-0.8B-Base`](https://huggingface.co/Qwen/Qwen3.5-0.8B-Base)).