GEHIRNS/magi-1.2-4B

---

Introduction

magi-1.2-4B is Qwen3.5-4B in 2.55 GiB at a 32K context, with capability density ρ = 0.968, against 0.939 for magi-1.1-4B and 0.278 for the BF16 model. On our 2,313-task benchmark it scores 81.9% (81.7% for magi-1.1-4B).

It has the same weights as magi-1.1-4B, which is Qwen3.5-4B with a LoRA that teaches it to finish its thinking. What changes is how the weights are quantized. magi-1.1-4B uses the magi-1.0 rules. Here the mixed-precision map was fitted to this model by measurement:

1. Importance matrix from the model's own reasoning. It was computed from 493 of Qwen3.5-4B's own correct, finished reasoning traces on training-split prompts. magi-1.0 used 32 code traces. This alone cut the quantized model's divergence from the BF16 model by 4%.
2. Per-layer sensitivity, measured. Each of 153 tensor groups (per layer: FFN, DeltaNet projections, attention) was moved one quantization step up and one down: 306 builds in all. Each build was scored by its KL divergence from the BF16 model on held-out reasoning and chat text. Bits were then moved to where the measurements say they matter most, and the 64 MiB of the least sensitive layers were stepped down (69 groups changed).

Against magi-1.1-4B on the same 2,313 tasks (paired): +0.18 points (standard error 0.62; 103 tasks gained, 100 lost), in 64 MiB less memory. The trade-off: it thinks 11% more (1,770 tokens per task on average against 1,597).

Capability density

Capability density is ρ = −log₂(1 − score) / GiB: how many times the benchmark error rate is halved per gibibyte of memory. Memory for all three models is measured the same way: llama.cpp with a q4_0 KV cache at a 32K context, from llama-fit-params. The magi-1.0-4B card's chart used Ollama's memory figures, so its numbers are not comparable with these.

Benchmarks

Every column was run on one RTX 5070 (12 GB) under one protocol:

• Tasks: 2,313, run with thinking on and a 4,096-token thinking budget (budget forcing), greedy decoding.
• Server: the magi columns on llama.cpp b11800 (llama-server) with a q4_0 KV cache, flash attention and the default micro-batch. The BF16 column is our earlier Ollama run of the same tasks; on magi-1.0-4B the two runtimes differ by about 0.15 points.
• Code grading: by executing tests (EvalPlus for HumanEval+ and MBPP+). LiveCodeBench uses 150 problems from contests between 2025-01-18 and 2025-04-06.

Memory. Memory is the resident size llama-fit-params projects for a 32K context with one sequence and micro-batch 128: weights, KV cache, recurrent state and compute buffers, GPU plus host, without the CUDA context.

Benchmark	Qwen3.5-4B (BF16) 9.51 GiB	magi-1.1-4B 2.61 GiB	magi-1.2-4B 2.55 GiB
Efficiency
Memory at 32K (GiB)	9.51	2.61	2.55
Benchmark score	83.96%	81.72%	81.90%
Capability density ρ	0.278	0.939	0.968
Decode speed (tok/s)	66	186	186
Mean thinking tokens	2,222	1,597	1,770
Tasks hitting the thinking budget	31.6%	18.3%	22.2%
Traces ending in a loop	5.7%	4.4%	5.8%
Coding
HumanEval+	91.46	82.93	86.59
MBPP+	72.18	69.42	68.92
LiveCodeBench	35.33	32.67	30.00
Math
GSM8K	87.33	92.67	93.33
Science and Knowledge
ARC-Challenge	98.67	98.00	98.00
MMLU	84.50	81.50	80.50
OpenBookQA	95.00	95.50	94.50
SciQ	98.00	98.00	97.00
TruthfulQA	82.50	76.50	78.50
Commonsense
HellaSwag	75.50	69.50	71.00
WinoGrande	87.00	84.00	84.50
Tool Use
Function calling	100.00	100.00	100.00

magi-1.2-4B against BF16: −2.05 points (paired across the two runtimes, standard error 0.61; 72 tasks gained, 126 lost) in 27% of the memory.

Decode speed: single-stream generation speed (llama-bench, 128 tokens, q4_0 KV). Both magi models were measured back to back on the same GPU, BF16 in an earlier session.

Recipe

Per-tensor types, read from the file (Qwen3.5-4B: 32 layers, 24 Gated DeltaNet and 8 full attention):

Tensor group	Types (tensors)	Share of parameters	bits/weight
FFN gate, up, down (all 32 layers)	IQ3_XXS ×76, IQ4_XS ×15, IQ2_S ×5	53.9%	3.22
Token embedding = tied output head	Q4_K	15.1%	4.50
DeltaNet input projection attn_qkv (24)	IQ3_XXS ×21, Q4_K ×3	12.0%	3.24
Full-attention q, k, v, output (8 layers)	Q4_K ×19, IQ3_XXS ×7, Q5_K ×6	7.0%	4.12
Gated-attention output gate (24 DeltaNet layers)	IQ3_XXS ×14, IQ2_S ×6, IQ4_XS ×4	6.0%	3.14
DeltaNet output projection ssm_out (24)	IQ3_XXS ×13, Q4_K ×9, Q5_K ×2	6.0%	3.80
DeltaNet gates ssm_alpha, ssm_beta	Q8_0 ×48	<0.1%	8.50
Norms, convolution, A, Δt bias	F32 ×177	<0.1%	32.00

The quantizer is llama-quantize --pure from the PrismML-Eng/llama.cpp fork, with base type IQ2_XXS and a per-tensor override file. The starting point was the magi-1.0 rules; each measured move changed one group by one step. The file holds 4,205,751,296 parameters (1,765.5 MiB of tensor data, −64.4 MiB vs magi-1.1-4B). File SHA-256: a07bcbd10de5c2e710dab7e21c576f28fc49fa70e85ba89a35b43c7560537474.

Training

No new training. The weights are magi-1.1-4B's: Qwen3.5-4B plus a LoRA (rank 16, α 32, 2 epochs, QLoRA) trained on 360 of the model's own correct, short reasoning traces, merged before quantization. See the magi-1.1-4B card for the data and its licenses.

How to run

llama.cpp (the evaluated configuration; uses the Qwen3.5 chat template embedded in the file):

llama-server -m magi-1.2-4B.gguf -c 32768 --jinja --flash-attn on --cache-type-k q4_0 --cache-type-v q4_0

Ollama:

ollama create magi-1.2-4B -f Modelfile
ollama run magi-1.2-4B

The benchmark used greedy decoding. For interactive use with thinking enabled, Qwen recommends temperature 0.6, top_p 0.95 and top_k 20.

Limitations

• Selected on this benchmark. The map was chosen among several candidates by a 250-task screen of this benchmark, then confirmed on the full set. Read small score differences to magi-1.1-4B as ties.
• It thinks more than magi-1.1-4B: 1,770 thinking tokens per task on average against 1,597 (+11%), and 22% of tasks hit the thinking budget against 18%. The memory saving comes with somewhat longer answers. If answer length matters more to you than memory, magi-1.1-4B thinks less at a similar score.
• Still loops sometimes: 6% of greedy benchmark traces end in a repetition loop.
• The name is ours. "4B" follows Qwen's naming; the file holds 4,205,751,296 parameters.
• Narrow evaluation: one benchmark, one GPU, each configuration run once; the magi columns on llama.cpp b11800, the BF16 column on Ollama. Text only: the base model's vision encoder is not included.

Related

• GEHIRNS/magi-1.1-4B: the same weights with the magi-1.0 rules (the middle column above).
• GEHIRNS/magi-1.0-4B: the recipe, training-free.
• GEHIRNS/magi-1.2-2B: the same re-fit on Qwen3.5-2B, with one round of training against repetition loops.

Citation

The recipe is described in:

@misc{sun2026wherethebitsgo,
  title     = {Where the Bits Go: Sensitivity-Allocated Mixed Precision for a 4B Reasoning Model on a 12 GB Consumer GPU},
  author    = {Sun, Eason},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.23630199},
  url       = {https://doi.org/10.5281/zenodo.23630199}
}

License

The weights are Apache 2.0, inherited from Qwen/Qwen3.5-4B; the license text is included in this repository.