Bits fitted to each model by measurement. magi-1.2-2B: loop-DPO + 2B map, density 1.305. magi-1.2-4B: 4B map, density 0.968.
AI & ML interests
None defined yet.
Recent Activity
Organization Card
Paper: Where the Bits Go (Sun, 2026) · Recipe and data: GEHIRNS/magi · Website: gehirns.pages.dev
Capability per GiB. We study how much a language model can do inside a single 12 GB consumer GPU. Every model is scored on a fixed 12-suite benchmark (coding, reasoning, knowledge, tool calls) with thinking on, and the score is set against the GPU memory the model actually occupies at a 32K context. All numbers come from our own runs on one RTX 5070.
Latest releases
Every row was measured the same way on one RTX 5070: the same 2,313-task benchmark with thinking on (4,096-token budget, greedy), llama.cpp with a q4_0 KV cache, and memory at a 32K context from llama-fit-params. These figures are not comparable with the magi-1.0-4B table below, whose memory comes from Ollama.
| Model | Base | GiB at 32K | Score | Density | What it is | Status |
|---|---|---|---|---|---|---|
| magi-1.2-4B | Qwen3.5-4B | 2.55 | 81.90% | 0.968 | magi-1.1-4B's weights with bits fitted to the 4B by measurement. 64 MiB smaller at the same score; thinks 11% more | new |
| magi-1.1-4B | Qwen3.5-4B | 2.61 | 81.72% | 0.939 | magi-1.0-4B plus a LoRA that teaches it to finish its thinking (28% less thinking) | recommended (4B) |
| magi-1.2-2B | Qwen3.5-2B | 1.23 | 67.09% | 1.305 | magi-1.1-2B plus one round of training against repetition loops, with bits fitted to the 2B; 27% less thinking | recommended (2B) |
| magi-1.1-2B | Qwen3.5-2B | 1.36 | 66.06% | 1.149 | The think-less LoRA on Qwen3.5-2B with the magi-1.0 recipe | |
| magi-1.0-2B | Qwen3.5-2B | 1.36 | 63.48% | 1.071 | The magi-1.0 recipe on Qwen3.5-2B, no training |
magi-1.0-4B
GEHIRNS/magi-1.0-4B (magi-1.0 in the paper) is Qwen3.5-4B in 1.80 GiB, with no training. Each tensor class is stored at the precision its measured sensitivity calls for, using only standard GGUF types, so the file runs in stock llama.cpp and Ollama. With an 8-bit KV cache (magi-1.0-4B + q8 KV) it keeps 96% of the full model's benchmark score in 30% of the GPU memory and decodes 2.7× faster.
| Baselines | Ours | ||||
|---|---|---|---|---|---|
| Benchmark | BF16 8.98 GiB | Q4_K_M 3.78 GiB | Q2_K 3.15 GiB | magi-1.0-4B 3.02 GiB | magi-1.0-4B + q8 KV 2.65 GiB |
| Efficiency | |||||
| GPU memory at 32K (GiB) | 8.98 | 3.78 | 3.15 | 3.023.0× less | 2.653.4× less |
| Benchmark score | 83.96% | 84.14% | 77.06% | 81.30%97% | 80.64%96% |
| Capability density | 0.294 | 0.703 | 0.675 | 0.8002.7× | 0.8933.0× |
| Decode speed (tok/s) | 66.3 | 146.6 | 165.5 | 180.02.7× | 181.42.7× |
The benchmark score is the mean pass rate over the 12 suites. Capability density ρ is −log₂(1 − score) per GiB of GPU memory, that is, how many times the error rate halves per gibibyte. Small numbers are relative to BF16. Q4_K_M is the more accurate choice; magi-1.0-4B is the choice when memory is the constraint. The newer releases above build on it.
Models
| Model | What it is | Status |
|---|---|---|
| magi-1.2-4B | magi-1.1-4B's weights, bits fitted to the 4B by measurement | new |
| magi-1.1-4B | magi-1.0-4B plus a think-less LoRA | recommended (4B) |
| magi-1.0-4B | Qwen3.5-4B, sensitivity-allocated mixed precision, no training (the paper's model) | first release |
| magi-1.2-2B | magi-1.1-2B plus loop training, bits fitted to the 2B | recommended (2B) |
| magi-1.1-2B | Qwen3.5-2B with the think-less LoRA and the magi-1.0 recipe | 2B |
| magi-1.0-2B | The magi-1.0 recipe on Qwen3.5-2B, no training | 2B |
| magi-beta-1 | magi-1.0-4B with the full-attention k, v and output projections at Q6_K | for comparison |
| magi-alpha-1, magi-alpha-2 | Qwen3.5-9B with supervised fine-tuning | retired, not recommended |
The alphas are an earlier, separate experiment. The fine-tuning did not improve the base model's coding (LiveCodeBench 49 and 48 of 150, against 50 for the untrained base, within run-to-run variation), and magi-alpha-1 can degenerate in chat. They stay online for transparency.
The paper Where the Bits Go: Sensitivity-Allocated Mixed Precision for a 4B Reasoning Model on a 12 GB Consumer GPU (Eason Sun, 2026; DOI: 10.5281/zenodo.23630199) describes the method and all measurements. The build recipe, the importance matrix and the data behind every table and figure are at GEHIRNS/magi on GitHub.
models 10
GEHIRNS/magi-1.2-4B
Text Generation • 4B • Updated
GEHIRNS/magi-1.2-2B
Text Generation • 2B • Updated • 27 • 1
GEHIRNS/magi-1.0-2B
Text Generation • 2B • Updated • 536
GEHIRNS/magi-1.1-2B
Text Generation • 2B • Updated • 738 • 1
GEHIRNS/magi-1.1-4B
Text Generation • 4B • Updated • 360 • 1
GEHIRNS/magi-1.0-4B
Text Generation • 4B • Updated • 1.46k • 5
GEHIRNS/seele-1.0-3.3B
Text Classification • 3B • Updated • 593 • 1
GEHIRNS/magi-alpha-2
9B • Updated • 877
GEHIRNS/magi-alpha-1
9B • Updated • 888
GEHIRNS/magi-beta-1
Text Generation • 4B • Updated • 1.38k • 2
datasets 0
None public yet