Results
CPU (2 cores), WikiText-2 test, 32 windows of 1024 tokens, one run per setting. Final package defaults: shaped coding with λ = 1 and κ = 1 (nothing tuned) and dynamic group grids. RTN uses the same GPTQ grid.
| model | group | dense | RTN INT4 | shaped INT4 | RTN mixed | shaped mixed | mixed bits/w |
|---|---|---|---|---|---|---|---|
| GPT-2 124M | 128 | 26.84 | 29.91 | 29.14 | 29.62 | 28.86 | 4.75 |
| OPT-125m | 128 | 29.27 | 31.98 | 31.39 | 30.71 | 30.44 | 4.75 |
| SmolLM2-135M (Llama) | 64 | 15.46 | 19.28 | 19.29 | 17.82 | 17.83 | 5.0 |
Uniform INT4 is 4 + 32/group bits per quantized weight: 4.25 for groups of 128, 4.5 for groups of 64. "Mixed" is INT4/INT8/FP16 per module chosen by the type rule at the stated budget.
Change in perplexity from shaped coding vs RTN:
- GPT-2 124M: INT4 -2.6%, mixed -2.6%
- OPT-125m: INT4 -1.9%, mixed -0.9%
- SmolLM2-135M (Llama): INT4 +0.1%, mixed +0.1%
Static group grids (grids fixed from the original weights), for comparison:
- GPT-2 124M: shaped INT4 29.09, shaped mixed 28.82
- OPT-125m: shaped INT4 32.03, shaped mixed 31.02
- SmolLM2-135M (Llama): shaped INT4 19.19, shaped mixed 17.74
Exact per-layer output error vs RTN under the true input covariance (OPT-125m, layers 0/4/8, evaluation only): geometric-mean ratio 0.868 (data-free H), 0.282 (true covariance). The layer-level gain only partly reaches perplexity.
70B scaling
Estimates from the real configs (shapequant memory):
| model | params | device peak | host buffer | INT4 checkpoint | largest H |
|---|---|---|---|---|---|
| Llama-3.1-70B | 70.6B | 21.2 GB | 5.6 GB | 37.8 GB | 28672 |
| Qwen2.5-72B | 72.7B | 22.1 GB | 5.6 GB | 39.4 GB | 29568 |
| Llama-2-70B | 69.0B | 21.2 GB | 5.6 GB | 34.8 GB | 28672 |
Per-layer core at Llama-3.1-70B shapes, executed on cpu (2 cores): stacked q/k/v (10240×8192) 26 s, o_proj (8192×8192) 19 s, attention-output covariance 1.7 s. The MLP-sized steps (28672-wide covariance, Monte Carlo) need ~7 GB for two 28672² fp32 matrices and were not run on this 7.6 GB machine. 70B needs a GPU (≥ 24 GB, 40–80 GB comfortable). Work per layer is the same order as GPTQ's.