| --- |
| title: ICE for Ornith-1.5-35B-A3B — the concrete instantiation |
| created: 2026-08-21 |
| status: SHIPPED 2026-08-21 — 4 tiers live, uniform expert allocation |
| principle: see PRINCIPLE.md (model-agnostic) |
| --- |
| |
| # ICE — Ornith-1.5-35B-A3B |
|
|
| The general method is in [PRINCIPLE.md](PRINCIPLE.md). This file is what it |
| produced for *this* model, with the arithmetic shown. |
|
|
| ## Architecture inputs |
|
|
| | | | |
| |---|---| |
| | blocks | 41 = 30 linear/SSM + 10 full-attention (3,7,…,39) + 1 MTP (`blk.40`) | |
| | residual width | 2048 | |
| | experts | 256, 8 active → **sparsity ratio 32** | |
| | expert FFN / shared FFN | 512 / 512 | |
| | KV heads | 2, head_dim 256, on 10 blocks only → ~20 KB/token | |
| | vocab | 248,320 → `output.weight` = 508.6 M in one tensor | |
| |
| **Every block is 95.6% expert weights** (805.3 M of ~842 M). The surgical field is |
| the remaining ~35 M per block plus the two 508 M end-caps. |
| |
| ## Active-compute geography (measured) |
| |
| | role | params | % of file | % of *active* compute | |
| |---|---:|---:|---:| |
| | routed experts (8/256) | 32.21 B | 92.94% | 34.2% | |
| | attention | 1.03 B | 2.96% | **34.9%** | |
| | output head | 0.51 B | 1.47% | **17.3%** | |
| | ssm | 0.26 B | 0.74% | 8.7% | |
| | shared expert | 0.13 B | 0.36% | 4.3% | |
| | token_embd | 0.51 B | 1.47% | 0% (`get_rows`) | |
|
|
| **65.8% of active parameters are non-routed.** Active total 2.95 B — which is what |
| the "A3B" label means, and a useful check that the accounting is right. |
|
|
| ## The assignment |
|
|
| | tensor | × | params | class | type | |
| |---|---:|---:|---|---| |
| | `ffn_gate_inp`, `ffn_gate_inp_shexp` | 80 | 21.0 M | DISCRETE | F32 † | |
| | `ssm_conv1d` | 30 | 1.0 M | RECURRENT | F32 † | |
| | `ssm_alpha`, `ssm_beta` | 60 | 3.9 M | RECURRENT | **F32 (pinned)** | |
| | `ssm_a`, `ssm_dt`, all norms | — | 0.3 M | RECURRENT / — | F32 † | |
| | `attn_k`, `attn_v` | 20 | 21.0 M | **CACHED** | **F16** | |
| | `attn_q` | 10 | 167.8 M | INSTANT | Q8_0 | |
| | `attn_output` | 10 | 83.9 M | INSTANT | Q8_0 | |
| | `attn_qkv` | 30 | 503.3 M | INSTANT | Q8_0 | |
| | `attn_gate` | 30 | 251.7 M | INSTANT | Q8_0 | |
| | `ssm_out` | 30 | 251.7 M | INSTANT | Q8_0 | |
| | `ffn_*_shexp` | 120 | 125.8 M | always-on | Q8_0 | |
| | `output.weight` | 1 | 508.6 M | always-on | Q8_0 | |
| | `token_embd` | 1 | 508.6 M | — | Q8_0 | |
| | `blk.40.nextn.eh_proj` + blk.40 attn/shexp | — | 39.3 M | draft projections | Q8_0 | |
| | **`ffn_{down,gate,up}_exps`** | 123 | **33.02 B** | INSTANT | **the dial** | |
| |
| † already F32 in llama.cpp: `ffn_gate_inp*` is excluded by name, `ssm_conv1d` has |
| first dim 4 (not a multiple of 256), 1-D tensors are never quantized. Pinning them |
| would be *inert* and fail GATE 0 — the intent is satisfied by the quantizer itself. |
| |
| **Fixed body ≈ 2.6 GB.** Propagating set = 47 M params = **0.14% of the model**, |
| exact for 0.15 GB. |
| |
| ### Expert split |
| |
| **UNIFORM** — `down` = `gate` = `up`. The first ladder shipped a 0.79 bpw bump on |
| `down` derived from `log4(0.6104/0.2048)`; a controlled test at identical size |
| refuted it (0.041192 uniform vs 0.046449 bumped, **11.3% better uniform**), and the |
| ladder was rebuilt. See PRINCIPLE.md Law 3. |
| |
| ### Draft head |
| |
| `blk.40`'s **experts follow the tier**; only its projections are Q8_0. Draft errors |
| are verified by the target model, so they cost *acceptance*, not correctness. |
| Frees ~0.40 GB versus pinning the whole block. |
|
|
| ### No depth grading |
|
|
| Refused deliberately. Water-filling over the real routing histogram shows the |
| optimal gain is **+0.139 bpw**, and the direction of a depth profile is unresolved |
| (PR says middle, convention says edges). Not worth the risk budget. |
|
|
| ## The ladder |
|
|
| | tier | built | expert bpw | down = gate = up | |
| |---|---:|---:|---| |
| | `19G-ICE` | 18.82 GB | 3.899 | IQ3_S / IQ4_XS | |
| | `21G-ICE` | 20.84 GB | 4.383 | IQ4_XS / Q4_K | |
| | `23G-ICE` | 22.83 GB | 4.868 | Q4_K / Q5_K | |
| | `25G-ICE` | 24.84 GB | 5.353 | Q5_K / Q6_K | |
|
|
| GATE 0: **0 double-matched, 0 uncovered, 0 inert** on all four. |
|
|
| ## Predicted position (active bpw, pre-measurement) |
|
|
| | build | disk | ACTIVE bpw | dense | blk.40 | |
| |---|---:|---:|---:|---:| |
| | UD-IQ4_XS | 18.68 G | 6.762 | 2.47 G | 0.90 G | |
| | **19G-ICE** | 18.84 G | **7.191** | 2.61 G | 0.43 G | |
| | **23G-ICE** | 22.83 G | **7.521** | 2.61 G | 0.53 G | |
| | UD-Q4_K_XL | 23.21 G | 7.471 | 2.59 G | 0.90 G | |
| | **25G-ICE** | 24.84 G | **7.691** | 2.61 G | 0.53 G | |
| | UD-Q5_K_S | 25.83 G | 7.693 | 2.59 G | 0.90 G | |
| |
| `25G-ICE` matches `UD-Q5_K_S`'s active precision at **0.99 GB less**. |
| |
| ## Falsifiable predictions — scored |
| |
| 1. ~~`25G-ICE` ≥ `UD-Q5_K_S` on measured KLD at ~1 GB less.~~ **FAILED.** 0.0303 vs |
| 0.0272; ICE is 0.99 GB smaller but not better. The 0.79 gap it rested on was then |
| refuted outright. |
| 2. **`19G-ICE` beats `UD-IQ4_XS`** — **HELD.** 0.060844 vs 0.0723, **15.8% better** |
| at 0.14 GB *smaller* (18.82 vs 18.68 GB is +0.14; after the uniform rebuild the |
| file is 18.82 GB and still wins). |
| 3. **Un-pinning `blk.40` costs <2 pp acceptance** — **HELD, direction unverified.** |
| 96.04% measured, above the 92.97% baseline, but on a different tier so not a |
| clean comparison. No collapse. |
| |
| Same harness as the UD/APEX ladder — wikitext-2-raw, 16 chunks, 2048 ctx — so the |
| numbers drop straight into the existing table. Changing the harness would make the |
| comparison meaningless. |
| |
| ## Results — shipped |
| |
| | tier | size | mean KLD | top-1 | vs UD curve at same size | |
| |---|---:|---:|---:|---:| |
| | `25G-ICE` | 24.84 GB | 0.030304 | 93.16% | −1.9% | |
| | `23G-ICE` | 22.83 GB | 0.036069 | 92.65% | −6.3% | |
| | `21G-ICE` | 20.84 GB | 0.041192 | 92.03% | **−18.6%** | |
| | `19G-ICE` | 18.82 GB | 0.060844 | 89.72% | **−13.6%** | |
| |
| **MTP draft acceptance: 96.04%** (388/404), measured on the shrunk `blk.40`. |
| Baseline with the block fully Q8_0 was 92.97% on a different tier, so this is not a |
| clean comparison — but it establishes the smaller draft head still drafts. |
|
|
| The advantage grows as size falls: at the top of the ladder the calibration floor |
| (~0.0185) dominates and no allocation choice can move it. |
|
|
| ### What did not survive measurement |
|
|
| | idea | verdict | |
| |---|---| |
| | `down` bumped above `gate`/`up` | **refuted, −11.3%** — ship uniform | |
| | `token_embd` → Q4_K | **not replicated** — helps at 21G, hurts at 19G | |
| | `attn_gate` / `ssm_out` → Q6_K | **harmful** — always-on, 32× law | |
| | `attn_q` demotion | dropped before shipping — arithmetic didn't hold | |
| | up↔down scale migration | **dead** — `down` columns uniform (p95/p05 = 1.09) | |
|
|
