appendix/original-ICE/RECIPE.md

metadata

title: ICE for Ornith-1.5-35B-A3B — the concrete instantiation
created: 2026-08-21T00:00:00.000Z
status: SHIPPED 2026-08-21 — 4 tiers live, uniform expert allocation
principle: see PRINCIPLE.md (model-agnostic)

ICE — Ornith-1.5-35B-A3B

The general method is in PRINCIPLE.md. This file is what it produced for this model, with the arithmetic shown.

Architecture inputs

blocks	41 = 30 linear/SSM + 10 full-attention (3,7,…,39) + 1 MTP (blk.40)
residual width	2048
experts	256, 8 active → sparsity ratio 32
expert FFN / shared FFN	512 / 512
KV heads	2, head_dim 256, on 10 blocks only → ~20 KB/token
vocab	248,320 → output.weight = 508.6 M in one tensor

Every block is 95.6% expert weights (805.3 M of ~842 M). The surgical field is the remaining ~35 M per block plus the two 508 M end-caps.

Active-compute geography (measured)

role	params	% of file	% of active compute
routed experts (8/256)	32.21 B	92.94%	34.2%
attention	1.03 B	2.96%	34.9%
output head	0.51 B	1.47%	17.3%
ssm	0.26 B	0.74%	8.7%
shared expert	0.13 B	0.36%	4.3%
token_embd	0.51 B	1.47%	0% (get_rows)

65.8% of active parameters are non-routed. Active total 2.95 B — which is what the "A3B" label means, and a useful check that the accounting is right.

The assignment

tensor	×	params	class	type
ffn_gate_inp, ffn_gate_inp_shexp	80	21.0 M	DISCRETE	F32 †
ssm_conv1d	30	1.0 M	RECURRENT	F32 †
ssm_alpha, ssm_beta	60	3.9 M	RECURRENT	F32 (pinned)
ssm_a, ssm_dt, all norms	—	0.3 M	RECURRENT / —	F32 †
attn_k, attn_v	20	21.0 M	CACHED	F16
attn_q	10	167.8 M	INSTANT	Q8_0
attn_output	10	83.9 M	INSTANT	Q8_0
attn_qkv	30	503.3 M	INSTANT	Q8_0
attn_gate	30	251.7 M	INSTANT	Q8_0
ssm_out	30	251.7 M	INSTANT	Q8_0
ffn_*_shexp	120	125.8 M	always-on	Q8_0
output.weight	1	508.6 M	always-on	Q8_0
token_embd	1	508.6 M	—	Q8_0
blk.40.nextn.eh_proj + blk.40 attn/shexp	—	39.3 M	draft projections	Q8_0
ffn_{down,gate,up}_exps	123	33.02 B	INSTANT	the dial

† already F32 in llama.cpp: ffn_gate_inp* is excluded by name, ssm_conv1d has first dim 4 (not a multiple of 256), 1-D tensors are never quantized. Pinning them would be inert and fail GATE 0 — the intent is satisfied by the quantizer itself.

Fixed body ≈ 2.6 GB. Propagating set = 47 M params = 0.14% of the model, exact for 0.15 GB.

Expert split

UNIFORM — down = gate = up. The first ladder shipped a 0.79 bpw bump on down derived from log4(0.6104/0.2048); a controlled test at identical size refuted it (0.041192 uniform vs 0.046449 bumped, 11.3% better uniform), and the ladder was rebuilt. See PRINCIPLE.md Law 3.

Draft head

blk.40's experts follow the tier; only its projections are Q8_0. Draft errors are verified by the target model, so they cost acceptance, not correctness. Frees ~0.40 GB versus pinning the whole block.

No depth grading

Refused deliberately. Water-filling over the real routing histogram shows the optimal gain is +0.139 bpw, and the direction of a depth profile is unresolved (PR says middle, convention says edges). Not worth the risk budget.

The ladder

tier	built	expert bpw	down = gate = up
19G-ICE	18.82 GB	3.899	IQ3_S / IQ4_XS
21G-ICE	20.84 GB	4.383	IQ4_XS / Q4_K
23G-ICE	22.83 GB	4.868	Q4_K / Q5_K
25G-ICE	24.84 GB	5.353	Q5_K / Q6_K

GATE 0: 0 double-matched, 0 uncovered, 0 inert on all four.

Predicted position (active bpw, pre-measurement)

build	disk	ACTIVE bpw	dense	blk.40
UD-IQ4_XS	18.68 G	6.762	2.47 G	0.90 G
19G-ICE	18.84 G	7.191	2.61 G	0.43 G
23G-ICE	22.83 G	7.521	2.61 G	0.53 G
UD-Q4_K_XL	23.21 G	7.471	2.59 G	0.90 G
25G-ICE	24.84 G	7.691	2.61 G	0.53 G
UD-Q5_K_S	25.83 G	7.693	2.59 G	0.90 G

25G-ICE matches UD-Q5_K_S's active precision at 0.99 GB less.

Falsifiable predictions — scored

1. 25G-ICE ≥ UD-Q5_K_S on measured KLD at ~1 GB less. FAILED. 0.0303 vs 0.0272; ICE is 0.99 GB smaller but not better. The 0.79 gap it rested on was then refuted outright.
2. 19G-ICE beats UD-IQ4_XS — HELD. 0.060844 vs 0.0723, 15.8% better at 0.14 GB smaller (18.82 vs 18.68 GB is +0.14; after the uniform rebuild the file is 18.82 GB and still wins).
3. Un-pinning blk.40 costs <2 pp acceptance — HELD, direction unverified. 96.04% measured, above the 92.97% baseline, but on a different tier so not a clean comparison. No collapse.

Same harness as the UD/APEX ladder — wikitext-2-raw, 16 chunks, 2048 ctx — so the numbers drop straight into the existing table. Changing the harness would make the comparison meaningless.

Results — shipped

tier	size	mean KLD	top-1	vs UD curve at same size
25G-ICE	24.84 GB	0.030304	93.16%	−1.9%
23G-ICE	22.83 GB	0.036069	92.65%	−6.3%
21G-ICE	20.84 GB	0.041192	92.03%	−18.6%
19G-ICE	18.82 GB	0.060844	89.72%	−13.6%

MTP draft acceptance: 96.04% (388/404), measured on the shrunk blk.40. Baseline with the block fully Q8_0 was 92.97% on a different tier, so this is not a clean comparison — but it establishes the smaller draft head still drafts.

The advantage grows as size falls: at the top of the ladder the calibration floor (~0.0185) dominates and no allocation choice can move it.

What did not survive measurement

idea	verdict
down bumped above gate/up	refuted, −11.3% — ship uniform
token_embd → Q4_K	not replicated — helps at 21G, hurts at 19G
attn_gate / ssm_out → Q6_K	harmful — always-on, 32× law
attn_q demotion	dropped before shipping — arithmetic didn't hold
up↔down scale migration	dead — down columns uniform (p95/p05 = 1.09)