docs/VALIDATION.md

Development Snapshot Validation

Date: 2026-09-08. This is not beta certification or a llama.cpp comparison.

Environment

• Windows x64, AMD Ryzen 5 5600, 6 physical cores / 12 logical processors.
• CPython 3.13, MSVC Visual Studio Build Tools 2022 17.14.7.
• One native inference thread, runtime-dispatched AVX2.
• Installed wheel: cism 0.1.0.dev0.
• NumPy 2.5.3, Transformers 5.16.1, PyTorch 2.14.0+cpu for reference tests only.
• FastAPI 0.141.1, Uvicorn 0.52.4, OpenAI Python SDK 2.54.0.

Tests

python -m pytest -q: 555 passed, two upstream TestClient deprecation warnings. Includes independent NumPy numerical references, Transformers tiny model parity, packed quantization, invalid checkpoint/config handling, generation metadata, cancellation, queue limits, SSE, official SDK, and a live Uvicorn TCP roundtrip using real native inference.

python -m pip check: no broken requirements.

Both scalar-only and AVX2 native builds were also tested separately during kernel development. Linux, sanitizers, long-running load tests, and full-context real checkpoint certification remain outstanding.

Real Checkpoints

Compared default FP32 eager HF computation with native last-token logits and four-token greedy continuations at prompt lengths 4, 8, and 16. All tested continuations agreed; maximum absolute logit error was below 0.0001.

• SupraLabs/Supra-Mini-v5-8M: FP32, INT8, hybrid INT4.
• veyra-ai/Veyra2-Blueberry-10M-Base: FP32.

INT8/INT4 references used reconstructed quantized weights, not original FP32 weights. These results establish implementation parity only, not a PPL quality budget or task-quality equivalence.

Local Throughput

cism bench SupraLabs/Supra-Mini-v5-8M --precision int8 --runs 5 --max-tokens 128

Model revision: bb98e3566a5ab3f24be16ee8db7f06f0cb884eb5. Packed weight storage including scales/norms: 8,022,784 bytes.

• Prompt: 6 tokens; generation: 128 tokens; EOS ignored for fixed-length timing.
• One warmup, then five runs; native generation through coarse Python bindings.
• Median decode throughput excluding prefill/first token: 1,622.86 tokens/s.
• Runs: 1,595.72 / 1,632.20 / 1,624.86 / 1,603.32 / 1,622.86 tokens/s.
• Median prefill plus first token: 3.744 ms.

No tokenization, detokenization, HTTP, or queue latency is included in decode timing. These are growing-context token-generation results, not weight sweeps. CPU affinity and clocks were not locked. No comparison with a tuned llama.cpp baseline has been run; the required speed advantage is not established.

Why These Numbers Differ From The Original Experiments

The earlier session results (hundreds of thousands of tok/s) are not comparable to these, for verifiable reasons on both sides:

• The supplied C harness (decode_int8/decode_hybrid) looped over flat weight buffers with no layers, no logits, no sampling, and no token feedback. Its "tok/s" measured buffer-processing rate, not decoding.
• Its Hugging Face baseline ran use_cache=False, which undercounts real HF decode speed; its DDR4 column in the showcase table was computed from 34500 / model_MB, not measured.
• Kernel improvements have raised this snapshot's decode rate several-fold from the first working build, and it remains compute-bound (see the probe results above), so neither the old synthetic numbers nor the current 1,974 tok/s represent a final ceiling.

Kernel Optimization History

Baseline (per-row indirect dot calls, no FMA, serial INT4 block reduces), Supra-Mini-8M, one thread, 128-token decode: FP32 757, INT8 1,623, hybrid INT4 1,152 tokens/s.

After the first kernel pass (explicit FMA with 4 independent accumulator chains, AVX2 matvec kernels that absorb the row loop so one native call serves a whole matrix, FMA required in the CPU dispatch check):

Precision	Kernel-only rate	Full-model decode	Before	Gain
FP32	30.8 GB/s	936 tok/s (31.5 MB weights)	757	+24%
INT8	24.2 GB/s	1,974 tok/s (8.0 MB weights)	1,623	+22%
Hybrid INT4	4.8 GB/s per byte	1,382 tok/s (6.6 MB weights)	1,152	+20%
SmolLM2-135M INT8	-	127 tok/s (135.4 MB weights)	120	+6%

Cache-residency probe (unchanged conclusion): decode effective bandwidth (15.8 GB/s at INT8) remains below even the cold DRAM scan rate (25.1 GB/s), so decode is still compute-bound. The INT4 kernel is the current kernel bottleneck (nibble unpacking without VNNI); full-model non-dot work (attention, norms, 16k-logit sampling, epilogue) accounts for roughly a third of decode time at INT8.

Next levers, by expected impact: multithreaded matvec (row-parallel), prompt-prefill matrix-matrix batching, and further INT4 unpack improvements.

Activation-quantized kernel follow-up (2026-09-12)

The experimental act_precision="int8" path is now implemented for quantized matrix products. It quantizes every activation row in 32-element blocks using 32767 / absmax, uses int16 pmaddwd MACs, and retains FP32 block scales for the epilogue. It is numerically sound but is not enabled by default: on this Ryzen 5 5600 (Zen 3, AVX2/FMA, no VNNI), the quantization work dominates small layer GEMMs. Supra-Mini-8M, one thread, measured about 1,753 tok/s with INT8 weights and FP32 activations versus 603 tok/s with the experimental path; hybrid-INT4 measured 1,297 versus 607 tok/s, and hybrid-FP4 1,238 versus 574. Custom PPL comparisons put the activation-mode delta near +0.01 on Supra-Mini and at or below +0.1 on SmolLM2-135M. Keep it as an explicit experimental API until a fused/coarser quantizer or a CPU with suitable integer-dot instructions shows a net benefit.

Unpack-tax reduction, phase 1 (2026-09-12)

Shipped: skip the nibble interleave (even/odd streams dot a permuted activation scratch built once per call from plain local buffers, no TLS), halving shuffle-port work per 32 weights; dead matvec_* family deleted (~350 lines, verified uncalled); 5-arg scalar adapters keep non-AVX2 builds working. Tried and reverted the same session: folding per-8 scale multiplies into a per-block scalar epilogue — the serial per-block reduce loses to four parallel mulps on Zen 3 out-of-order execution (measured slower, not faster).

Measured on Ryzen 5 5600: full suite green, Mini PPL identical to 3 decimals, production AutoBench once showed fp4-2T beating int4-2T 1254 to 1015 (+24%) then losing the rematch — direction flips are inside this box's ±15-20% multithread noise, so neither is claimed. Drift-controlled A/B (alternating int8/fp4 in one process): 0.87 vs 0.90 pre-change. Verdict: the remaining ~26 µops/32 (converts, scale muls, act loads) are structural on AVX2 without VNNI; 4-bit cannot beat int8 kernel efficiency on this chip. Realistic next levers: int8-container upcast at load (int4 quality, near-int8 speed, int8 size — wins wherever cache residency makes bytes free), the q8 integer-MAC track, stronger ISA (VNNI/AMX/tensor cores), or DRAM-bound regimes where the 2x byte saving dominates.

Fused epilogues tried and reverted (2026-09-12): accumulating o/down dots directly onto x_ and SiLU into the up-gemm epilogue measured ~7% SLOWER on Supra2-Medium int8-1T in a back-to-back stash A/B (580 vs 623, tight clusters, wrong order for a thermal explanation). Likely cause: branches plus a temp in the FMA loop beat clean separate vector passes on wide rows, and an unconditional permute scratch taxed int8/fp32 paths (since made lazy). Lesson: on Zen 3 streaming hardware, an extra cache-resident vector pass can be cheaper than a branch in the hot loop — fuse only with A/B proof.

Session scratch arena (2026-09-12): per-call permute/row mallocs replaced by Session-persistent grow-only buffers (caller-owned, pool-safe; q8 TLS left alone). PPL identical; 4-bit/int8 tok/s ratios up ~0.1 at 2T on both models (Medium int4 0.83 -> 0.98, fp4 0.83 -> 0.96) — consistent with removing malloc-lock contention across pool threads rather than raw bytes. Model now reports info["compiled"] = {key, hit: False, mode: "generic", canonical} as the Phase-0 key plumbing goes live (execution still generic).

Policy: AutoBench now defaults to 512 decode tokens and compiles nothing unless asked (inductor warmup taxed every default run); thread guidance is 2T on a loaded server, never 12T (oversubscription convoy collapses pooled paths while serial paths sail through).

Supra2-Medium-Base production run (2026-09-12, live server, 512 tokens)

First model that spills past L3 (fp32 96.78 MB, DRAM-streaming):

Precision	tok/s (2T)	PPL	vs fp32
int8 (7.65 MB L3)	540	59.40	+0.62%
hybrid-fp4 (20.32 MB L3)	533	60.28	+2.12%
hybrid-int4 (20.89 MB L3)	475	62.61	+6.07%
fp32 (96.78 MB DRAM)	278	59.03	—
HF eager fp32	93	59.03	—
HF dynamic int8	72	95.45	+61.7%

hybrid-fp4 beats hybrid-int4 on speed (+12%) and quality (+2.1% vs +6.1%) where matrices are large enough to amortize the unpack. int8 still leads overall at near-zero quality cost. Speculation accepts at 96-99% and is the largest single lever here (fp32 +65% at spec 8, int4 +17%); HF dynamic int8 is outclassed on both axes. Threads peak at 2-4T; 12T convulses as usual.

Spec K curve (2026-09-12, Medium-Base int8, 2T, 64 tokens)

Cap raised 8 -> 16 to test whether near-100% acceptance favors longer drafts. Measured: K=0: 844, K=2: 974 (84% acc), K=4: 1016 (76%), K=8: 954 (71%), K=12: 915 (69%), K=16: 834 (71%). Peak at K=4: acceptance decays with draft length while block-verify cost grows linearly, so dilution loses past the peak. Optimal K is prompt-dependent (longer, more repetitive generations accept more); the sweep now covers 0-16 with per-thread accept% tables so each setup can read its own peak. Thread sweep preset trimmed to 1,2,4 (above 4T never pays on these sizes).

SmolLM2-135M precision x thread (2026-09-13, single-harness, act fp32, rev 93efa2f0)

Weight bytes (engine.info): int8 135.4 MB (4.2x L3), hybrid-int4 105.1 MB, hybrid-fp4 100.2 MB; all spill, fits_l3=false. 512-token decode, greedy, spec_k=0, one live engine at a time:

Threads	int8 tok/s (GB/s)	hybrid-int4 tok/s (GB/s)	hybrid-fp4 tok/s (GB/s)
1T	88 (11.9)	72 (7.5, 0.82x)	73 (7.3, 0.83x)
2T	120 (16.2)	107 (11.3, 0.89x)	106 (10.6, 0.88x)
4T	124 (16.8)	122 (12.8, 0.98x)	122 (12.2, 0.99x)

PPL (514 scored tokens, window 128): int8 34.74, hybrid-int4 42.07 (+21.1% vs int8), hybrid-fp4 41.32 (+18.9% vs int8; fp4 beats int4 on quality at fewer bytes, speed tied within noise). Central 64-token 1T check: int8 120.9, int4 90.4, fp4 89.0 tok/s with identical weight bytes — same ordering. 128-token bracket int8-1T 114.5 tok/s / 15.5 GB/s matches the old 120-127 table; 512-window context growth costs ~21%, so ratios are intra-run only.

Verdict: int8 is bandwidth-bound (saturates at 2T, +3.6% to 4T), 4-bit paths are kernel-bound at every thread count (1T->4T ~1.7x scaling, DRAM idle, never cash the 1.29x byte edge). Kernel bet unchanged on Zen 3 / no-VNNI: int8 default everywhere, fp4 the 4-bit pick. 360M not fetched (724 MB BF16 shard); expect the amplified same pattern. Next levers: spec on DRAM-bound int8-135M, int8-container upcast at load for cache-resident sizes (test-gated), VNNI-class ISA. Compile Phase-0 stays hit:False/mode:generic (plumbing only, execution generic) — correct until a PPL-gated execution change ships.

Threading Results

Persistent native pool, row-sliced matvecs, pause-spin barriers (yield/sleep only after long idle). Supra-Mini-8M INT8, 256-token decode, medians:

Threads	Tokens/s	vs 1T
1	1,548	1.00x
2	2,148	1.39x
4	2,036	1.31x (regresses)

FP32 (31.5 MB, DRAM-bound) shows no thread scaling: 998 -> 1,004 tok/s at 4T. The Ryzen 5600 sweet spot is 2 threads for cache-sized quantized models; beyond that, shared L3/LSU bandwidth saturates. gVisor-class sandboxes showed ~19 us sync round trips, so threading there may need larger models to pay off. cism bench/serve --threads N sets the pool size; default remains 1.

Speculative Decoding (Prompt Lookup, Greedy)

Implemented: n-gram drafter over full token history (up to 4-gram tails), block verification via per-matrix GEMM (all T candidates share one weight stream), greedy argmax acceptance with KV rollback, budget/EOS boundaries. Speculative greedy output is bitwise identical to plain greedy decode (tested across architectures, precisions, rejection paths, and budgets).

SmolLM2-135M (135 MB INT8 / 540 MB FP32 - exceeds the 32 MB L3), one thread, 256 tokens, repetitive prompt for high draft acceptance:

Configuration	Tokens/s	Gain
FP32 plain	51	1.00x
FP32 spec_k=4	83	1.63x
INT8 plain	106	1.00x
INT8 spec_k=4	130	1.23x

Reason: speculation amortizes weight-bandwidth (one stream per verify block), so gains appear when decoding is DRAM-bound - the FP32 regime. The INT8 kernel is compute-bound (conversion-heavy SIMD), so amortizing bandwidth buys little until integer kernels land. On the AVX-512 VNNI cloud target, integer kernels should make INT8 bandwidth-bound and transfer the full speculation multiplier. Two structural costs: the bonus token costs one extra single-token pass (ideal (K+1)/2, measured 1.6x at K=4), and low-acceptance text can make speculation slightly negative - disable it when drafts do not match the domain.

Memory Control Verification

Real-model page locking and touch on this machine (Supra-Mini-8M INT8):

• lock_pages(): 8,022,880 of 8,022,784 weight bytes locked (page rounding).
• touch(): 8,022,880 bytes swept.
• Generation while keep-warm active and pages locked: normal.
• unlock_pages(): 6,268,768 bytes reported. Smaller than the lock count because adjacent weight regions share 4 KiB pages; a page released by one region cannot be released again by its neighbor. Counts are approximate byte sums of successful OS calls, not exact page accounting.

x86 cannot pin cache lines; locking prevents OS paging, and sweeping only refreshes residency while running. Suite at time of writing: 571 tests passed.

Cache-Residency Probe Results

cism cache-report measures a warm full-weight scan, a cold scan after evicting caches with a 96 MiB scrub buffer, and decode effective bandwidth (weight bytes x decode tok/s). AMD Ryzen 5 5600, INT8, one thread:

Model	Weights	Warm scan	Cold scan	Decode	Effective	Verdict
Supra-Mini-8M	8.0 MB	28.5 GB/s	25.1 GB/s	1,646 tok/s	13.2 GB/s	Compute-bound
SmolLM2-135M	135.4 MB	21.4 GB/s	23.1 GB/s	120 tok/s	16.3 GB/s	Compute-bound

Interpretation:

• Decode bandwidth is well below even the cold (post-eviction) scan rate in both cases. The decoder does not saturate DRAM, so cache residency is not the current bottleneck; kernel efficiency is (scalar conversion-heavy quantized kernels, sequential prefill, single thread).
• The warm/cold scan ratio is small for these linear probes; hardware prefetchers hide much of DRAM latency on sequential streams, so this probe bounds bandwidth rather than proving residency. Definitive per-event counts (LLC miss) require hardware PMU tooling such as WindowsPerf or AMD uProf.
• The earlier session's implied "SRAM 3x-6.5x multiplier" was not observed by this probe on this hardware and remains unproven.

Speed work priorities implied by this data: quantized kernel quality, prefill matrix-matrix, and multithreading before further cache-tuning.

Measured Comparison Against PyTorch 2.14 And torch.compile

Supra-Mini-v5-8M, revision bb98e356, FP32 checkpoint, greedy, one torch/CISM thread, 6-token prompt, 256-token fixed-length decode (EOS ignored both sides), warmup absorbed, Dynamo counters verified zero recompilations during measurement. Torch baselines use a KV cache (never use_cache=False). Script: scripts/bench_vs_torch.py; CISM: cism bench --max-tokens 256.

Decoder	Tokens/s	CISM FP32 (998.5)	CISM INT8 (1,716.4)
HF generate(use_cache=True) eager	158.9	6.28x	10.80x
Manual eager loop + StaticCache	195.8	5.10x	8.77x
torch.compile default, dynamic=False	260.1	3.84x	6.60x
torch.compile default, dynamic=True	224.9	4.44x	7.63x

The fair comparison is against torch.compile dynamic=False, its best configuration here: 3.84x at matched FP32 precision, 6.60x for CISM INT8.

Reasons, with evidence:

1. Per-token dispatch. torch.compile lowers eager's per-op Python/ATen dispatch (159 -> 260 tok/s, +64%) but still launches dozens of generated kernels through a Python-level decode loop each token. CISM executes the whole token in one C++ call. Eager's effective bandwidth is 31.5 MB x 159 = 5.0 GB/s and compile's is 8.2 GB/s - both overhead-bound, while CISM FP32 runs memory-bound at ~30 GB/s kernel rate.
2. Memory traffic. CISM INT8 reads 8.0 MB/token versus 31.5 MB for the FP32 checkpoint, cutting the memory-bound floor ~4x on top of the dispatch elimination.
3. Quality. CISM FP32 matches eager within 1e-5 max logit error (established above). INT8 quality reporting remains a beta gate.

Caveats: compile mode is "default" (max-autotune not used); one compile configuration initially crashed with an Inductor codegen bug on this model (index-out-of-bounds in a generated kernel) and only ran after cache-reset handling; multithreaded torch was not measured; prefill/TTFT is excluded from decode rates. This is one cache-sized model on one machine - not yet the agreed multi-model llama.cpp gate.

4-bit quantization scheme study (2026-09-12)

NumPy simulation mirroring the native hybrid-int4 quantizer (round-half-away, blocks of 32 along the row, per-block scale; validated against the native engine to 8 decimal places on all checkpoints). PPL over the fixed AutoBench corpus (514 tokens, window 128), MLP matrices 4-bit, all other 2D matrices per-row INT8, vectors FP32.

Machine: Ryzen 5 5600, local checkpoints, scripts/quant4_study.py.

scheme (MLP element + scale)	bits/w	8M ΔPPL	100M ΔPPL	135M ΔPPL
int4 + FP32 b32 (hybrid-int4)	5.00	+15.4%	+6.1%	+23.2%
int4 + E4M3 b32	4.25	+15.5%	+6.0%	+26.7%
int4 + E8M0 b32	4.25	+308%	+80.8%	+537%
E2M1 + E8M0 b32 (OCP MXFP4)	4.25	+318%	+92.4%	+323%
E2M1 + E4M3 b32	4.25	+18.7%	+3.5%	+23.3%
E2M1 + E4M3 b16 (NVFP4 granularity)	4.50	+15.0%	+2.1%	+21.0%
int4 + E4M3 b16	4.50	n/a	+8.1%	n/a
E2M1 + FP32 b32	5.00	+13.7%	+4.4%	+19.9%
NF4 + FP32 b32 (QLoRA)	5.00	+44.1%	+8.8%	+32.0%
int4 + error feedback b32	5.00	+36.5%	+16.9%	+80.9%

Findings:

• Scale precision dominates element format; power-of-two (E8M0) scales are catastrophic for both uniform and FP elements.
• E2M1 elements beat uniform INT4 at equal scale precision and block size (100M model, b16+e4m3: +2.1% vs +8.1%).
• E2M1 + E4M3 b16 is the only scheme that improves on hybrid-int4 at every model size while using fewer bits (4.50 vs 5.00): tied on 8M, 3x better on 100M, better on 135M.
• NF4 (QLoRA) loses badly on these trained small models; naive per-block error feedback with a fixed scale inflates residuals and hurts everywhere.

Why hybrid-int4 decodes slower than int8 (measured, 1 thread, same checkpoints): the nibble unpack is a ~10-instruction dependent chain per block, so hybrid-int4 delivers only 7.6-8.1 GB/s of weight-equivalents vs 12.5-14.0 for int8 and 26.6 for fp32, despite moving 2.5x fewer bytes than fp32. The E2M1 LUT kernel replaces that chain with a 16-entry table lookup and is the reason to build the mode.

Compile one-shot canonical parity (2026-09-14)

Phase-0 key plumbing is now cross-language exact. Python compile_cache.canonical_model_config() builds the native 12-field pipe model_type|hidden|intermediate|layers|heads|kv_heads|head_dim|vocab| context|hexfloat(eps)|hexfloat(rope_theta)|tied with FP32-round-trip struct.pack('f') hexfloat, matching Model::refresh_compile_key bit-for-bit. Verified: tiny Llama 16/32/1/4/2/4/32/32 probe Python canonical equals Model.info["compiled"]["canonical"], and native.compile_key(python_canonical, precision, act, threads) equals Model.info["compiled"]["key"] (d4547092c73a7aac on the probe). Legacy canonical_config() sorted-JSON remains for early tests only. Kept as a stable model fingerprint (zero runtime cost, tested); the packed-artifact load idea is dropped with the project (see below).

SmolLM2-135M full AutoBench matrix (2026-09-13, 512 tokens, act fp32)

Local file autobench_135M_512.json (untracked measurement, torch baselines disabled by policy include_compile=False): 4 precisions x threads (1,2,4) x spec_k (0,2,4,8,12,16) = 72 decode cells, PPL once per precision (514 scored tokens, window 128). Revision 93efa2f0 (report does not yet stamp revision; next run should copy engine.info["revision"]). All fits_l3=false (L3 cache size 32.0 MB).

Weight storage in megabytes: int8 129.16 MB (135,439,104 bytes), hybrid-int4 100.27 MB (105,141,504 bytes), hybrid-fp4 95.52 MB (100,164,864 bytes), fp32 513.13 MB.

Decode rate in tokens/s at spec_k=0 (effective bandwidth in GB/s = weight bytes x tok/s):

Precision	1 thread tok/s (GB/s)	2 threads tok/s (GB/s)	4 threads tok/s (GB/s)	Best tok/s (threads, spec_k)	PPL (+vs int8)
fp32	43.78 (23.6)	57.44 (30.9)	55.34 (29.8)	106.41 (4T, K=8)	34.15
int8	91.53 (12.4)	124.82 (16.9)	130.03 (17.6)	147.19 (4T, K=2)	34.74 (--)
hybrid-int4	71.93 (7.6)	106.54 (11.2)	115.69 (12.2)	140.59 (4T, K=16)	42.07 (+21.1%)
hybrid-fp4	71.85 (7.2)	102.10 (10.2)	117.51 (11.8)	132.90 (4T, K=2)	41.32 (+18.9%)

Ratios int8 vs 4-bit at spec 0: 1T 1.27x, 2T 1.17-1.22x, 4T 1.11-1.12x. Scaling 1T to 4T: int8 1.42x but 2T to 4T only +4.2% (saturates, bandwidth-bound); int4 1.61x, fp4 1.64x, still climbing with DRAM idle (kernel-bound nibble unpack, no VNNI). Verdict unchanged: int8 default everywhere, fp4 the 4-bit pick (better quality at fewer bytes, speed tied within ±15-20% multithread noise; ratios single-harness only).

Spec K-curve on 135M DRAM-bound int8 (512 tokens, greedy)

Accept rate in percent is deterministic across threads per K (88.0% at K=2 decaying to 72.4% at K=16).

Threads	K=0 tok/s	K=2 tok/s	K=4 tok/s	K=8 tok/s	K=12 tok/s	K=16 tok/s	Peak gain vs K=0
1T	91.53	98.42	100.47	99.45	96.14	93.67	K=4, +9.8%
2T	124.82	136.05	132.62	131.34	129.72	126.94	K=2, +9.0%
4T	130.03	147.19	146.55	135.13	136.81	143.53	K=2, +13.2%

Global best: int8 147.19 tok/s at 4 threads, spec_k=2. Contrast fp32 +73-92% peaking K=8 (bandwidth-bound, block verify amortizes DRAM) vs 4-bit +5-21% erratic (kernel-bound). Peak at small K because acceptance decays with draft length while GEMM verify cost grows linearly, plus the bonus-token extra pass. Prompt-lookup drafter (up to 4-gram, greedy-only, bitwise identical) helps repetitive prompts only; it does not fix the 135M slow complaint (still ~147 tok/s vs ~1,600 tok/s cache-resident 8M). Default 135M int8: 4 threads, spec_k=2 (2-thread servers also K=2).

trust_remote_code custom-arch assessment (2026-09-14)

trust_remote_code=True authorizes tokenizer Python only, never custom modeling operators; trusted code itself is not sandboxed. Enforcement: loader _ARCHITECTURES={"llama","qwen3"}, validate_config rejects non-llama/qwen3 model_type and any auto_map beyond AutoTokenizer even with trust enabled, cross-repo refs rejected, _expected_shapes plus _load_weights reject extra/missing tensors, native Config::validate/parse_config reject bias/MoE/sliding-window/ non-silu/non-default-RoPE. Custom Negative/CMA/Ember2/Spark2A/Blaze targets (ROADMAP gate 4: recurrence, routing, lane mixing, Engram state) need new native Config fields, operators, KV layouts, quant definitions, and PPL/parity gates per arch — large work, not a flag flip. Keep explicit UnsupportedModelError; server stays opt-in --allow-remote-code-load (403 otherwise).

Compile project dropped (2026-09-14, verdict: correctly built, not worth it)

Target was one-time load cost for faster batch-1 CPU decode (generic AVX2, PPL-gated). Verdict: the plumbing is correct and tested, but the goal is unreachable on this CPU, so the project is dropped and its speedup fiction removed:

• What was real: cross-language canonical parity (bit-exact, tested), hoisted dispatch loops (same kernels/order, bitwise identical, kept as plain code), manifest helpers (kept, unused by load path).
• What was fiction: manifest-hit as "compile hit" (bookkeeping, not a faster layout), compiled.hit/mode in Engine info and tester cards (removed), any decode tok/s attributed to compile (zero: mode was generic everywhere; the +13.2% is spec K=2).
• Why not worth it: remaining ~26 µops/32 are structural on AVX2/no-VNNI; int8 is bandwidth-bound (2T saturates), 4-bit kernel-bound; fresh 512 runs show hoisting moved 4-bit ~+23% but int8 K0 flat (-0.9%), confounded by idle-machine effect — no clean decode win to claim; inductor costs 107 s warmup for 32 tok/s against CISM's 190 (6-8x lead from the pre-existing single-call C++ path + quant, not compile). VNNI/VBMI or integer-MAC kernels would reopen the question; until then the line is closed. Fingerprint key + hoisted loops stay; hit/UI/ future-speedup prose goes.

Fresh 512-token 4T confirmation (SmolLM2-135M, all precisions, spec sweep)

Re-ran 24 cells (4 precisions x 4T x spec 0,2,4,8,12,16), 512 tokens, 127.3 s wall, torch off. Accept% deterministic and identical to the old file; PPL/weights identical (fp32 34.15, int8 34.74, int4 42.07, fp4 41.32; 513.13/129.16/100.27/95.52 MB, all fits_l3=false). Absolute tok/s ran +11-30% hotter than the old file (idle machine); cells >25% off are marked SUSPECT — use this table, do not mix files.

Precision	K=0	K=2	K=4	K=8	K=12	K=16	Best
fp32	58.77	72.89	101.92	125.09	122.38	121.18	K=8 125.09
int8	128.90	159.99	162.64	171.76	171.37	177.03 (SUSPECT vs old)	K=16 177.03
hybrid-int4	142.63	155.69	151.97	159.86	160.29	156.40	K=12 160.29
hybrid-fp4	144.58	150.73	153.96	160.41	155.88	153.82	K=8 160.41

Accept%: fp32 96.5/94.0/89.9/88.1/85.5; int8 88.0/83.4/77.1/73.0/72.4; int4 90.1/84.4/79.1/75.6/75.3; fp4 84.1/76.6/70.3/66.7/67.6 (K=2..16). Note 4-bit now leads int8 at K=0 (142.63/144.58 vs 128.90) in this run — confounded by machine state (old file had int8 leading), so no compile attribution claimed without a same-state A/B. fp4 vs int4: tied within noise at every K (max gap 3.7%), fp4 keeps the quality edge (+18.9% vs +21.1% PPL), so fp4 stays the 4-bit pick.

4T K=2 vs regular baselines (SmolLM2-135M int8, 512 tokens)

Global best cell is int8 4 threads spec_k=2 at 147.19 tok/s (accept 87.96%). Regular baselines from the same autobench_135M_512.json run (include_torch=False, so no HF eager/compile in this file):

Baseline (4T)	Tok/s	vs 4T K=2 (147.19)
int8 K=0 (same weights, no spec)	130.03	+13.2% with spec
int8 2T K=0	124.82	+17.9% (threads+spec)
int8 1T K=0	91.53	+60.8%
fp32 K=0	55.34	2.66x
fp32 best (K=8)	106.41	1.38x
hybrid-int4 K=0	115.69	1.27x
hybrid-int4 best (K=16)	140.59	1.05x
hybrid-fp4 K=0	117.51	1.25x
hybrid-fp4 best (K=2)	132.90	1.11x

PPL in same run: int8 34.74, fp32 34.15, int4 42.07, fp4 41.32. Spec pays (+13.2%) because 135M int8 spills (129.16 MB vs 32 MB L3): block verify streams weights once per block via GEMM. It does not make 135M fast in absolute terms (still ~147 vs ~1,600 cache-resident 8M).

HF eager/compile baselines (SmolLM2-135M, 512 tokens, 4T, same harness)

One run_benchmark (src/cism/autobench.py:223) with include_torch=True, include_compile=True, max_new_tokens=512, threads 4, spec sweep 0-16, rev 2c4d40c. Whole sweep ran ~4-15% under the fresh reference (mild thermal throttle); HF rows are baselines, not throttle-sensitive conclusions.

Decoder (512 tok, 4T)	Tok/s	PPL (514 scored tokens)
CISM int8 K=0	131.88	34.74 (nll 3.5480, +1.73% vs eager)
CISM int8 best K=16	167.23	same
CISM fp32 K=0 / best K=8	53.32 / 114.67	34.15 (+0.002%)
CISM hybrid-int4 K=0 / best K=8	137.20 / 153.07	42.07 (+23.18%)
CISM hybrid-fp4 K=0 / best K=16	137.38 / 150.25	41.32 (+21.00%)
HF eager fp32	23.79 (21.52 s)	34.15 (nll 3.5308)
HF dynamic int8	19.74 (25.94 s)	81.12 (broken, +137.5%)
HF torch.compile fp32	25.18 (20.33 s, warmup 147.3 s)	not scored
HF compile int8	16.86 (30.38 s)	not scored
torchao int4	skipped (not installed)	--

Speedups (same harness): CISM int8 K=0 5.54x eager / 5.24x compile; best K=16 7.03x / 6.64x; fp32 K=0 2.24x/2.12x, best K=8 4.82x/4.55x. Compile barely beats eager (+5.9%) after a 147 s warmup; dynamic int8 is slower than fp32 AND destroyed (PPL 81). Weights: 513.13/129.16/100.27/ 95.52 MB, all fits_l3=false (L3 32 MB).

Surjo hybrid native Phase1 (loader+fp32 decode)

Scope: SurjoLabs Surjo hybrid import plus native fp32 decode runs. No Surjo tok/s, no Surjo PPL, no speedup claim in this section. Reference dense numbers below are context only, single-harness, ±15-20% multithread noise (see 135M matrix notes); do not mix files or attribute cross-run deltas to Surjo.

Config fingerprint (Surjo-50m, tests/test_surjo.py:20): model_type surjo / SurjoForCausalLM, vocab 32768, hidden 512, intermediate 1536, layers 10, context 2048, prelude 1 / recurrent 8 / coda 1, groups 2 / passes 2 / gdn_per_xsa 3, heads 8 / kv 4 / head_dim 64, gdn_v_heads 8 / gdn_k_dim 64 / gdn_v_dim 64 / kernel 4, gdn_allow_neg_eigval false, xsa_projection true, bias false, dropout 0.0, hidden_act silu, eps 1e-5, theta 10000.0, default RoPE (partial_rotary_factor 1.0), tied true, layer_types [F,L,L,L,F,L,L,L,F,F], auto_map pinned to configuration_surjo/modeling_surjo (never executed). Base checkpoints ship use_cache=false; import_model (src/cism/loader.py:666) preserves the flag verbatim and it does not disable native caching (XSA KV slots + GDN recurrent/conv state always allocate).

Checkpoint header: 178xF32 tensors in model.safetensors, tied lm_head omitted, no rotary inv_freq buffers (RoPE is computed in forward()). _expected_shapes_surjo length is 179 including the tied-head alias entry; spot-checked in test_surjo_50m_expected_shapes_match_header (embed (32768,512), XSA q (512,512) / k (256,512) / q_norm (64,), GDN q/k (512,512) / v (512,512) / conv (512,1,4) / f0 (64,512) / f1 (512,64) / A_log (8,) / dt_bias (512,) / g1.bias (512,) / o_norm (64,)).

Loader (src/cism/loader.py): validate_surjo_config, surjo_exec_plan (18 steps: 1 prelude + 2 passes x 2 groups x (3 GDN + 1 XSA) + 1 coda; XSA slots [0..5], 12 GDN (layer,pass) steps), surjo_num_xsa_slots (=6), _expected_shapes_surjo:393, _load_weights Surjo branch (no buffers, strict shape/dtype/finite checks, tied-head alias by identity), import_model dispatch (model_type=="surjo" -> Surjo path, else dense). _SURJO_SUPPORT string still reads "next phase" in this snapshot; Phase1 below is what landed natively.

Native (native/runtime.{hpp,cpp}, native/bindings.cpp:468, src/cism/engine.py:80 dispatch to _native.SurjoModel): SurjoModel/SurjoSession fp32 decode. XSA: 6 slots, KV [slots,capacity,kv_width] with kv_width=256; at max context 2048 that is 4 MiB/slot (K+V), 24 MiB total (session-sized by prompt+max_new, so 24 MiB is the upper bound). GDN: 12 states (6 GDN layers x 2 passes), each S is H*K*V=8*64*64=32768 floats = 128 KiB (12x128 KiB = 1.5 MiB), plus depthwise-conv raw FIFO (kernel-1)=3: q/k/v FIFOs ~72 KiB each, ~0.21 MiB total. MLP is ClampedMLP (gate clamped to [-15,15] then silu*up, runtime.cpp:2010). spec_k is rejected for Surjo (next_tokens throws unless spec_k==0; Engine.stream(spec_k=2) raises ValueError, covered by test_surjo_engine_decode_runs). Prefill is sequential per-token (forward_tokens loop); logits / nll teacher-forcing paths exist but are unscored against HF so far.

Quant mapping reuse, no new format: protected_storage (embed, head, all XSA/GDN projections incl. f0/f1/g0/g1/o_proj) is fp32 or int8; MLP gate/up/down follow the dense hybrid rule (hybrid-int4 -> int4, hybrid-fp4 -> fp4). Same Matrix kernels/scratch/act_q8 path as dense; only norms/vectors/conv/A_log/dt_bias stay fp32.

Parity gate: dense bar is max abs logit error <1e-4 vs HF eager. Surjo fp32 decode/smoke runs (tiny shapes fixture, tied-head identity, Engine.logits/generate finite, spec rejection). HF modeling_surjo.py parity script and Surjo PPL are pending — no parity number claimed here.

Batch/paging: Phase0 files unchanged. Single-request serialized Engine generation lock, create_session/next_tokens, cancel/finish, lock_pages/touch/scan weight-memory controls are reused as-is. No continuous batching, no paged-KV layout, no new server path for Surjo.

FLA CPU verdict: decode needs no Flash-Linear-Attention (single-token GDN recurrence is O(1) state + per-slot causal XSA). Chunkwise/FLA prefill is an optional long-prompt throughput optimization only, not correctness; current sequential prefill stands.

Dense context (same autobench_135M_512.json, include_torch=False, NOT Surjo results): 4T K=0 fp32 55.34 / int8 130.03 / hybrid-int4 115.69 / hybrid-fp4 117.51 tok/s; best cells fp32 106.41 (K=8), int8 147.19 (K=2, accept 87.96%), hybrid-int4 140.59 (K=16), hybrid-fp4 132.90 (K=2); PPL fp32 34.15 / int8 34.74 / int4 42.07 / fp4 41.32. Ratios single-harness only; 4-bit vs int8 gaps sit inside ±15-20% noise. Surjo inherits none of these numbers.

Surjo-50m full AutoBench (Phase2, 2026-09-15, 4T K0)

File autobench_surjo50m_512.json (local snapshot 35c9fa6e, 512-token HF window, CISM greedy K0 only, act fp32, threads 1/2/4). Weights (engine.info): fp32 205.21 MB, int8 51.84 MB, hybrid-int4 43.27 MB, hybrid-fp4 41.86 MB; L3 32.0 MB, all fits_l3=false. PPL over the fixed AutoBench corpus (527 scored tokens): fp32 59.29 / int8 59.46 (+0.30% vs HF eager) / hybrid-int4 62.90 (+6.10%) / hybrid-fp4 61.80 (+4.25%). CISM int8/fp32 PPL matches HF eager fp32 59.28 to 0.3% — quant quality gate holds at int8, 4-bit pays +4-6%.

CISM decode at 4T K0 (greedy; CISM stopped on EOS at 43-48 tokens, HF forced 512 via min_new_tokens — lengths differ, ratios indicative):

Precision	1T tok/s	2T tok/s	4T tok/s	PPL
fp32	62.18	81.03	81.06	59.29
int8	129.86	158.39	207.80	59.46
hybrid-int4	109.05	159.04	175.49	62.90
hybrid-fp4	105.29	153.13	171.31	61.80

HF baselines (same harness, FP32 CPU, same thread count):

Decoder (4T)	Tok/s	PPL
HF eager fp32	26.27 (1T 28.08 / 2T 27.74 — inverse scaling, MT noise)	59.28
HF dynamic int8	18.18 (main cell 19.93; sweep 17.42/18.95/18.18)	66.35 destroyed (+11.9%)
HF torch.compile fp32	31.72 INVALID — StaticCache path, see below	not scored
HF compile int8	16.73 (same StaticCache invalidity)	not scored

Speedups at 4T K0 vs HF eager 26.27: int8 7.91x, hybrid-int4 6.68x, hybrid-fp4 6.52x, fp32 3.09x. speedup_vs_compile in the JSON divides by the invalid 31.72 and is void — do not cite.

Compile correction: modeling_surjo.py:645 discards non-SurjoCache past (if not isinstance(past, SurjoCache): past = None), so the autobench StaticCache loop allocated a fresh cache per forward call and state never carried — 31.72 tok/s is not a Surjo compile number. src/cism/autobench.py now detects surjo_mode and threads returned past_key_values (SurjoCache) under torch.compile(dynamic=True), recording {"skipped": "StaticCache invalid for Surjo"} if the threaded path cannot run — never a StaticCache tok/s. Correct path proven by scripts/bench_compiler.py: threaded SurjoCache + dynamic=True 41 tok/s (compile warmup 43-119 s across runs; 189.3 s in the old JSON was the invalid-path warmup); dynamic=False unusable (0.06 tok/s, recompiles per token as XSA KV length grows). torchao int4 blocked on Windows (no mslk wheel — Requires mslk >= 1.0.0); torchao int8wo is the CPU-working control only. FLA CPU dead here (Triton needs GPU; fla.ops.gdn2 chunk/fused probe does not run on CPU) — decode needs no FLA anyway (single-token GDN recurrence is O(1) state + per-slot causal XSA).

Spec: unsupported for Surjo hybrid (GDN state destructive) — K0 only (spec_note in JSON); K2≈K0 where probed, no speculation gain claimed. Native SurjoSession rejects spec_k!=0; Engine.stream(spec_k=2) raises.

Rigor: this JSON predates the harness (matrix rows lack rep_rates/thermal_throttled/cpu_snapshot). Current src/cism/autobench.py:_timed reports 1 warmup + median-of-reps (--reps/--runs wired through cism bench; default 3) with min/max/ std, thermal_throttled when max-min spread exceeds 15% (wider than the ±15-20% MT noise band), and best-effort cpu_snapshot (clock/temp; {} when unavailable, never fails the run). Next Surjo rerun picks up medians + thermal flags automatically; do not mix old single-sample cells with new median cells.

Surjo-50m rerun on current code (2026-09-15 night, 72 cells, reps=3)

autobench_surjo50m_512.json overwritten with current native (GDN-OPT + XSA-blocked + forward_block + spec verify). Threads 1/2/4 x spec 0/2/4/8/12/16, 512tok, act fp32, median-of-3, all thermal_throttled=false except two fp32-4T spec cells. CISM EOS-stops 43-48tok vs HF forced-512 — same caveat as before.

K0 medians: fp32 60.32/84.73/88.86, int8 141.76/197.34/234.87, int4 114.59/169.65/197.28, fp4 113.54/169.98/214.61. Best cells: fp32 K2 92.66, int8 K4 236.91, int4 K4 201.33, fp4 K0 214.61. Spec accept on bench prompt is 0.0 for fp32/int8 (no gain, no loss beyond noise); int4 6.25%->1.7%, fp4 8.3%->2.7% decaying with K, and spec K>0 is slower than K0 there (verify cost, no amortization win on this prompt). PPL unchanged: 59.29/59.46/62.90/61.80.

HF equal-opportunity, same harness, threads swept, median-of-3: eager 31.22/30.43/29.50 (4T 29.50 PPL 59.28), dynamic-int8 20.27/22.59/22.37 (4T 22.69 PPL 66.35), compile threaded-SurjoCache dynamic=True 38.77 warmup 100.6s, compile-int8 21.10, torchao-int4 failed Requires mslk>=1.0.0. Speedups vs eager 29.50: int8 K0 7.96x, best K4 8.03x; vs compile 38.77: K0 6.06x. Old invalid StaticCache 31.72 is superseded — do not cite it.

Supra-50M-Base full AutoBench (dense Llama, 2026-09-15 night)

File autobench_supra50M_512.json, Hub rev 521bfd3d3901fbeefa943a2f3d461a9d193c52ec, dense Llama (hidden 512, layers 12, heads 8/kv 4/head_dim 64, intermediate 1408, vocab 32000, ctx 1024, tied, use_cache=false preserved). Threads 1/2/4 x spec 0/2/4/8/12/16 = 72 cells, 512tok, act fp32, median-of-3. Weights: fp32 ~209MB / int8 ~53MB class (all fits_l3=false). PPL (527 scored tokens): fp32 63.32 / int8 63.79 (+0.7%) / int4 65.36 (+3.2%) / fp4 68.90 (+8.8%).

CISM K0 medians: fp32 108.29/131.54/123.55, int8 214.81/284.36/292.58, int4 178.04/247.49/250.87, fp4 171.67/234.54/251.38. Best cells: fp32 K8 268.19 (2T), int8 K12 349.10 (4T), int4 K2 257.43, fp4 K8 274.25. Accept on bench prompt is 100% for fp32/int8/fp4 at every K (hence spec pays: int8 4T K0 292.58 -> K12 349.10, +19.3%; fp32 K0 123.55 -> K8 268.19, +117%); int4 86%->70%.

HF same harness, threads swept, median-of-3: eager 48.59/47.90/43.61 (4T 43.61 PPL 63.32), dynamic-int8 37.70/40.49/37.78 (4T 34.91 PPL 101.68 destroyed), compile StaticCache dynamic=False 55.23 warmup 143.0s, compile-int8 33.14, torchao-int4 Requires mslk>=1.0.0. Speedups vs eager 43.61: int8 K0 6.71x, best K12 8.00x; vs compile 55.23: K0 5.30x, best 6.32x.

FWKV Myosotis-1-base full support + AutoBench (2026-09-15 night)

Files inspected in snapshot 031fe84a34f667bc8bfff7dc3f8a77ef3c17efb2: config.json (model_type fwkv, d_model 768, d_emb 192, n_layers 13, ffn_mult 4, vocab 50257, wkv_floor 0.1, tied), configuration_fwkv.py (FWKVConfig + HF aliases), modeling_fwkv.py (FactorizedTiedHead, FWKVBlock with proj_k/v/r/out + W recurrence + GELU FFN + 2x LayerNorm, _supports_cache_class=False, tuple-state threading), model.safetensors (147 F32 tensors, no rotary buffers).

Native support landed, mirroring the Surjo vertical slice: loader.validate_fwkv_config + _expected_shapes_fwkv (147 shapes) + _load_weights fwkv branch (also fixed an eager config["head_dim"] default crash affecting any arch without head_dim) + import_model dispatch; native FwkvConfig / FwkvModel / FwkvSession (WKV recurrence, exact GELU, LayerNorm eps 1e-5, factorized head, fp32/int8/int4/fp4 reuse via Matrix, forward_block batched projections + sequential recurrence, spec verify with ~40KB state checkpoint/restore); bindings FwkvModel/FwkvSession; engine dispatch; tests/test_fwkv.py 11 passed; scripts/parity_fwkv.py: tiny 1.19e-07, live vs HF fp32 5.72e-06 (<1e-4 OK). Autobench: trust_remote_code=True for fwkv HF loads, threaded-FWKV-state compile path (StaticCache invalid, same class of bug as Surjo), _hf_context fallback to seq_len (FWKVConfig has no max_position_embeddings).

File autobench_fwkv_512.json, 72 cells, reps=3 (v4 drafter rerun, 2026-09-15 late night; supersedes the v1-drafter run below). Weights: fp32 425.38MB / int8 107.64MB. PPL: fp32 156.68 (= HF eager 156.68, gate holds) / int8 156.80 (+0.07%) / int4 156.63 (-0.04%) / fp4 158.53 (+1.18%). High absolute PPL is base model quality, faithfully reproduced.

CISM K0: fp32 64.65/93.73/87.24, int8 169.01/268.22/342.27, int4 121.54/215.32/284.66, fp4 119.21/205.16/303.52. Best: fp32 K16 177.34, int8 K4 382.74, int4 K2 290.19, fp4 K2 305.98. Accept (bench prompt): fp32 85%->61%, int8 85%->62%, int4 77%->45%, fp4 74%->44% (K2->K16). Spec pays on fp32/int8: int8 4T K0 342.27 -> K4 382.74 (+11.8%); fp32 K0 87.24 -> K16 177.34 (+103%); 4-bit int4 still peaks near K0-K2, fp4 peaks K2.

HF same harness, threads swept, median-of-3: eager 42.72/45.59/43.42 (PPL 156.68), blocks-only dynamic-int8 34.74/43.88/51.26 (4T 51.22 PPL 158.42 +1.1% ok — fixed by quantizing blocks Linears only; full-model quantize fails on FactorizedTiedHead.proj.weight.t()), compile threaded-FWKV-state dynamic=True 49.73 warmup 45.9s, compile-int8 same path 46.93 warmup 36.2s, torchao-int4 Requires mslk>=1.0.0 (no Windows wheel). Speedups vs eager 43.42: int8 K0 7.88x, best K4 8.81x; vs compile 49.73: K0 6.88x, best 7.70x. Previous run (v1 drafter): K0 59.82/83.70/87.54 etc., best int8 K8 359.97 — v4 gains come from restored accept (19%->85% @K2).

Spec drafter v4 + doom-loop check (2026-09-15 late night)

v1 (most-recent single match, verbatim tail truncated at history end): Surjo bench 0% (correct text, nothing to match), Supra bench 100% (degenerate loop flatters it). v2 (pure frequency vote) collapsed on drifting text (FWKV bench int8 K2 85%->19%, K8 69%->9%). v3 (recency-decayed vote) same failure. v4 (landed): most-recent strictly-inside occurrence per position + iterative extension (fixes v1 truncation) + rejected-block latch REMOVED (the 32-propose/<1/3-accept latch fired during the short-history warmup and blinded the session to later 85%-accept regions; rejected blocks stream weights once per block via forward_block, so worst case on 0%-accept text is ~K0 speed). Shared by dense/Surjo/FWKV; tests/test_native.py::test_speculative_draft_hits_and_misses reimplemented against v4; suite 759 passed; K0==Kx exact everywhere probed.

Doom-loop probe (greedy, 128 tok, fp32+int8, 3 prompts): Surjo stops coherently on bench/code prompts (48/15 tok, uniq 0.73/0.93, loop 0) but loops story prompts (128 tok, uniq 0.09-0.16, 5/11-period loops); Supra loops ALL prompts (128 tok, uniq 0.07-0.11, 7/9/12-period loops — bench, story, AND code); FWKV never hard-loops bench/story (uniq 0.28-0.54, loop 0; int8 bench drifts into a 10-period body/brain cycle) but degenerates to 128 spaces on code (uniq 0.02 both precisions). So Supra's 100% spec accept is a looping artifact, Surjo's 0% is healthy text, FWKV sits between. No repetition penalty exists anywhere (greedy only); spec accept must always be read next to uniqueness.

Portable kernels: Intel/AMD tiers + ARM64 phones (2026-09-16)

Tier ladder (runtime-dispatched, scalar fallback always works): scalar (any CPU) < avx2+FMA (x86-64 baseline, all current numbers) < avx-vnni (Zen4/Zen5, Alder Lake+: int8 VNNI fast path) < avx512-vnni (server AVX512) / neon (ARM64 phones, Apple Silicon, Raspberry Pi). New files: native/kernels_vnni.cpp (dpbusd int8xint8, int8-activation ABI via quantize_row_i8), native/kernels_neon.cpp (fp32/int8/int4/fp4 + uzp permute), CPUID detection in native/kernels.cpp (AVX-VNNI = leaf7:1 EAX[4], AVX512-VNNI = leaf7:0 EBX[11] + foundation + ZMM state; MSVC VNNI TU is EVEX so it dispatches only on AVX512-VNNI, GCC VEX on AVX-VNNI), per-file codegen flags in CMakeLists (dispatcher/runtime stay generic). kernel_name() keeps the fp32 family; new kernel_variant() + has_avx_vnni / has_avx512_vnni / has_neon bindings; cpu_features gains avx_vnni; Model.info["kernels"] gains variant. tests/test_kernels_portable.py (5 passed): tier reporting, flag consistency, pre-VNNI fallback untouched, i8 quantizer round-trip bounds, dpbusd bias-identity in numpy.

Proven on Ryzen 5 5600: builds clean (MSVC /arch:AVX512 VNNI TU accepted), kernel avx2 / variant avx2 / vnni False, suite 764 passed (759 + 5 new) — zero behavior change, as designed (VNNI selector is null here so Matrix keeps the int16 Q8 / fp32-activation paths byte-for-byte). Pending hardware CI: VNNI tok/s + int8-activation PPL re-gate on Zen4/ADL+ (expect prefill/spec-verify wins ~2-3x integer MACs; decode stays bandwidth-bound so batch-1 gains concentrate in K>0), NEON tok/s + parity on ARM64 hardware (expects the usual Silicon/Android numbers; tails fall back to scalar by construction). No speedup is claimed for either until measured.