- epic_quant/eval/smoke.py: CPU-runnable smoke test that chains the
engine's forward_one_layer through all 42 layers on a fixed prompt
and reports top-10 next-token predictions. Validates the engine
produces a coherent forward end-to-end on real E4B weights.
- epic_quant/eval/kaggle_notebook.ipynb: full eval sweep for a Kaggle
GPU box (T4 or P100). Three evals:
- WikiText-103 PPL (50K tokens, 512-token chunks)
- MMLU Pro 4-shot accuracy (200-question subsample, loglik over A/B/C/D)
- MRCR v2 8-needle retrieval (8K context; 128K OOMs on T4)
Compares 1.58 / 3 / 4 / 16-bit policies against the published
Gemma 4 E4B 69.4% MMLU Pro and 25.4% MRCR v2 8-needle 128K.
Smoke test ran on this 13.8 GB RAM / CPU box:
- 16-bit: 1.37 s/layer, top-1 token id 26069
- 3-bit: 2.50 s/layer, top-1 token id 26069 (matches)
Top-1 is robust; top-10 overlap is 3/10, consistent with the
L2 recon numbers.
Add 1.58/3/4/16-bit sweep, packed weights, real SDPA, COMPARISON.md
- New: 2-bit / 3-bit / 4-bit / 16-bit packed weight formats in packed.py
with real on-RAM byte accounting.
- New: --sweep mode in bench.py runs four policies and emits
sweep.json.
- New: build_report.py turns sweep.json into a markdown comparison
table (COMPARISON.md).
- forward.py now uses F.scaled_dot_product_attention with a 4D
additive mask (causal + sliding-window for sliding, causal-only
for global). Replaces the prior O(seq^2) einsum.
- Diagnostic scripts moved to scripts/ to keep root clean.
- LICENSE: Apache 2.0, matching Gemma 4 E4B upstream.
- README rewritten as the canonical repo entrypoint pointing to
WRITEUP.md and COMPARISON.md.
Headline finding: the brief's 1.58-bit ternary on sliding attn
gives L2 recon > 1.0 (broken); 3-bit drops to 0.29 for +114 MB.
3-bit is the realistic floor.
Epi-Stochastic Predictive Fetching & Context-Aware Bit-Shifting.
Three pillars: attention bit-discarding, PLE sparse caching, KV eviction.
Full WRITEUP.md with measured numbers on real E4B safetensors.
