Publish 35.74 GB FP8/NVFP4 policy with held-out quality, exact export replay and generation checks
95e7465 verified | { | |
| "purpose": "Test conditional layer rankings and seek measured size/quality Pareto policies.", | |
| "primary_format": "Native-layout FP8 E4M3 with 128x128 FP32 scales; NVFP4 and limited BF16 are explicit alternatives.", | |
| "ranking_data": "24 new disjoint-source 1024-token windows; two 12-window panels, each four prose/code/math windows.", | |
| "text_holdout": "12 additional disjoint-source 4096-token windows; frozen before profiling, not used to rank layers.", | |
| "whole_model_interventions": ["Every MoE layer: quantize only that layer to FP8 in BF16", "Every MoE layer: restore that layer to BF16 in all-FP8", "Every MoE layer: downgrade that layer to NVFP4 in all-FP8"], | |
| "local_interventions": "All gate/up/down projections in all 47 MoE layers, at identical BF16-path and FP8-path inputs. Measure operator error and change from baseline.", | |
| "propagation": "Hidden error and expert-set changes at intervention, +1/+4/+8 layers where available, and final layer. Early/late end-to-end effects are not intrinsic local sensitivity.", | |
| "controls": ["Identical BF16 execution repeats bit-exactly", "Step implementation matches existing fused reference at dense and MoE layers", "Same-input changes preserve router IDs", "All measured processes have zero swap"], | |
| "reporting": ["Per-domain and per-panel metrics", "Sequence-cluster bootstrap intervals, not independent-token intervals", "Rank replication and bootstrap rank uncertainty", "Exact tensor bytes and precision fractions, including unquantized non-experts"], | |
| "footprint_budgets_GB": [28, 32, 36, 40, 44, 48], | |
| "anchors": "All-NVFP4 approximately 25.16 GB; all-FP8 approximately 41.73 GB; source BF16 as accuracy reference only.", | |
| "policy_comparisons": "Conditional-rank allocation vs prefix/suffix and fixed-seed random allocations at matched tensor sizes. Evaluate complete policies; marginal costs are proposal heuristics.", | |
| "final_validation": "Freeze promising non-dominated policies before longer-context and new answer-level task validation. Report fidelity and task accuracy separately; do not call small uncertain differences improvements.", | |
| "publication": "Keep current FP8 weights unchanged unless an improved candidate is independently validated; report precision mixture literally.", | |
| "limitations": "Decoded-weight quality reference, not native-kernel or production-throughput qualification. This cannot exhaust every precision assignment." | |
| } | |