Mamb2_8B_FP4_Recall: audited paired results
Full official WikiText-2 TEST PPL is 6.717746660814748 with final Resurface, versus6.717800294314242 for its paired repaired base. Normal CONFIRM recall is 353/384(91.9270833%), versus131/384(34.1145833%). All four fixed quality conditions and both combined CPU audits pass.
This portable result document describes the exact repaired FP4 G16 + native
FP16-state model in this bundle. Relative evidence links resolve from the
bundle or source repository. It is derived from the frozen research results
document SHA256
738075edd8d6d1edc3ca88dd325c8c36b6567b445b9f7f1e77c9b634b4dc7c9a;
that original document remains unchanged and is recorded as source provenance.
The frozen adapter protocol
and base-repair protocol govern the experiments.
Final same-process comparison
| Arm | Full WT2 TEST PPL | Pooled target NLL | Normal MK / 384 | Target-removed / 384 |
|---|---|---|---|---|
| Adapter off | 6.717800294314242 | 573262.5137534142 | 131(34.1145833%) | 0 |
| Final Resurface on | 6.717746660814748 | 573260.1109187603 | 353(91.9270833%) | 0 |
| Adapter removed | 6.717800294314242 | 573262.5137534142 | 131(34.1145833%) | 0 |
| CONFIRM stratum | Adapter off | Final Resurface | Adapter removed |
|---|---|---|---|
| N16 normal / 192 | 101(52.6041667%) | 191(99.4791667%) | 101(52.6041667%) |
| N64 normal / 192 | 30(15.625%) | 162(84.375%) | 30(15.625%) |
| N16 target removed / 192 | 0 | 0 | 0 |
| N64 target removed / 192 | 0 | 0 | 0 |
Normal paired gain is 222/384 = 57.8125 percentage points. There are 225
improvements, 3 regressions and 156 unchanged cases. The paired10,000-draw
bootstrap, seed 20260928, gives95% interval
[0.5260416666666666, 0.6276041666666666], or
[52.6041667,62.7604167] percentage points.
PPL is nearly unchanged: absolute change-0.000053633499494054604, relative change-0.0007983788910648215%. Pooled NLL is lower by 2.40283465385437, or0.000007983820781472707 nats per target. Of147 windows, 83 have lower NLL and 64 have higher NLL. The pooled no-worse gate passes; this tiny score change does not establish a broad language-model improvement.
| Predeclared condition | Outcome |
|---|---|
| Full test adapted PPL no higher than the paired repaired-base PPL | PASS |
| Positive normal-MK correct-count gain | PASS: +222 |
| Paired95% lower confidence bound strictly positive | PASS |
| Exact adapter-removed147 window scores and 768 generated sequences | PASS |
The full comparison records the individual quality flags, restoration receipt, package/weight identity and memory scope. All147 baseline/removed window NLLs, PPLs and target counts are exactly equal. All768 generated-token sequences, outputs, predictions, scores and cache receipts restore exactly. Native forwards/hooks, frozen parameter identities, reset/prefill/one-step hidden/cache probes and repeated-reset checks pass.
Data and execution protocol
PPL uses official Salesforce/wikitext, wikitext-2-raw-v1, test, revision
b08601e04326c79dfdd32d625aee71d232d685c3, two-newline document joins,
the original tokenizer and no automatic BOS/EOS. All300,963 next-token targets
are scored in 147 reset windows of up to2,048 targets, including the final
1,955 targets. The300,964-token stream has SHA256
5b82bd46e833e77fcfc0af62bafeaac62e70e68cfdf214d375f0b7b132d4b608.
Pooled PPL is exp(total NLL / 300963). Native parallel SSD prefill resets
each window; full-vocabulary head GEMMs feed FP32 cross entropy in 64-token
chunks. There is no persistent PPL cache or StateQuant requantization.
MK uses exactly 768 CONFIRM prompts:384 normal +384 with the queried binding
removed. N16 and N64 each have192 prompts per condition. Greedy decoding
emits at most12 tokens, stops at EOS, and checks the first standalone six-digit
answer. The canonical CONFIRM SHA256 is
445f1ebae5cfeeafccc0b6c02f1a514d8d7858360637af9ec37ab5e48212a091.
Target-removed matching is a removed-value control, not successful retrieval
from the prompt. Its zero result does not prove general hallucination resistance.
The GPU run used NVIDIA RTX PRO6000 Blackwell Server Edition, Python3.10.12,
Torch2.11.0+cu128, CUDA12.8, mamba-ssm2.3.2.post1, Triton3.6.0,
NumPy1.26.4, datasets4.8.5 and sentencepiece0.2.1. Singleton warp16 RMSNorm
was pinned before model loading; TF32 was off for matmul and cuDNN, FP32
matmul precision was highest, and eight CPU threads were used. Backend
and imported source identities stayed fixed through the paired run.
Fixed repaired base and fresh adapter
The base is pure Mamba2-8B:56 blocks, 8,236,999,680 parameters and 507 tensors. The114 large matrices use E2M1 FP4 codes with 16-element input-axis groups, E4M3FN block scales and FP32 matrix scales. The393 existing small tensors are FP16. Base repair provenance binds TRAIN finetuning of those 393 tensors at selected step 448; the 114 FP4 matrices and 118 FP4 shard files retain their original identities and storage.
Base repair completed 896 successful updates over two fixed passes through 448 WT2 TRAIN windows. The fixed224/448/672/896 candidate pool chose448 by minimum full-validation NLL, before its final test. That earlier base-only test comparison recorded PPL6.717854132348465; this adapter experiment independently reloads and rescores the same base at 6.717800294314242. The gains above use only the new same-process pair. The original FP16 control in base repair was an unfinetuned source reference, not a matched post-training control.
All114 FP4 and 393 repaired FP16 tensors were frozen during fresh Resurface training. The separate teacher loads the same repaired, unadapted base. Only224 adapter tensors /1,154,104 parameters are learned. Fresh post-D soft gates initialize V=0, g=1, router_w=0, router_b=-4. Numeric TRAIN CE1, WT2 TRAIN prose CE0.5, repaired-base teacher KL0.5 and prose gate closure3 use native SSD forward/backward, without a StateQuant controller or surrogate state-gradient replacement.
The formal training report records1536 successful updates /1542 attempts /6 overflow retries and checkpoints384, 768,1152 and 1536. Only the final1536 export entered quality evaluation; no PPL/MK adapter checkpoint selection occurred. All224 final FP16 tensors equal saved FP32 masters cast to FP16; optimizer/scaler/checkpoint roundtrip receipts pass. Independent teacher/student storage, 507 actual value checks at both endpoints, initial/export/removal parity at 128/512 tokens and native student/teacher restoration pass.
The initial smoke stopped before loading a model or performing an update: backend pinning's Quack hardware query initialized CUDA before a source-closure ordering assertion. The corrected trainer binds source first and rechecks it after backend setup, with protocol/objective/loader unchanged. A fresh one-update retry passed1 update /1 attempt / 0 overflows and 128/512 parity; its export was discarded. Formal training then restarted with fresh masters, optimizer and scaler.
Formal elapsed time is 1780.7513492107391 seconds (finished_unix-started_unix),
including loading/export/parity work. Full paired evaluation took
1505.3940229415894 seconds. These are job durations, not isolated throughput.
Storage and CUDA memory
| Explicit component | Bytes |
|---|---|
| Encoded base weight tensor payload | 4,638,460,360 |
| Actual119 base weight safetensors files | 4,638,539,864 |
| Final adapter FP16 tensor payload | 2,308,208 |
| Final serialized adapter | 2,381,119 |
| Base + adapter encoded tensor payload | 4,640,768,568 |
| Actual119 weights + adapter file | 4,640,920,983 |
| Decoded resident FP16 base weights | 16,473,999,360 |
| Batch1 MK native SSM cache | 117,440,512 |
| Batch1 MK native convolution cache | 4,587,520 |
| Total batch1 MK cache, 112 distinct FP16 tensors across56 layers | 122,028,032 |
| Added adapter recurrent cache | 0 |
The combined file total excludes metadata, tokenizer, code, licenses and evidence. Decoded base weights, one MK cache and adapter FP16 payload sum to16,598,335,600 bytes; this is a component sum, not total CUDA memory. PPL's native SSD internal accumulator precision is separately scoped from the inspected FP16 MK caches. The native reference expands all weights to resident FP16; file compression does not demonstrate packed-VRAM inference.
| Observed CUDA scope | Peak allocated bytes | Peak reserved bytes |
|---|---|---|
| Formal training, two expanded bases and adapter training | 34,600,171,520 | 34,781,265,920 |
| Evaluation baseline | 16,953,243,136 | 16,968,056,832 |
| Evaluation Resurface | 16,864,286,208 | 16,970,153,984 |
| Evaluation adapter removed | 16,849,382,912 | 16,970,153,984 |
Evaluation peaks are sequential one-process measurements containing temporary buffers and allocator history. Their differences do not isolate adapter memory or show an adapter-driven VRAM reduction. No throughput or ASIC result is inferred.
Bound evidence and independent audit
| Compact evidence | Bytes | SHA256 |
|---|---|---|
| Comparison | 235,203 | f16227b149e1aa9c7113a35cd56ed1f95849ae6bdc6b7e1460d077a4847d91cb |
| Baseline | 3,868,006 | ec407fad97262a991415c0cb1855d9ea1e67d118981a7f6ae7809561ee0b2a82 |
| Resurface | 3,912,805 | 04047a78097be9412d4038886749d12721866d282828fa64e5c321608e1225a7 |
| Removed | 3,868,013 | 6e8a801ebe2632cae6426e66a22dea6f4ef7ea00a52ffec2fc727680ece91be8 |
| Formal training | 1,084,073 | 84c4f5751726e035300f42ddaa63eae8faa748f192e3392a0c65626e42dda9ce |
| Training audit | 1,905 | a73d1a265608d203d05342c67b3790593e522bafda404654dd3c97a6364941a1 |
| GPU-host combined CPU audit | 3,935 | d71975e3b80e5e4c5e7fda8fc54cc7eb98c09c1fb17c2b2531316b6ed4b89a4b |
| Mac combined CPU audit | 3,934 | f3600b884bb60268dd8bf27c9651fe21cb8b93daac3046011b9e007577e15dae |
The repaired weight manifest is
2d2208b690c2ec0def90578d8590cc122bb1e4542b0d49ae4417a48f72b0b73c.
The final adapter is
dca3cda5a9ab8a665abd1a355c6e5b294acee023d2a1a5a8e20f4e84c9674580.
The fixed base selection is
2f1ff559f3e907b3803bcc6d7036d39dab88c11dbb5e69f1de57e2dff5ca452c.
The adapter protocol is
9592a0dddcaa5c7d000422e5ac902b49df362781e4a4d9f668d99cb849cf8a7f;
trainer/evaluator/auditor sources are bound by the reports and release manifest.
Both combined audits check complete reports, adapter/checkpoint file hashes, TRAIN schedule/retry accounting, repaired507 identities, native parity/cache receipts, saved token/window identities, all PPL/MK arithmetic, bootstrap and exact removed-arm restoration. GPU-host audit used NumPy1.26.4; Mac used NumPy2.3.5 and reproduced the same confidence interval. Neither CPU audit imports Torch or recomputes GPU logits/hidden states. The GPU evaluator streamed actual packed shards through the strict loader and checked all 507 decoded values at load, after each arm and at completion; CPU audits bind those recorded execution proofs. Torch's recorded shuffle is checked for completeness and retry consistency, not independently regenerated.
Optimizer/checkpoint archives, full prose/token corpora and private job logs are not part of the model download. The published audits record archive checks already completed on the research artifacts; the download alone cannot repeat the full training-archive audit. Minimal reproduction provides complete-file verification, one-case strict recall smoke, prompt generation and optional fresh full three-arm PPL/MK replay.
Interpretation and licenses
WT2 validation/test and CONFIRM have prior project exposure, so they are not untouched project holdouts. TRAIN optimization and fixed candidate selection are separate from evaluation, but independent auditing does not remove that historical exposure. Pretraining contamination is unaudited. These results apply to the fixed measured protocol; longer contexts and other benchmarks are unmeasured.
Read LICENSES.md and third-party notices: the modified base is Apache-2.0, while the project runtime and this new adapter are GPL-3.0. No Quamba or research-only quantized checkpoint is used.