code/docs/PROTOCOL.md

Independent W4A16 Mamba2-8B Resurface protocol

Declared 2026-09-28 before quantization, training or quality evaluation of this repository's independent W4 candidate. The user requires redistributable weights. No Quamba weights, rotation, implementation or adapter are dependencies.

Frozen model and quantization

Source: NVIDIA nvidia/mamba2-8b-3t-4k, Apache-2.0, revision b915550c63ba9359f88f44d1f6a600d85af27302, checkpoint SHA256 47c2766f6aad89d73beafbeaecb334aab902d7370906d081764a90bb7a8bbbcb; tokenizer SHA256 5862e2f71caf762bc9845662be5fec2867deb58d874568235a02a36c5111cd09. This is the pure 56-layer Mamba2-8B model, with untied 256K embedding/head.

Independently quantize all 114 large matrices (56 input projections, 56 output projections, embedding and head) to affine uniform four-bit codes. Groups contain 128 adjacent input-axis weights. Each group stores a FP16 scale and FP16 offset; two codes are packed per byte. Metadata adds 0.25 bits per quantized weight, so these matrices require approximately 4.25 bits/weight before file headers. The other 393 small tensors remain FP16. Do not describe every parameter as INT4.

For each group, fit using weight-space MSE only, over fixed range factors 1.00, .99, .98, .97, .96, .95, .94, .92, .90 around its min/max midpoint. Compare actual serialized FP16 scale/offset and FP16 reconstruction. No prose, MK data, validation labels or source activation data enter quantization. This fixed quantizer defines one candidate; no evaluation-based quantizer search. Constant/zero groups, nonfinite inputs and truncated files must be handled explicitly. Write/read actual packed files, verify exact reconstruction and per-file hashes, and bind the resulting manifest to both training and evaluation.

Quality execution decodes the saved W4 package into frozen FP16 native weights. This is a W4A16 numerical reference, not a packed-resident GPU kernel. Native FP16 residual orchestration and FP16 state cache match our existing source control. The source BF16-to-FP16 runtime is not asserted to reproduce Megatron BF16 exactly. No rotation, residual weight correction or small-tensor readaptation is included.

Use the same post-D adapter as the compressed experiment at all 56 native gated RMSNorm inputs: y + sigmoid(w dot u+b) * g*(V@y) across 128 heads. Its 224 tensors contain 1,154,104 FP32 trainable master parameters, exported as FP16 for inference. Initialize V=0, g=1, w=0, b=-4; use soft sigmoid at train and eval, no task switch, EMA, or new recurrent cache. The adapter is external to the frozen base; check its identities and absence of base grads.

Training

Reuse the same exact data generator contract and seeds as the compressed adapter: 1,536 TRAIN numeric bindings over three templates and N=16/64, disjoint six-digit key/value intervals; full 256K CE on every answer suffix token. Use seed 2026092803 and torch.randperm(1536) once for example order. Each step pairs one MK example with a 512-token segment from the already prepared 448 WikiText-2 TRAIN windows. Their historical manifest and file SHA256 are pinned by the train command. Heldout and validation windows are not included in training. Reusing this fixed TRAIN text gives the control the same training exposures as the compressed arm.

Use a second, independently loaded unadapted W4-decoded FP16 model as the frozen prose teacher. This is the same own-unadapted-base teacher policy as our previous experiments. The fixed per-step loss is MK answer CE + 0.5 prose CE + 0.5 KL(own unadapted teacher || student) + 3 C, where C is the same prose-only router closure, with budget 0.006 and excess coefficient 10. Temperature 1, 511 prose targets per step, 64-token staged head chunks, and one optimizer update after both task gradients.

AdamW FP32 masters: V/g LR 1e-4, w/b LR 3e-4, betas (.9,.999), eps 1e-8, weight decay 0, clip norm 1. Multiplicative schedule 0.1+0.9*(1+cos(pi*j/1535))/2 at successful index j. Native FP16 forward, block checkpointing, gradient scale 1024 with growth interval 2000. Overflow retries the exact same pair without optimizer/master update, at most 8 retries and 1,544 total attempts. Run 1,536 successful updates; the final step is the only candidate. Small GPU smokes are discarded before formal training.

Evaluation and interpretation

After training, restore the actual serialized FP16 adapter onto the same W4-decoded base. Evaluate baseline, adapter enabled and restored baseline in a single process, with identical tokenizer, MK prompts and full WikiText-2 validation windows. For MK use full 256K greedy generation up to 12 tokens, first standalone six-digit number, normal and target-removed controls, native prefill plus recurrent decode with fresh FP16 cache. For PPL use all 130 nonoverlapping reset windows and all 264,764 next-token targets. Record all raw scores, differences and file hashes; no early selection or task switch.

The earlier compressed arm's independent CONFIRM set has now been observed and uses the same three template families. Running this W4 experiment on it gives a matched historical comparison, not a newly untouched holdout. The WikiText-2 validation text also informed earlier development. We will not promote a W4+adapter result to a general recall conclusion without new templates, longer distances and a truly untouched corpus.

Report the new W4 base and W4+adapter together, with matched source FP16 historical metrics clearly identified as historical. Verify prompt/window hashes before comparing them. Report PPL and recall together; no packed-runtime speed or memory claim follows from these quality measurements.