Differential Huffman recoding and provisional seeded codebook
Reproduce with python3 scripts/analyze_huffman_codebook.py; validate with python3 scripts/test_huffman_codebook.py. huffman-codebook.json retains source/disassembly/report/model hashes, native graphs, differential checks, modeled training checkpoints, tree/codewords and bounded native-instruction symbol round trips. The compact recoding is huffman_model.py.
The codebook is model-derived, not runtime captured or produced by evaluating every native training instruction. Differential checks compare complete sampled states against synthetic native instruction compositions, with the same stopped-call/frame/register-restoration limitations as adaptive updates. Original processes and native callees never execute. Full branch completeness, native runtime/ABI, other platforms and complete message/packet conformance remain open.
CODE-001 — Compact update recoding and tie-heavy differentials (RE-361)
The provisional model represents nodes, shared leader cells and freelist links directly. It recodes the observed insertion/increment behavior: byte-normalized lookup, NYT insertion, equal-weight next-node leader selection, tree swaps unless the leader is the parent, rank-list swaps and adjacency repair, previous-node leader retention or cell freeing, new weight-block sharing/allocation, recursive parent updates and post-parent adjacency/head repair.
Complete model states match all 22 previously recorded native-instruction sequence fixtures, including 152 retained prefix comparisons across both engines and their final states. Four additional sequence patterns exercise alternation, ascending sixteen bytes, repeated boundary-byte mixtures and ascending/reversed eight-byte orders. Each is compared after every update for both initialized objects and both engines: sixteen fresh cases, 288 native-prefix comparisons. Comparisons include all nodes, tree/list/lookup pointers, weights, shared heads, every allocated pointer cell and freelist, rather than just counts or encoded output.
Model event counts record new/existing entries, equal-next ties, tree/list swaps, leader-is-parent skips, old-head previous reuse, cell release/reuse, new-head sharing and post-parent adjacency/head changes. The matched sample corpus exercises each of these event classes. These counters belong to the recoding; they are not a branch-complete native coverage report or proof that every malformed/alias/capacity/tie path is equivalent. The model intentionally covers supplied valid byte trees, not general corrupted-object behavior.
CODE-002 — Both initialized objects (RE-362)
Fresh compositions reconstruct the source-matched sparse seed bytes of both object halves. The second starts at first +0xc830 and retains its initially nonzero pointer slot +24. The model retains this slot too, so absolute state comparisons preserve the initializer asymmetry.
Within the fresh corpus, the first and second halves generate matching normalized tree/count/rank/cell invariants and model event counts. Every prefix matches its respective native-instruction state with relocated addresses and its own tail-slot value. Outside object canaries remain intact. This establishes sampled update equivalence despite the retained slot difference, not full paired initialization lifetime, million-step native training, concurrent use, or runtime equivalence.
CODE-003 — Exact seed schedule and provisional codewords (RE-363)
The collector re-extracts the 256 frequency words directly from each verified ELF image and checks them against the frequency report. It trains the compact first-object model in the exact ascending-symbol repetition order, totaling 1,053,340 add operations. At every completed symbol it records a complete-state hash and validates independent leaf counts, parent/child reachability, internal subtree weights, rank ordering, leader targets and live/free cell partition.
The resulting modeled tree has 513 nodes and leaves for symbols 0–256, including NYT 256 with weight zero. Root weight is 1,053,340. It allocates 493 pointer cells, with 489 live and four free at the final checkpoint. These are model outputs, not measurements of the game's running allocator. The exported codebook has lengths from two through eleven bits and SHA-256:
a5a945498ab44f54d98b0b75346e03099e3adb212278f63e170b1a2eb57438c3
That digest uses sorted compact JSON of symbol/bits/length records; it is not the frequency-table hash or a native packet checksum. Bits are listed in root-to-leaf transmission order. The codebook is prefix-free and its full binary leaf set satisfies exact Kraft equality. Tests independently derive every code from the exported tree and verify leaf weights against the actual native frequency table.
Eight paired native-instruction encoder/decoder compositions then use this modeled tree as supplied memory. Cases are all 256 bytes, ascending first sixteen at bit seven, and repeated boundary-byte mixtures at bits zero/seven. Transmit/send/receive instructions produce matching symbols, variable codeword lengths/cursors and independent LSB-packed bytes with buffer canaries intact. The buffer is explicitly oversized; no capacity/truncation claim is made. This checks those helpers against the supplied modeled tree, not the accuracy of native full training or complete MSG_WriteBits/MSG_ReadBits/network packets.
The exported tree and codebook are research artifacts for subsequent comparison. Do not treat their provisional hash as a compatibility guarantee or replace the original frequency-training contract without validation.
Validation and next work
Eight tests check prior complete-state differential equivalence, fresh first/second-object prefixes and invariants, sampled model event classes, full seed leaf/subtree/reachability/count checks, checkpoint/cell summaries, independent tree-derived prefix-free codewords/Kraft equality, native-helper symbol packing/cursors/canaries and paired helper outputs. No shared evaluator extension was needed.
RE-364: prove individual native swap/tie/alias/error/capacity/wrap paths, validate full native-seed training/codewords against an independent authoritative implementation or permitted original-runtime captures, establish all-platform equivalence, and compose complete message writers/readers with actual packets, overflow/truncation and snapshot/renderer integration.
Message follow-up: byte writer/reader compositions connects the modeled tree to native message bit functions and cursor accounting (RE-365–367). Actual native tree training, full message widths and packet/runtime coverage remain open.