Recover message initialization and mixed-width codec contracts

docs/ASSET_PIPELINE_TODO.md

@@ -226,3 +226,5 @@ Adaptive-codec follow-up: [update compositions](research/HUFFMAN_UPDATES.md) val
 Codebook follow-up: [provisional modeled codewords](research/HUFFMAN_CODEBOOK.md) adds exact native seed training in a differentially checked recoding. Full native/runtime codeword and message/packet validation remains required before network/map/renderer integration (RE-364).
 
 Message-codec follow-up: [modeled-tree byte streams](research/MESSAGE_BYTE_CODEC.md) validates native message/helper compositions and byte accounting. Full native codebook, general widths and real packet/buffer contracts remain prerequisites for network/map/renderer integration (RE-368).
+
+Research progress: [message width codecs](research/MESSAGE_WIDTHS.md) adds bounded record/OOB/mixed-width contracts (RE-369–371). RE-372 leaves wrappers, lifecycle, real packet/runtime/platform and native seeded-codebook validation open; no network recoding/conversion is declared complete.

docs/FINDINGS.md

@@ -408,3 +408,9 @@ Bounded qagame paths recover bomb-plant/inactivity correction requests, direct f
 **F-134 — Observed value diagnostics:** range failures increment a separate global counter without stopping the sampled path. Compressed width eight masks to low byte; selected negative-width bounds use pre-normalization x86 shift counts. General signed/OOB/wrapper and legal-caller behavior remain open. [BYTE-002](research/MESSAGE_BYTE_CODEC.md).
 
 **F-135 — Observed modeled-tree message byte composition:** native writer/reader/helper instructions agree on exact compressed bytes and per-symbol cursors using supplied model-derived trees. Size/readcount use bit/8 +1, including an extra byte at exact boundaries. Native trained-codebook, real packet/padding/overflow/runtime/platform conformance remains open. [BYTE-003](research/MESSAGE_BYTE_CODEC.md).
+
+**F-136 — Observed message record lifecycle:** warm initialization clears the 40-byte ELF64 record, installs data/maxsize and sets OOB mode when requested. Cold initialization sets msgInit before requesting Huff_Init. Clear retains readcount/mode/data/maxsize while zeroing overflow/size/bit. Full training/lifecycle/platform behavior remains open. [WIDTH-001](research/MESSAGE_WIDTHS.md).
+
+**F-137 — Observed OOB composition:** accepted 8/16/32-bit writes store low bytes little-endian and advance size/bit directly; selected negative widths normalize, while -32 is rejected by writer validation. Matching reads retain byte order and signed tails. Capacity/EOF/real-header/runtime coverage remains open. [WIDTH-002](research/MESSAGE_WIDTHS.md).
+
+**F-138 — Observed mixed-width composition:** compressed writes emit raw low-bit remainder then modeled-tree Huffman bytes. Independent packing and native reader compositions agree for every positive width 1–32 and selected negative widths; negative sign tails use the mutated remaining byte width. Native seed/codeword, legal-caller, buffer and real-packet/platform conformance remain open. [WIDTH-003](research/MESSAGE_WIDTHS.md).

docs/PROVISIONAL_TODO.md

@@ -538,3 +538,5 @@ Research progress: [adaptive Huffman updates](research/HUFFMAN_UPDATES.md) compl
 Research progress: [differential Huffman codebook](research/HUFFMAN_CODEBOOK.md) completes bounded RE-361–363. Broad network/runtime milestones remain open; RE-364 tracks full native-training/codeword conformance and complete real packet/platform contracts.
 
 Research progress: [message byte codec](research/MESSAGE_BYTE_CODEC.md) completes bounded RE-365–367. Broad network/runtime milestones remain open; RE-368 tracks general message/snapshot codecs, overflow/truncation/padding and real native packet/platform validation.
+
+Research progress: [message width codecs](research/MESSAGE_WIDTHS.md) completes bounded RE-369–371. Broad network/runtime milestones remain open; RE-372 tracks wrappers/delta/snapshot codecs, message lifecycle and actual packet/platform validation.

docs/RESEARCH_LEDGER.md

@@ -229,3 +229,7 @@ Completed bounded RE-361–363 with `huffman_model.py` and `analyze_huffman_code
 ## 2026-10-11 — Message writer guards and modeled-tree byte codec
 
 Completed bounded RE-365–367 with `analyze_message_byte_codec.py`: ten verified graphs across the Linux ELF64 engines, 576 writer guard/range probes and 36 compressed byte streams. BYTE-001–003/F-133–135 distinguish requested-width oldsize accounting from compressed bit lengths, the early signed space guard from later width errors, the global range diagnostic from message overflow, and pre-normalization negative-width shifts from ordinary signed-range expectations. Byte streams compose writer/transmit/send and reader/receive against relocated MODEL-DERIVED seed trees; independent codeword packing verifies payloads, decoded low bytes and every per-byte cursor. Exact byte boundaries retain an extra size/readcount byte, zero only because these supplied buffers are initialized to zero. Save/restore omission, stopped CALL boundaries and explicit caller-register restoration remain documented; no original process or native callee executes. Eight tests verify guard priority/accounting, width normalization, signed range diagnostics, independent payload/cursor packing, low-byte conversion/canaries, extra-byte boundaries, helper arguments and paired results. All 342 tests pass. New and preceding reports regenerate byte-identically; storage, links and whitespace checks pass. RE-364 remains open; RE-368 tracks general raw/signed/multi-byte/OOB and wrapper/snapshot codecs, capacity/overflow/reset/truncation/EOF/padding invariants, real native full-training codewords and actual packet/original-runtime/all-platform/snapshot/renderer contracts.
+
+## 2026-10-11 — Message initialization, OOB and mixed-width codecs
+
+Completed bounded RE-369–371 with `analyze_message_widths.py`: twenty verified graphs across the Linux ELF64 engines, 24 initialization probes, four clear probes, 78 OOB probes and 328 compressed writer/reader compositions. WIDTH-001–003/F-136–138 distinguish warm 40-byte record reset from the cold first Huff_Init boundary, clear's overflow/size/bit reset from retained readcount/mode/data/maxsize, and direct OOB byte accounting from compressed size/readcount. Accepted OOB widths emit low bytes little-endian; all positive compressed widths 1–32 and selected negative widths compose raw-bit remainder followed by modeled-tree Huffman bytes. Independent packing verifies exact payloads and cursors; generated writer bytes confirm the reader's mutated-width signed tail. The seeded tree remains MODEL-DERIVED. Prologue/epilogue omissions, stopped CALL boundaries and explicit caller-register restoration are documented; no original process or native callee executes. Eight tests verify initialization/cold boundary/canaries, clear retention, OOB errors/endian/sign/cursors, raw/symbol requests, independent packing, signed tails/diagnostics/accounting, paired outcomes and narrow MOVW/BX/implicit-one shift evaluator support. All 350 tests pass; new and preceding reports regenerate byte-identically, with storage/link/whitespace checks passing. RE-368 remains open; RE-372 tracks wrappers/string/float/angle/delta/entity/player/snapshot codecs, capacity/overflow/allowoverflow/reset/truncation/EOF/prefix/padding/legal-width invariants, native seeded codewords and actual packet/runtime/all-platform/renderer contracts.

docs/REVERSE_ENGINEERING_TODO.md

@@ -442,3 +442,8 @@ Purpose: turn the supplied installation into an evidence-backed specification of
 - [x] RE-366 Recover sampled signed value-range diagnostic gates, negative-width normalization and compressed byte masking ([BYTE-002](research/MESSAGE_BYTE_CODEC.md)).
 - [x] RE-367 Compose modeled-tree native message byte writes/reads with exact payload, helper arguments, size/readcount and padding oracles ([BYTE-003](research/MESSAGE_BYTE_CODEC.md)).
 - [ ] RE-368 Recover general raw/signed/multi-byte/OOB and wrapper/snapshot codecs, compressed capacity/overflow/reset/truncation/EOF/padding invariants; verify native full-training codewords and actual packet/original-runtime/all-platform/snapshot/renderer contracts.
+
+- [x] RE-369 Recover warm message-record initialization, cold first-helper boundaries and exact clear-field retention ([WIDTH-001](research/MESSAGE_WIDTHS.md)).
+- [x] RE-370 Compose bounded little-endian OOB writes/reads, width dispatch/errors and byte/bit cursors ([WIDTH-002](research/MESSAGE_WIDTHS.md)).
+- [x] RE-371 Compose positive widths 1–32 and selected signed compressed writes/reads through raw-bit and modeled-tree Huffman helpers ([WIDTH-003](research/MESSAGE_WIDTHS.md)).
+- [ ] RE-372 Recover message byte/short/long/data/read/string/float/angle/delta/entity/player wrappers and snapshot codecs; validate capacity/overflow/allowoverflow/reset/truncation/EOF/dirty-prefix/padding/legal-signed-width invariants, native seeded codewords and actual packet/runtime/all-platform/renderer contracts.

docs/research/MESSAGE_BYTE_CODEC.md

@@ -37,3 +37,5 @@ Buffers are explicitly oversized to 4096 bytes with outside canaries. The guard
 Eight independent tests verify guard priority/oldsize, valid width/normalization, signed range counters, codeword-derived payload and per-byte cursors, low-byte diagnostic behavior, exact-boundary extra bytes, helper arguments/canaries and paired engine results. No shared evaluator extension was needed.
 
 RE-368: compose general raw/signed/multi-byte and OOB writer-reader paths, wrappers and actual snapshot messages; recover compressed capacity/overflow/reset and truncation/EOF/bit-cursor invariants, verify the real native trained codebook and buffer padding, then validate original-runtime/all-platform and packet/snapshot/renderer contracts.
+
+Follow-up: [message width codecs](MESSAGE_WIDTHS.md) composes bounded OOB writes/reads, all positive widths 1–32 and selected negative compressed widths, plus record initialization/clear. The broader RE-368 remains open; RE-372 tracks wrappers, lifecycle and real packet/platform validation.

docs/research/MESSAGE_WIDTHS.md

@@ -0,0 +1,67 @@
+# Message initialization, OOB stores and mixed-width codecs
+
+Reproduce with `python3 scripts/analyze_message_widths.py`; validate with `python3 scripts/test_message_widths.py`. [message-widths.json](message-widths.json) retains twenty source/disassembly-hash-verified graphs across both Linux ELF64 engines, 24 initialization probes, four clear probes, 78 OOB probes and 328 compressed writer/reader compositions. It hashes the supplied [model-derived codebook report](HUFFMAN_CODEBOOK.md).
+
+These are synthetic native instruction compositions over explicitly supplied memory/registers, with caller-register restoration at stopped helper boundaries. Save/restore prologues and epilogues are excluded. No original process or native callee executes. The compressed seed tree is MODEL-DERIVED; complete native seed training, real packets and runtime/all-platform conformance remain open. Supplied buffers are oversized and zero initialized, not evidence of the original buffer lifecycle.
+
+## WIDTH-001 — Message record initialization and clear (RE-369)
+
+The warm MSG_Init/MSG_InitOOB path is evaluated with msgInit already one. Each zeroes the entire 40-byte record, writes the supplied 64-bit data pointer at +16 and supplied maxsize word at +24. MSG_InitOOB then sets the mode word at +8 to one; ordinary initialization leaves zero. Size probes retain supplied size words 0, 64 and 0xffffffff without validation in this slice. This does not establish valid caller capacities.
+
+The recovered local ELF64 layout is:
+
+| Offset | Bytes | Observed use |
+| --- | --- | --- |
+| +0 | 4 | Cleared initialization word; allowoverflow semantics remain to be verified |
+| +4 | 4 | Writer overflow flag |
+| +8 | 4 | OOB mode word |
+| +12 | 4 | Cleared padding/unidentified record bytes |
+| +16 | 8 | Data pointer |
+| +24 | 4 | Maximum size |
+| +28 | 4 | Current size |
+| +32 | 4 | Read count |
+| +36 | 4 | Bit cursor |
+
+Cold probes start with msgInit zero and stop at the first Huff_Init request. They observe msgInit set to one before that request, while the supplied message record still retains its original 0xa5 bytes. Cold full training and subsequent record initialization are not composed here; see [initialization analysis](HUFFMAN_INITIALIZATION.md) for bounded schedule evidence.
+
+MSG_Clear zeroes only overflow +4, cursize +28 and bit +36. It retains readcount, mode, maxsize, data pointer and the other record bytes. Record canaries remain unchanged. Buffer contents and complete clear/reset/read-mode lifecycle remain open.
+
+## WIDTH-002 — OOB writer/reader composition (RE-370)
+
+Each engine has 36 canonical-mode probes: widths 8/16/32/-8/-16/1/7/24/-32, value words zero/0x8196a5ff and starting cursize zero/three. Three more probes use mode word two with positive 8/16/32; this sampled nonzero value takes the same OOB dispatch. Initial bit cursor is deliberately 19, distinct from byte-position accounting.
+
+With ample remaining capacity, accepted widths normalize to 8, 16 or 32. The writer stores the input's low one/two/four bytes at data + signed cursize in little-endian order, increments cursize by the byte count and increments bit by the normalized width. There is no extra padding byte from the compressed cursor formula. Positive and selected negative requests update oldsize by the original requested word; range diagnostics remain separate from message overflow.
+
+Valid-but-unsupported OOB widths 1/7/24 stop before the error epilogue, without changing size, bit or buffer. The error callee and its final arguments are not executed/recovered by this boundary. Width -32 stops at generic writer validation, before mode dispatch or diagnostics; the earlier reader analysis accepted -32. Width-error precedence under insufficient space is covered separately by [writer guards](MESSAGE_BYTE_CODEC.md).
+
+Resetting supplied readcount to the initial byte position and bit to 19 lets MSG_ReadBits read those same output bytes directly. Readcount and bit agree with the writer's final size/bit; negative -8/-16 results sign extend at their full normalized widths. These supplied buffers do not test reading beyond cursize, capacity exhaustion, signed/wrapping offsets or actual network headers.
+
+The evaluator adds narrowly scoped MOVW/BX support (low 16-bit register writes retain upper bits) and implicit-one 32-bit shifts, verified by an independent instruction fixture. Other 16-bit register aliases and high-byte register operations remain unsupported.
+
+## WIDTH-003 — Raw remainder and Huffman byte composition (RE-371)
+
+Each engine has 164 compressed probes: positive widths 1–32 plus selected negative widths -1/-7/-8/-9/-15/-16/-17/-24/-31, two value words and initial bit positions zero/seven. The first and second modeled objects are relocated to the native encoder/decoder addresses. Both writer and reader instruction slices stop before saved-register restoration and compose every encountered raw-bit or Huffman helper explicitly.
+
+For normalized width W, the writer masks the input to its low W bits. It emits the low W modulo eight bits first, individually and least-significant first through Huff_putBit. It then Huffman-encodes the remaining bytes in low-to-high order. Exact helper requests agree with that decomposition. Independent codeword packing verifies every output byte; actual reader traversal of the supplied tree decodes those bytes, rather than receiving supplied symbols or code lengths at helper boundaries.
+
+Writer size and reader count both become `(signed_bit >> 3) + 1`; bit advances by the raw remainder plus modeled codeword lengths. The initial partial-byte prefix is zero in these fixtures. Dirty-prefix preservation, buffer initialization and required boundary padding remain unverified.
+
+For positive widths, every decoded value equals the input masked to W bits. For selected negative widths, the reader sign tail uses the remaining byte-multiple width after removing the raw remainder, confirming the earlier [reader analysis](MESSAGE_BITS.md) against bytes generated by the writer. Examples for input word 0x8196a5ff and initial bit zero:
+
+| Requested width | Decoded word |
+| --- | --- |
+| -1 | 0x00000001 |
+| -7 | 0x0000007f |
+| -9 | 0xffffffff |
+| -15 | 0xffffffff |
+| -17 | 0xffffa5ff |
+| -31 | 0xff96a5ff |
+| +32 | 0x8196a5ff |
+
+These are recovered local behaviors, not corrected signed-codec specifications or proof that those negative widths occur in legal game traffic. The sampled paths do not exercise the writer's wider unrolled-loop body or unsupported high-byte register operations. Complete width/value/alias/overflow/wrap branch coverage remains open.
+
+## Validation and next work
+
+Eight independent tests verify warm/cold record effects and canaries, exact clear retention, OOB width errors/little-endian/sign/cursors, raw-remainder and symbol arguments, independent modeled-codeword packing, signed-tail/diagnostic/accounting effects, paired engine results, and the restricted evaluator extension. The report preserves graph provenance and compact composition trace hashes.
+
+RE-372: recover MSG_WriteByte/Short/Long/Data and read wrappers, string/float/angle/delta/entity/player codecs and snapshot messages; validate capacity/overflow/allowoverflow/clear/read-reset lifecycle, truncation/EOF/dirty-prefix/padding and legal signed-width callers. Complete native seed/codeword validation, actual packets, original-runtime/all-platform and renderer contracts remain open.

docs/research/README.md

@@ -139,3 +139,5 @@ The extension collector adds five focused tests for native tail calls, command-r
 [Differential recoding and provisional Huffman codebook](HUFFMAN_CODEBOOK.md) adds RE-361–363: full sampled state comparisons, both object halves and modeled seed training/codewords with native-helper symbol compositions. Reproduce with `python3 scripts/analyze_huffman_codebook.py`; check with `python3 scripts/test_huffman_codebook.py`. Native full-training, runtime/platform and complete packet compatibility remain open.
 
 [Message byte codec and writer guards](MESSAGE_BYTE_CODEC.md) adds RE-365–367: space/width/range priorities and native byte writer/reader compositions against model-derived trees. Reproduce with `python3 scripts/analyze_message_byte_codec.py`; check with `python3 scripts/test_message_byte_codec.py`. General codecs, padding/overflow/truncation, native codebook and actual packet/runtime/platform coverage remain open.
+
+[Message initialization, OOB and mixed-width codecs](MESSAGE_WIDTHS.md) adds RE-369–371: warm/cold record effects, exact clear retention and native writer/reader compositions for OOB and raw-remainder/Huffman paths. Reproduce with `python3 scripts/analyze_message_widths.py`; check with `python3 scripts/test_message_widths.py`. Wrappers, lifecycle, real packets and runtime/platform validation remain open.

scripts/analyze_collision_composition.py

@@ -14,7 +14,7 @@ from analyze_vm_extensions import native_functions,verified
 
 def register(r):
     aliases={'eax':('rax',32),'edi':('rdi',32),'esi':('rsi',32),'edx':('rdx',32),
-             'ecx':('rcx',32),'ebp':('rbp',32),'ebx':('rbx',32),'bl':('rbx',8),'al':('rax',8),'sil':('rsi',8),'dl':('rdx',8),'cl':('rcx',8),'dil':('rdi',8),'bpl':('rbp',8)}
+             'ecx':('rcx',32),'ebp':('rbp',32),'ebx':('rbx',32),'bx':('rbx',16),'bl':('rbx',8),'al':('rax',8),'sil':('rsi',8),'dl':('rdx',8),'cl':('rcx',8),'dil':('rdi',8),'bpl':('rbp',8)}
     r=r.lstrip('%')
     if r in aliases:return aliases[r]
     if re.fullmatch(r'r\d+d',r):return r[:-1],32
@@ -52,7 +52,7 @@ def evaluate(graph,start,stops,memory,registers,limit=128,capture_registers=Fals
         val&=(1<<(width*8))-1
         if arg.startswith('%'):
             key,size=register(arg)
-            if size==8:regs[key]=(regs.get(key,0)&~255)|val
+            if size in [8,16]:regs[key]=(regs.get(key,0)&~((1<<size)-1))|val
             else:regs[key]=val  # 32-bit writes clear high bits.
         else:
             a=address(arg);put_word(m,a,val,width);writes.append({'address':a,'bytes':width})
@@ -62,8 +62,8 @@ def evaluate(graph,start,stops,memory,registers,limit=128,capture_registers=Fals
         i=indices[pc];pc=rows[i+1]['address'] if i+1<len(rows) else None
         regs['rip']=pc
         if op in ['nop','nopl','nopw']:pass
-        elif op in ['movb','movl','movq','movss','movabsq']:
-            a,b=args.split(', ');width=8 if op in ['movq','movabsq'] else 1 if op=='movb' else 4;write(b,read(a,width),width)
+        elif op in ['movb','movw','movl','movq','movss','movabsq']:
+            a,b=args.split(', ');width=8 if op in ['movq','movabsq'] else 1 if op=='movb' else 2 if op=='movw' else 4;write(b,read(a,width),width)
         elif op in ['movzbl','movzwl']:
             a,b=args.split(', ');write(b,read(a,1 if op=='movzbl' else 2),4)
         elif op=='sbbl':
@@ -109,7 +109,7 @@ def evaluate(graph,start,stops,memory,registers,limit=128,capture_registers=Fals
         elif op=='negl':
             write(args,-read(args,4),4);equal=None;signed_le=signed_lt=negative=None;ge=greater=below=None
         elif op in ['sarl','shll','shrl']:
-            a,b=args.split(', ');amount=read(a,4)&31;v=read(b,4)
+            a,b=args.split(', ') if ', ' in args else ('$0x1',args);amount=read(a,4)&31;v=read(b,4)
             if amount:
                 signed_v=v if v<2**31 else v-2**32
                 write(b,signed_v>>amount if op=='sarl' else v>>amount if op=='shrl' else v<<amount,4)

scripts/analyze_message_widths.py

@@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+"""Bounded native message initialization, OOB and mixed-width compositions."""
+import gzip
+import hashlib
+import itertools
+import json
+from pathlib import Path
+from analyze_area_queries import site
+from analyze_bsp_references import global_address, trace_fields
+from analyze_collision_composition import get_word, put_word
+from analyze_engine_control import control_flow
+from analyze_message_byte_codec import MSG, DATA, STACK, install_tree, offset_transmit, run
+from analyze_qvm import write_json
+from analyze_vm_extensions import native_functions, verified
+
+
+def writer(graphs,m,width,value):
+    g=graphs['MSG_WriteBits'];rows=g['instructions']
+    start=next(x['address'] for x in rows if x['opcode']=='addl' and '(%rip)' in x['operands'])
+    end=next(x['address'] for x in rows if x['opcode']=='movq' and x['operands'].split('#')[0].strip()=='0x8(%rsp), %rbx')
+    # Stop before the unsupported-OOB-width epilogue restores saved registers.
+    bad=[x['address'] for x in rows if x['opcode']=='movl' and x['operands'].split('#')[0].strip()=='%r14d, %edx'][-1]
+    calls={x['address']:i for i,x in enumerate(rows) if x['opcode']=='callq'}
+    regs={'%rdi':MSG,'%rbp':MSG,'%rbx':value,'%rdx':width&0xffffffff,'%rsp':STACK};trace=[];requests=[];error=None
+    while True:
+        t=run(g,start,list(calls)+[end,bad],m,regs);m=t['memory'];regs=t['registers'];trace+=t['trace']
+        if t['stop']==end:break
+        if t['stop']==bad:error='oob_width';break
+        i=calls[t['stop']];target=rows[i]['operands']
+        if '<Com_Error>' in target:error='width';break
+        if '<Huff_offsetTransmit>' in target:
+            requests.append(['symbol',regs['rsi']]);m,child=offset_transmit(graphs['Huff_offsetTransmit'],graphs['send'],m,regs);trace+=child
+        elif '<Huff_putBit>' in target:
+            requests.append(['raw',regs['rdi']]);h=graphs['Huff_putBit'];child=run(h,h['instructions'][0]['address'],[x['address'] for x in h['instructions'] if x['opcode']=='retq'],m,regs);m=child['memory'];regs['rax']=child['registers']['rax'];trace+=child['trace']
+        else:raise ValueError('unexpected writer helper '+target)
+        start=rows[i+1]['address']
+    return m,requests,error,trace
+
+
+def reader(graphs,m,width):
+    g=graphs['MSG_ReadBits'];rows=g['instructions'];start=site(g,'testl','%esi, %esi');end=next(x['address'] for x in rows if x['opcode']=='movl' and x['operands'].split('#')[0].strip()=='%ebp, %eax');calls={x['address']:i for i,x in enumerate(rows) if x['opcode']=='callq'}
+    regs={'%r12':MSG,'%r15':width&0xffffffff,'%rsi':width&0xffffffff,'%rsp':STACK};trace=[];error=None
+    while True:
+        t=run(g,start,list(calls)+[end],m,regs);m=t['memory'];regs=t['registers'];trace+=t['trace']
+        if t['stop']==end:break
+        i=calls[t['stop']];target=rows[i]['operands']
+        if '<Com_Error>' in target:error='width';break
+        name='Huff_getBit' if '<Huff_getBit>' in target else 'Huff_offsetReceive' if '<Huff_offsetReceive>' in target else None
+        if name is None:raise ValueError('unexpected reader helper '+target)
+        h=graphs[name];child=run(h,h['instructions'][0]['address'],[x['address'] for x in h['instructions'] if x['opcode']=='retq'],m,regs);m=child['memory'];trace+=child['trace']
+        if name=='Huff_getBit':regs['rax']=child['registers']['rax']
+        start=rows[i+1]['address']
+    return m,None if error else regs['rbp']&0xffffffff,error,trace
+
+
+def main():
+    cp=Path('docs/research/huffman-codebook.json');code=json.loads(cp.read_text());init=json.loads(Path('docs/research/huffman-initialization.json').read_text())
+    out={'native':{},'codebook_report_sha256':hashlib.sha256(cp.read_bytes()).hexdigest(),'limitations':'Synthetic Linux ELF64 instruction compositions only; original processes and native callees never execute. Warm initialization omits saves/restores; cold initialization stops at first Huff_Init. Explicit caller-register restoration and supplied virtual memory. Compressed tree is MODEL-DERIVED, not full native training/runtime capture. Oversized buffers, selected bit/value states only; no capacity/truncation/lifecycle/packet/all-platform conformance.'}
+    for label in ['linux--Quake3-UrT-Ded.x86_64','linux--Quake3-UrT.x86_64']:
+        report=json.loads(Path(f'docs/research/native/{label}.json').read_text());verified((Path('.cache/UrbanTerror43')/report['source']['path']).read_bytes(),report['source']['sha256']);a=report['artifacts']['disassembly.txt'];fs=native_functions(verified(gzip.open(a['path'],'rb').read(),a['uncompressed_sha256']).decode())
+        names=['MSG_Init','MSG_InitOOB','MSG_Clear','MSG_WriteBits','MSG_ReadBits','Huff_putBit','Huff_getBit','Huff_offsetTransmit','Huff_offsetReceive','send'];graphs={n:control_flow(fs[n]) for n in names}
+        if code['native'][label]['source']!=report['source'] or code['native'][label]['disassembly_sha256']!=a['uncompressed_sha256']:raise ValueError('codebook provenance')
+        initialization=[]
+        for name,flag,size in itertools.product(['MSG_Init','MSG_InitOOB'],[0,1],[0,64,0xffffffff]):
+            g=graphs[name];rows=g['instructions'];load=next(x for x in rows if x['opcode']=='movl' and '<msgInit>' in x['operands']);address=global_address(load);end=next(x['address'] for x in rows if x['opcode']=='movq' and x['operands'].split('#')[0].strip()=='0x8(%rsp), %rbx');call=next(x['address'] for x in rows if '<Huff_Init>' in x['operands'])
+            m={MSG+i:0xa5 for i in range(-1,41)};put_word(m,address,flag)
+            t=run(g,load['address'],[end,call],m,{'%r13':MSG,'%r15':DATA,'%rdx':size,'%rsp':STACK})
+            initialization.append({'function':name,'initial_flag':flag,'size_word':size,'outcome':'cold_huff_init' if t['stop']==call else 'warm_record','final_flag':get_word(t['memory'],address),'record_bytes':[t['memory'][MSG+i] for i in range(40)],'canaries':[t['memory'][MSG-1],t['memory'][MSG+40]],'huff_init_pointer':t['registers']['rdi'] if t['stop']==call else None,**trace_fields(t,False)})
+        clears=[]
+        for fill in [0,0xa5]:
+            g=graphs['MSG_Clear'];m={MSG+i:fill for i in range(-1,41)};t=run(g,g['instructions'][0]['address'],[x['address'] for x in g['instructions'] if x['opcode']=='retq'],m,{'%rdi':MSG});clears.append({'initial_byte':fill,'record_bytes':[t['memory'][MSG+i] for i in range(40)],'canaries':[t['memory'][MSG-1],t['memory'][MSG+40]],**trace_fields(t,False)})
+        base=init['native'][label]['entry_request']['huff_init_pointer'];template={};state=code['model_derived_seed_training']['final_state'];install_tree(template,state,base);root=install_tree(template,state,base+0xc830)
+        for x in graphs['MSG_ReadBits']['instructions']:
+            if x['opcode']=='movq' and '(%rip)' in x['operands']:put_word(template,global_address(x),root,8)
+        g=graphs['MSG_WriteBits'];old=global_address(next(x for x in g['instructions'] if x['opcode']=='addl' and '(%rip)' in x['operands']));over=global_address(next(x for x in g['instructions'] if x['opcode']=='addl' and '$0x1,' in x['operands'] and '(%rip)' in x['operands']))
+        groups={}
+        for mode,widths,values,positions in [(1,[8,16,32,-8,-16,1,7,24,-32],[0,0x8196a5ff],[0,3]),(2,[8,16,32],[0x8196a5ff],[0]),(0,list(range(1,33))+[-1,-7,-8,-9,-15,-16,-17,-24,-31],[0,0x8196a5ff],[0,7])]:
+            probes=[]
+            for width,value,position in itertools.product(widths,values,positions):
+                m=dict(template) if mode==0 else {}
+                for i in range(256):m[DATA+i]=0
+                m[DATA-1]=m[DATA+256]=0x7b
+                initial_size=position if mode else position//8+1;initial_bit=19 if mode else position
+                for offset,v in [(4,0),(8,mode),(24,256),(28,initial_size),(32,0),(36,initial_bit)]:put_word(m,MSG+offset,v)
+                put_word(m,MSG+16,DATA,8);put_word(m,old,0);put_word(m,over,0)
+                m,requests,error,trace=writer(graphs,m,width,value);bit=get_word(m,MSG+36);size=get_word(m,MSG+28);payload=[m[DATA+i] for i in range(size)];counter=get_word(m,over);account=get_word(m,old)
+                decoded=None;read_error=None;read_bit=None;read_count=None
+                if error is None:
+                    put_word(m,MSG+36,initial_bit);put_word(m,MSG+32,position if mode else 0)
+                    m,decoded,read_error,child=reader(graphs,m,width);trace+=child;read_bit=get_word(m,MSG+36);read_count=get_word(m,MSG+32)
+                probes.append({'width':width,'value_word':value,'mode_word':mode,'position':position,'initial_bit':initial_bit,'initial_size':initial_size,'write_error':error,'requests':requests,'payload_bytes':payload,'write_bit':bit,'write_size':size,'oldsize_word':account,'range_counter':counter,'message_overflow':get_word(m,MSG+4),'decoded_word':decoded,'read_error':read_error,'read_bit':read_bit,'read_count':read_count,'canaries':[m[DATA-1],m[DATA+256]],**trace_fields({'trace':trace},False)})
+            groups['compressed' if mode==0 else 'oob' if mode==1 else 'noncanonical_oob']=probes
+        out['native'][label]={'source':report['source'],'disassembly_sha256':a['uncompressed_sha256'],'graphs':graphs,'initialization':initialization,'clears':clears,**groups}
+    write_json(Path('docs/research/message-widths.json'),out)
+    print('Recovered initialization/clear, OOB and mixed-width message compositions')
+
+
+if __name__=='__main__':main()

scripts/test_message_widths.py

@@ -0,0 +1,97 @@
+#!/usr/bin/env python3
+"""Independent record, byte order, bit packing and signed-tail oracles."""
+import json
+from pathlib import Path
+import unittest
+from analyze_collision_composition import evaluate
+from test_bsp_planes import graph
+
+
+class MessageWidthTests(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        cls.r=json.loads(Path('docs/research/message-widths.json').read_text())
+        cls.words={x['symbol']:x['bits'] for x in json.loads(Path('docs/research/huffman-codebook.json').read_text())['model_derived_seed_training']['codebook']}
+
+    def test_warm_record_cold_boundary_and_canaries(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['initialization']),12)
+            for p in n['initialization']:
+                self.assertEqual(p['final_flag'],1);self.assertEqual(p['canaries'],[0xa5]*2)
+                if not p['initial_flag']:
+                    self.assertEqual(p['outcome'],'cold_huff_init');self.assertEqual(p['record_bytes'],[0xa5]*40);self.assertIsNotNone(p['huff_init_pointer'])
+                else:
+                    expected=[0]*40;expected[16:24]=list((0xa0000).to_bytes(8,'little'));expected[24:28]=list(p['size_word'].to_bytes(4,'little'));expected[8]=int(p['function'].endswith('OOB'))
+                    self.assertEqual(p['record_bytes'],expected);self.assertEqual(p['outcome'],'warm_record')
+
+    def test_clear_only_overflow_size_bit(self):
+        for n in self.r['native'].values():
+            for p in n['clears']:
+                expected=[p['initial_byte']]*40
+                for offset in [4,28,36]:expected[offset:offset+4]=[0]*4
+                self.assertEqual(p['record_bytes'],expected);self.assertEqual(p['canaries'],[p['initial_byte']]*2)
+
+    def test_oob_width_dispatch_endian_and_cursors(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['oob']),36)
+            for p in n['oob']+n['noncanonical_oob']:
+                width=abs(p['width']);valid=width in [8,16,32] and p['width']!=-32
+                self.assertEqual(p['write_error'],None if valid else 'width' if p['width']==-32 else 'oob_width');self.assertEqual(p['requests'],[])
+                if not valid:
+                    self.assertEqual(p['write_bit'],19);self.assertEqual(p['write_size'],p['position']);continue
+                payload=[0]*p['position']+list((p['value_word']&((1<<width)-1)).to_bytes(width//8,'little'))
+                self.assertEqual(p['payload_bytes'],payload);self.assertEqual(p['write_size'],p['position']+width//8);self.assertEqual(p['write_bit'],19+width);self.assertEqual(p['read_count'],p['write_size']);self.assertEqual(p['read_bit'],p['write_bit'])
+                value=p['value_word']&((1<<width)-1)
+                if p['width']<0 and value&(1<<(width-1)):value|=(0xffffffff<<width)&0xffffffff
+                self.assertEqual(p['decoded_word'],value)
+
+    def test_compressed_requests_raw_remainder_then_low_byte_symbols(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['compressed']),164)
+            for p in n['compressed']:
+                width=abs(p['width']);rem=width%8;value=p['value_word']&((1<<width)-1)
+                requests=[['raw',(value>>i)&1] for i in range(rem)]+[['symbol',(value>>(rem+8*i))&255] for i in range(width//8)]
+                self.assertEqual(p['requests'],requests);self.assertIsNone(p['write_error']);self.assertIsNone(p['read_error'])
+
+    def test_independent_mixed_width_codeword_packing(self):
+        for n in self.r['native'].values():
+            for p in n['compressed']:
+                width=abs(p['width']);rem=width%8;value=p['value_word']&((1<<width)-1)
+                bits=[(value>>i)&1 for i in range(rem)]
+                for i in range(width//8):bits+=self.words[(value>>(rem+8*i))&255]
+                final=p['initial_bit']+len(bits);expected=[0]*(final//8+1)
+                for i,bit in enumerate(bits):pos=p['initial_bit']+i;expected[pos//8]|=bit<<(pos%8)
+                self.assertEqual(p['payload_bytes'],expected);self.assertEqual(p['write_bit'],final);self.assertEqual(p['write_size'],final//8+1);self.assertEqual(p['read_bit'],final);self.assertEqual(p['read_count'],final//8+1)
+
+    def test_signed_tail_uses_mutated_width_and_accounting(self):
+        for n in self.r['native'].values():
+            for p in n['compressed']:
+                width=abs(p['width']);value=p['value_word']&((1<<width)-1);remaining=width-width%8
+                if p['width']<0 and remaining and value&(1<<(remaining-1)):value|=(0xffffffff<<(remaining&31))&0xffffffff
+                self.assertEqual(p['decoded_word'],value);self.assertEqual(p['oldsize_word'],p['width']&0xffffffff);self.assertEqual(p['message_overflow'],0);self.assertEqual(p['canaries'],[0x7b]*2)
+
+        for n in self.r['native'].values():
+            for p in n['compressed']+n['oob']+n['noncanonical_oob']:
+                width=p['width'];value=p['value_word'];value=value if value<2**31 else value-2**32
+                if width==-32:overflow=False  # Width validation stops before diagnostics.
+                elif width==32:overflow=False
+                elif width>0:overflow=value<0 or value>(1<<width)-1
+                else:
+                    bound=1<<((width-1)&31);overflow=value < -bound or value > bound-1
+                self.assertEqual(p['range_counter'],int(overflow))
+                self.assertEqual(p['oldsize_word'],width&0xffffffff)
+                self.assertEqual(p['canaries'],[0x7b]*2)
+
+    def test_paired_results(self):
+        a,b=self.r['native'].values()
+        for key in ['initialization','clears','oob','noncanonical_oob','compressed']:
+            def clean(rows):return [{k:v for k,v in p.items() if k not in ['trace','trace_sha256','huff_init_pointer']} for p in rows]
+            self.assertEqual(clean(a[key]),clean(b[key]))
+
+    def test_word_move_partial_register_and_implicit_shift(self):
+        g=graph([('movw','%bx, (%rdi)'),('movw','$0x1234, %bx'),('sarl','%eax'),('nop','')])
+        t=evaluate(g,0,[3],{}, {'%rbx':0xdeadbeefa5ff,'%rdi':100,'%rax':0x80000001},capture_registers=True)
+        self.assertEqual([t['memory'][100],t['memory'][101]],[255,165]);self.assertEqual(t['registers']['rbx'],0xdeadbeef1234);self.assertEqual(t['registers']['rax'],0xc0000000)
+
+
+if __name__=='__main__':unittest.main()