Compose message byte codec and writer guard contracts

docs/ASSET_PIPELINE_TODO.md

@@ -224,3 +224,5 @@ Initialization follow-up: [native frequency tables](research/HUFFMAN_INITIALIZAT
 Adaptive-codec follow-up: [update compositions](research/HUFFMAN_UPDATES.md) validates sampled supplied-tree insertion/count/list/block behavior. Actual seed-trained codebooks, real packets and runtime/platform validation remain required before network/map/renderer integration (RE-360).
 
 Codebook follow-up: [provisional modeled codewords](research/HUFFMAN_CODEBOOK.md) adds exact native seed training in a differentially checked recoding. Full native/runtime codeword and message/packet validation remains required before network/map/renderer integration (RE-364).
+
+Message-codec follow-up: [modeled-tree byte streams](research/MESSAGE_BYTE_CODEC.md) validates native message/helper compositions and byte accounting. Full native codebook, general widths and real packet/buffer contracts remain prerequisites for network/map/renderer integration (RE-368).

docs/FINDINGS.md

@@ -402,3 +402,9 @@ Bounded qagame paths recover bomb-plant/inactivity correction requests, direct f
 **F-131 — Observed paired-object update samples:** first/second initialized halves match their respective recoding states after every fresh update, with the +24 pointer-slot asymmetry retained. Full initialization/training lifetime and runtime/platform equivalence remain open. [CODE-002](research/HUFFMAN_CODEBOOK.md).
 
 **F-132 — Model-derived seeded codebook:** exact native counts produce a provisional 513-node tree, 2–11-bit prefix-free codewords and compatible bounded native-helper symbol round trips against supplied modeled memory. Full native seeded training/codeword/packet conformance remains unverified. [CODE-003](research/HUFFMAN_CODEBOOK.md).
+
+**F-133 — Observed writer guard priority:** requested width updates oldsize before capacity/width validation; signed remaining space <=3 sets message overflow and returns first. With enough space, zero and widths outside -31..32 request an error. Full buffer/lifecycle/wrap/runtime behavior remains open. [BYTE-001](research/MESSAGE_BYTE_CODEC.md).
+
+**F-134 — Observed value diagnostics:** range failures increment a separate global counter without stopping the sampled path. Compressed width eight masks to low byte; selected negative-width bounds use pre-normalization x86 shift counts. General signed/OOB/wrapper and legal-caller behavior remain open. [BYTE-002](research/MESSAGE_BYTE_CODEC.md).
+
+**F-135 — Observed modeled-tree message byte composition:** native writer/reader/helper instructions agree on exact compressed bytes and per-symbol cursors using supplied model-derived trees. Size/readcount use bit/8 +1, including an extra byte at exact boundaries. Native trained-codebook, real packet/padding/overflow/runtime/platform conformance remains open. [BYTE-003](research/MESSAGE_BYTE_CODEC.md).

docs/PROVISIONAL_TODO.md

@@ -536,3 +536,5 @@ Research progress: [Huffman initialization](research/HUFFMAN_INITIALIZATION.md)
 Research progress: [adaptive Huffman updates](research/HUFFMAN_UPDATES.md) completes bounded RE-357–359. Broad network/runtime milestones remain open; RE-360 tracks complete swap/tie paths, second-object equivalence, actual trained codebooks and real packet/platform fixtures.
 
 Research progress: [differential Huffman codebook](research/HUFFMAN_CODEBOOK.md) completes bounded RE-361–363. Broad network/runtime milestones remain open; RE-364 tracks full native-training/codeword conformance and complete real packet/platform contracts.
+
+Research progress: [message byte codec](research/MESSAGE_BYTE_CODEC.md) completes bounded RE-365–367. Broad network/runtime milestones remain open; RE-368 tracks general message/snapshot codecs, overflow/truncation/padding and real native packet/platform validation.

docs/RESEARCH_LEDGER.md

@@ -225,3 +225,7 @@ Completed bounded RE-357–359 with `analyze_huffman_updates.py`: four verified
 ## 2026-10-11 — Differential Huffman recoding and provisional seeded codebook
 
 Completed bounded RE-361–363 with `huffman_model.py` and `analyze_huffman_codebook.py`: compact observed update recoding, complete prior-state comparisons (152 retained prefixes plus 22 final cases), sixteen fresh first/second-object cases (288 native-prefix comparisons), exact native-frequency model training and eight native-helper symbol round trips against the modeled tree. CODE-001–003/F-130–132 distinguish sampled state equivalence from branch-complete/runtime equivalence, retained second-object slot asymmetry from normalization, and MODEL-DERIVED seeded trees/codewords from full native training or runtime captures. The 1,053,340-operation model schedule yields 513 nodes, 493 allocated cells (489 live/four free), 257 prefix-free leaf codes with lengths 2–11, and explicit state/codebook/source/report hashes. All 256 model symbol checkpoints validate tree/count/rank/leader/pool invariants. Native transmit/send/receive instruction compositions agree with independent modeled-codeword packing, including every byte value; no original process or native callee executes. Eight tests verify prior/fresh differentials, sampled event classes, modeled training counts/checkpoints, independent tree/codeword/Kraft checks and native-helper bytes/symbol/cursor/canary/paired results. All 334 tests pass; new and preceding reports regenerate byte-identically, and storage/link/whitespace checks pass. RE-360 remains open; RE-364 tracks full native-seed/codeword validation, authoritative/runtime comparison, complete tie/swap/alias/capacity/wrap/platform coverage and actual message/packet/snapshot/renderer contracts.
+
+## 2026-10-11 — Message writer guards and modeled-tree byte codec
+
+Completed bounded RE-365–367 with `analyze_message_byte_codec.py`: ten verified graphs across the Linux ELF64 engines, 576 writer guard/range probes and 36 compressed byte streams. BYTE-001–003/F-133–135 distinguish requested-width oldsize accounting from compressed bit lengths, the early signed space guard from later width errors, the global range diagnostic from message overflow, and pre-normalization negative-width shifts from ordinary signed-range expectations. Byte streams compose writer/transmit/send and reader/receive against relocated MODEL-DERIVED seed trees; independent codeword packing verifies payloads, decoded low bytes and every per-byte cursor. Exact byte boundaries retain an extra size/readcount byte, zero only because these supplied buffers are initialized to zero. Save/restore omission, stopped CALL boundaries and explicit caller-register restoration remain documented; no original process or native callee executes. Eight tests verify guard priority/accounting, width normalization, signed range diagnostics, independent payload/cursor packing, low-byte conversion/canaries, extra-byte boundaries, helper arguments and paired results. All 342 tests pass. New and preceding reports regenerate byte-identically; storage, links and whitespace checks pass. RE-364 remains open; RE-368 tracks general raw/signed/multi-byte/OOB and wrapper/snapshot codecs, capacity/overflow/reset/truncation/EOF/padding invariants, real native full-training codewords and actual packet/original-runtime/all-platform/snapshot/renderer contracts.

docs/REVERSE_ENGINEERING_TODO.md

@@ -437,3 +437,8 @@ Purpose: turn the supplied installation into an evidence-backed specification of
 - [x] RE-362 Compare both initialized object halves after every sampled update, retaining slot asymmetry and relocated pointers ([CODE-002](research/HUFFMAN_CODEBOOK.md)).
 - [x] RE-363 Train the provisional model with exact native seeds, export/check tree and prefix-free codewords, and compose native-helper symbol round trips against that supplied tree ([CODE-003](research/HUFFMAN_CODEBOOK.md)).
 - [ ] RE-364 Validate complete native-seed training/codewords against an independent authoritative implementation or permitted original-runtime captures; prove remaining swap/tie/alias/error/capacity/wrap/platform paths and compose full message/real-packet/overflow/truncation/snapshot/renderer contracts.
+
+- [x] RE-365 Recover bounded writer space-guard priority, width errors and original requested-bit counter updates ([BYTE-001](research/MESSAGE_BYTE_CODEC.md)).
+- [x] RE-366 Recover sampled signed value-range diagnostic gates, negative-width normalization and compressed byte masking ([BYTE-002](research/MESSAGE_BYTE_CODEC.md)).
+- [x] RE-367 Compose modeled-tree native message byte writes/reads with exact payload, helper arguments, size/readcount and padding oracles ([BYTE-003](research/MESSAGE_BYTE_CODEC.md)).
+- [ ] RE-368 Recover general raw/signed/multi-byte/OOB and wrapper/snapshot codecs, compressed capacity/overflow/reset/truncation/EOF/padding invariants; verify native full-training codewords and actual packet/original-runtime/all-platform/snapshot/renderer contracts.

docs/research/HUFFMAN_CODEBOOK.md

@@ -37,3 +37,5 @@ The exported tree and codebook are research artifacts for subsequent comparison.
 Eight tests check prior complete-state differential equivalence, fresh first/second-object prefixes and invariants, sampled model event classes, full seed leaf/subtree/reachability/count checks, checkpoint/cell summaries, independent tree-derived prefix-free codewords/Kraft equality, native-helper symbol packing/cursors/canaries and paired helper outputs. No shared evaluator extension was needed.
 
 RE-364: prove individual native swap/tie/alias/error/capacity/wrap paths, validate full native-seed training/codewords against an independent authoritative implementation or permitted original-runtime captures, establish all-platform equivalence, and compose complete message writers/readers with actual packets, overflow/truncation and snapshot/renderer integration.
+
+Message follow-up: [byte writer/reader compositions](MESSAGE_BYTE_CODEC.md) connects the modeled tree to native message bit functions and cursor accounting (RE-365–367). Actual native tree training, full message widths and packet/runtime coverage remain open.

docs/research/MESSAGE_BYTE_CODEC.md

@@ -0,0 +1,39 @@
+# Message writer guards and modeled-tree byte codec
+
+Reproduce with `python3 scripts/analyze_message_byte_codec.py`; validate with `python3 scripts/test_message_byte_codec.py`. [message-byte-codec.json](message-byte-codec.json) retains five hash-verified graphs per Linux ELF64 engine: MSG_WriteBits, MSG_ReadBits, Huff_offsetTransmit, Huff_offsetReceive and send. It records the hash of the supplied [model-derived codebook report](HUFFMAN_CODEBOOK.md).
+
+All results are synthetic native instruction compositions with explicit supplied message/buffer/stack/tree memory. The full seed-trained tree comes from the provisional recoding, not native runtime capture. Register-save/restore prologues and epilogues are excluded; wrapper PUSH/POP are omitted/stopped; all helper CALLs stop and compose with explicit caller-register restoration. Original processes and native callees never execute. These are byte-width codec contracts, not complete network packet or native-codebook conformance.
+
+## BYTE-001 — Space-guard priority and requested-bit accounting (RE-365)
+
+Five hundred seventy-six paired guard probes cover requested widths 0/1/7/8/16/31/32/33/-1/-8/-31/-32, eight supplied value words and remaining space 0/3/4 bytes. The sampled message uses cursize 100 and maxsize 100 +remaining, with overflow flag initially zero. This characterizes local nonnegative space arithmetic, not all capacity/wrap/alias states.
+
+Before checking capacity or width validity, MSG_WriteBits adds the original requested width word to global oldsize modulo 2^32. The collector seeds oldsize seven, so the result is `(7 + requested_width) mod 2^32`, including negative and invalid widths.
+
+The writer computes maxsize minus cursize and continues only when the signed difference exceeds three. Otherwise it sets message overflow flag +4 to one and returns before width validation, range checking or helper requests. Width zero therefore requests no width error when this earlier guard fails. The valid-space slice requests a width error for zero, widths above 32 and widths below -31; -32 is rejected here even though the earlier reader probes accepted absolute width 32. The error callee is stopped. Full overflow/allowoverflow reset/recovery policy and actual packet buffers remain open.
+
+## BYTE-002 — Range diagnostics and low-byte output (RE-366)
+
+With adequate space and a valid width, positive width 32 bypasses the value-range diagnostic. Positive widths 1–31 increment global overflows when the signed input is negative or above `(1 << width) -1`. This counter is distinct from the message overflow flag. It does not stop the sampled write path.
+
+Negative widths normalize to their absolute value after the observed range checks. Their pre-normalization bounds are calculated through a native 32-bit shift using `(requested_width -1) &31`. In the sampled -1/-8/-31 cases, the accepted signed interval is `[-bound, bound-1]`, where `bound = 1 << ((requested_width -1) &31)`. For -8 this yields ±2^23 rather than ordinary signed-byte bounds. This is a recovered instruction-slice result, not a corrected protocol specification or proof that these caller inputs are legal. Other negative widths and general signed/raw-bit emission remain open.
+
+For compressed width eight, the writer masks to the low byte before requesting Huff_offsetTransmit. Stream fixtures include input words 256/257/0xffffffff/0x1234; they increment the range counter but encode bytes 0/1/255/52, respectively. The message overflow flag remains zero in these ample-buffer fixtures. Complete MSG_WriteByte/WriteData wrapper checks and diagnostics are not composed here.
+
+## BYTE-003 — Compressed byte streams, size and readcount (RE-367)
+
+Thirty-six paired streams cover initial bit cursors 0/7/8 and six sequences: one zero; one 255; boundary bytes 0/1/127/128/255; all 256 bytes ascending; repeated 0/255/128/1; and the out-of-range words above. Both supplied modeled trees are relocated to the native objects' addresses. Encoder requests forward the first object, masked symbol, output buffer and message +36 bit-pointer. The reader forwards the second object's supplied root and traverses its modeled nodes.
+
+Each stream composes MSG_WriteBits(width=8), Huff_offsetTransmit and send, retaining native buffer/cursor effects. It then resets the supplied bit/read cursor to the starting position and composes MSG_ReadBits(width=8) plus Huff_offsetReceive against those exact output bytes. No Huffman symbol or code length is supplied at the individual call boundary: the supplied tree determines them. The tree's actual full native training remains unverified.
+
+Independent codeword packing verifies every payload byte and decoded low-byte value. Writer cursize at message +28 becomes `(signed_bit >>3) +1`; reader readcount at +32 uses the same formula. All sampled cursors are nonnegative. Each requested byte advances the bit cursor by its variable modeled code length, and writer size/reader count agree after every symbol. At an exact byte boundary, the reported size includes one extra byte. That byte is zero in these explicitly zero-initialized buffers; this does not prove original buffer initialization or required padding contents.
+
+Global oldsize becomes eight times the number of requested byte operations, even though compressed bit lengths differ. This separates requested-bit diagnostic accounting from actual compressed cursors. The report preserves exact helper arguments, size/readcount pairs, decoded words, diagnostic counters and canaries.
+
+Buffers are explicitly oversized to 4096 bytes with outside canaries. The guard is recovered separately; streams do not test near-capacity compressed expansion, truncation/EOF/bad-read handling, arbitrary starting prefix bytes, OOB mode, general bit widths or actual snapshot messages.
+
+## Validation and next work
+
+Eight independent tests verify guard priority/oldsize, valid width/normalization, signed range counters, codeword-derived payload and per-byte cursors, low-byte diagnostic behavior, exact-boundary extra bytes, helper arguments/canaries and paired engine results. No shared evaluator extension was needed.
+
+RE-368: compose general raw/signed/multi-byte and OOB writer-reader paths, wrappers and actual snapshot messages; recover compressed capacity/overflow/reset and truncation/EOF/bit-cursor invariants, verify the real native trained codebook and buffer padding, then validate original-runtime/all-platform and packet/snapshot/renderer contracts.

docs/research/README.md

@@ -137,3 +137,5 @@ The extension collector adds five focused tests for native tail calls, command-r
 [Adaptive Huffman update compositions](HUFFMAN_UPDATES.md) adds RE-357–359: insertion/dispatch, recursive count/tree invariants and rank/block/freelist reuse. Reproduce with `python3 scripts/analyze_huffman_updates.py`; check with `python3 scripts/test_huffman_updates.py`. Full branch/tie/second-object/seeded-codebook/message/runtime/platform contracts remain open.
 
 [Differential recoding and provisional Huffman codebook](HUFFMAN_CODEBOOK.md) adds RE-361–363: full sampled state comparisons, both object halves and modeled seed training/codewords with native-helper symbol compositions. Reproduce with `python3 scripts/analyze_huffman_codebook.py`; check with `python3 scripts/test_huffman_codebook.py`. Native full-training, runtime/platform and complete packet compatibility remain open.
+
+[Message byte codec and writer guards](MESSAGE_BYTE_CODEC.md) adds RE-365–367: space/width/range priorities and native byte writer/reader compositions against model-derived trees. Reproduce with `python3 scripts/analyze_message_byte_codec.py`; check with `python3 scripts/test_message_byte_codec.py`. General codecs, padding/overflow/truncation, native codebook and actual packet/runtime/platform coverage remain open.

scripts/analyze_message_byte_codec.py

@@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+"""Writer guard slices and native byte-codec compositions with modeled trees."""
+import copy
+import gzip
+import hashlib
+import itertools
+import json
+from pathlib import Path
+from analyze_area_queries import site
+from analyze_bsp_references import global_address,trace_fields
+from analyze_collision_composition import evaluate,get_word,put_word
+from analyze_engine_control import control_flow
+from analyze_huffman_encoding import send
+from analyze_qvm import write_json
+from analyze_vm_extensions import native_functions,verified
+
+MSG,DATA,STACK=0x90000,0xa0000,0xb0000
+
+
+def run(g,start,stops,m,regs):return evaluate(g,start,stops,m,regs,limit=10000,capture_registers=True)
+
+
+def install_tree(m,s,base):
+    original=0x10000;delta=base-original
+    def pointer(v):return v+delta if v else 0
+    for i in range(0xc830):m[base+i]=0
+    put_word(m,base,s['node_count']);put_word(m,base+4,s['pointer_cell_count'])
+    for offset,key in [(8,'root'),(16,'list_start'),(24,'tail_slot'),(0x828,'free_head')]:put_word(m,base+offset,pointer(s[key]),8)
+    for symbol,value in s['lookup']:put_word(m,base+32+8*symbol,pointer(value),8)
+    for i,value in enumerate(s['pointer_cells']):put_word(m,base+0xb030+8*i,pointer(value),8)
+    for node in s['nodes']:
+        address=pointer(node['address'])
+        for offset,key in [(0,'left'),(8,'right'),(16,'parent'),(24,'next'),(32,'previous'),(40,'head_cell')]:put_word(m,address+offset,pointer(node[key]),8)
+        put_word(m,address+48,node['weight']);put_word(m,address+52,node['symbol'])
+    return pointer(s['root'])
+
+
+def offset_transmit(g,sg,m,regs):
+    graph=copy.deepcopy(g);rows=graph['instructions']
+    for x in rows:
+        if x['opcode']=='pushq':x['opcode']='nop';x['operands']=''
+    idx=next(i for i,x in enumerate(rows) if x['opcode']=='callq');end=next(x['address'] for x in rows if x['opcode']=='popq')
+    t=run(graph,rows[0]['address'],[rows[idx]['address'],end],m,regs);trace=t['trace'];m=t['memory']
+    if t['stop']!=end:
+        r=t['registers'];child=send(sg,m,r['rdi'],r['rsi'],r['rdx']);m=child['memory'];trace+=child['trace']
+        t=run(graph,rows[idx+1]['address'],[end],m,r);m=t['memory'];trace+=t['trace']
+    return m,trace
+
+
+def byte_write(graphs,m,value):
+    g=graphs['MSG_WriteBits'];rows=g['instructions'];start=next(x['address'] for x in rows if x['opcode']=='addl' and '(%rip)' in x['operands']);end=next(x['address'] for x in rows if x['opcode']=='movq' and x['operands'].split('#')[0].strip()=='0x8(%rsp), %rbx');calls={x['address']:i for i,x in enumerate(rows) if x['opcode']=='callq'}
+    regs={'%rdi':MSG,'%rbp':MSG,'%rbx':value,'%rdx':8,'%rsp':STACK};trace=[];requests=[]
+    while True:
+        t=run(g,start,list(calls)+[end],m,regs);trace+=t['trace'];m=t['memory'];regs=t['registers']
+        if t['stop']==end:break
+        idx=calls[t['stop']]
+        if '<Huff_offsetTransmit>' not in rows[idx]['operands']:raise ValueError('unexpected byte writer helper')
+        requests.append([regs[k] for k in ['rdi','rsi','rdx','rcx']]);m,child=offset_transmit(graphs['Huff_offsetTransmit'],graphs['send'],m,regs);trace+=child;start=rows[idx+1]['address']
+    return m,trace,requests
+
+
+def byte_read(graphs,m):
+    g=graphs['MSG_ReadBits'];rows=g['instructions'];start=site(g,'testl','%esi, %esi');end=next(x['address'] for x in rows if x['opcode']=='movl' and x['operands'].split('#')[0].strip()=='%ebp, %eax');calls={x['address']:i for i,x in enumerate(rows) if x['opcode']=='callq'}
+    regs={'%r12':MSG,'%r15':8,'%rsi':8,'%rsp':STACK};trace=[]
+    while True:
+        t=run(g,start,list(calls)+[end],m,regs);trace+=t['trace'];m=t['memory'];regs=t['registers']
+        if t['stop']==end:break
+        idx=calls[t['stop']]
+        if '<Huff_offsetReceive>' not in rows[idx]['operands']:raise ValueError('unexpected byte reader helper')
+        h=graphs['Huff_offsetReceive'];child=run(h,h['instructions'][0]['address'],[x['address'] for x in h['instructions'] if x['opcode']=='retq'],m,regs);m=child['memory'];trace+=child['trace'];start=rows[idx+1]['address']
+    return m,trace,regs['rbp']&0xffffffff
+
+
+def main():
+    cp=Path('docs/research/huffman-codebook.json');code=json.loads(cp.read_text());initial=json.loads(Path('docs/research/huffman-initialization.json').read_text());out={'native':{},'codebook_report_sha256':hashlib.sha256(cp.read_bytes()).hexdigest(),'limitations':'Linux ELF64 synthetic native writer/reader/helper compositions using MODEL-DERIVED supplied seed trees. No original process/native callee/runtime/codebook capture or real packet executes. Saves/restores/PUSH/POP excluded; explicit caller restoration. Byte-width only; general raw/signed/multi-byte message codecs, capacities, truncation, packet/snapshot/all-platform and full native seed equivalence remain open.'}
+    sequences=[[0],[255],[0,1,127,128,255],list(range(256)),[0,255,128,1]*8,[256,257,0xffffffff,0x1234]]
+    for label in ['linux--Quake3-UrT-Ded.x86_64','linux--Quake3-UrT.x86_64']:
+        r=json.loads(Path(f'docs/research/native/{label}.json').read_text());verified((Path('.cache/UrbanTerror43')/r['source']['path']).read_bytes(),r['source']['sha256']);a=r['artifacts']['disassembly.txt'];fs=native_functions(verified(gzip.open(a['path'],'rb').read(),a['uncompressed_sha256']).decode());graphs={n:control_flow(fs[n]) for n in ['MSG_WriteBits','MSG_ReadBits','Huff_offsetTransmit','Huff_offsetReceive','send']}
+        if code['native'][label]['source']!=r['source'] or code['native'][label]['disassembly_sha256']!=a['uncompressed_sha256']:raise ValueError('codebook provenance')
+        g=graphs['MSG_WriteBits'];rows=g['instructions'];start=next(x['address'] for x in rows if x['opcode']=='addl' and '(%rip)' in x['operands']);end=next(x['address'] for x in rows if x['opcode']=='movq' and x['operands'].split('#')[0].strip()=='0x8(%rsp), %rbx');mode=site(g,'movl','0x8(%rbp), %r9d');error=next(x['address'] for x in rows if '<Com_Error>' in x['operands']);old=global_address(next(x for x in rows if x['opcode']=='addl' and '(%rip)' in x['operands']));over=global_address(next(x for x in rows if x['opcode']=='addl' and '$0x1,' in x['operands'] and '(%rip)' in x['operands']));guards=[]
+        for width,value,remaining in itertools.product([0,1,7,8,16,31,32,33,-1,-8,-31,-32],[0,127,128,255,256,0xffffffff,0x7fffffff,0x80000000],[0,3,4]):
+            m={};put_word(m,MSG+24,100+remaining);put_word(m,MSG+28,100);put_word(m,MSG+4,0);put_word(m,old,7);put_word(m,over,11)
+            t=run(g,start,[end,mode,error],m,{'%rdi':MSG,'%rbp':MSG,'%rbx':value,'%rdx':width&0xffffffff,'%rsp':STACK});regs=t['registers']
+            guards.append({'requested_width':width,'input_word':value,'remaining_bytes':remaining,'outcome':'space_overflow' if t['stop']==end else 'width_error' if t['stop']==error else 'mode_dispatch','normalized_width_word':regs['r14'],'overflow_flag':get_word(t['memory'],MSG+4),'range_overflow_counter':get_word(t['memory'],over),'oldsize_word':get_word(t['memory'],old),**trace_fields(t,False)})
+        streams=[];base=initial['native'][label]['entry_request']['huff_init_pointer'];s=code['model_derived_seed_training']['final_state']
+        for sequence,bit in itertools.product(sequences,[0,7,8]):
+            m={};install_tree(m,s,base);install_tree(m,s,base+0xc830)
+            for x in graphs['MSG_ReadBits']['instructions']:
+                if x['opcode']=='movq' and '(%rip)' in x['operands']:put_word(m,global_address(x),s['root']+base+0xc830-0x10000,8)
+            for i in range(4096):m[DATA+i]=0
+            m[DATA-1]=m[DATA+4096]=0x7b
+            for offset,value in [(4,0),(8,0),(24,4096),(28,bit//8+1),(32,0),(36,bit)]:put_word(m,MSG+offset,value)
+            put_word(m,MSG+16,DATA,8);put_word(m,old,0);put_word(m,over,0);trace=[];writes=[];requests=[]
+            for value in sequence:
+                m,part,req=byte_write(graphs,m,value);trace+=part;requests+=req;writes.append([get_word(m,MSG+36),get_word(m,MSG+28)])
+            size=get_word(m,MSG+28);payload=[m[DATA+i] for i in range(size)];put_word(m,MSG+36,bit);put_word(m,MSG+32,0);decoded=[];reads=[]
+            for _ in sequence:
+                m,part,value=byte_read(graphs,m);trace+=part;decoded.append(value);reads.append([get_word(m,MSG+36),get_word(m,MSG+32)])
+            streams.append({'symbols':sequence,'initial_bit':bit,'payload_bytes':payload,'write_bit_size_pairs':writes,'read_bit_count_pairs':reads,'decoded_words':decoded,'transmit_requests':requests,'overflow_flag':get_word(m,MSG+4),'range_overflow_counter':get_word(m,over),'oldsize_word':get_word(m,old),'canaries':[m[DATA-1],m[DATA+4096]],**trace_fields({'trace':trace},False)})
+        out['native'][label]={'source':r['source'],'disassembly_sha256':a['uncompressed_sha256'],'graphs':graphs,'writer_guard_probes':guards,'byte_streams':streams}
+    write_json(Path('docs/research/message-byte-codec.json'),out);print('Recovered 576 writer guards and 36 modeled-tree native message-byte streams')
+
+
+if __name__=='__main__':main()

scripts/test_message_byte_codec.py

@@ -0,0 +1,82 @@
+#!/usr/bin/env python3
+"""Independent capacity priority, range, codeword packing and cursor oracles."""
+import json
+from pathlib import Path
+import unittest
+
+
+def signed(v):return v-(1<<32) if v&(1<<31) else v
+
+
+class MessageByteCodecTests(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        cls.r=json.loads(Path('docs/research/message-byte-codec.json').read_text());cls.words={w['symbol']:w['bits'] for w in json.loads(Path('docs/research/huffman-codebook.json').read_text())['model_derived_seed_training']['codebook']}
+
+    def test_space_guard_precedes_width_error_and_updates_oldsize(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['writer_guard_probes']),288)
+            for p in n['writer_guard_probes']:
+                self.assertEqual(p['oldsize_word'],(7+p['requested_width'])&0xffffffff)
+                if p['remaining_bytes']<=3:
+                    self.assertEqual(p['outcome'],'space_overflow');self.assertEqual(p['overflow_flag'],1);self.assertEqual(p['range_overflow_counter'],11)
+
+    def test_width_range_and_negative_normalization(self):
+        for n in self.r['native'].values():
+            for p in n['writer_guard_probes']:
+                if p['remaining_bytes']<=3:continue
+                width=p['requested_width'];valid=-31<=width<=32 and width!=0
+                self.assertEqual(p['outcome'],'mode_dispatch' if valid else 'width_error');self.assertEqual(p['overflow_flag'],0)
+                if valid:self.assertEqual(p['normalized_width_word'],abs(width))
+
+    def test_native_signed_range_counter_including_negative_width_shift(self):
+        for n in self.r['native'].values():
+            for p in n['writer_guard_probes']:
+                if p['outcome']!='mode_dispatch':continue
+                width=p['requested_width'];value=signed(p['input_word'])
+                if width==32:overflow=False
+                elif width>0:overflow=value<0 or value>(1<<width)-1
+                else:
+                    bound=1<<((width-1)&31);overflow=value < -bound or value > bound-1
+                self.assertEqual(p['range_overflow_counter'],11+int(overflow))
+
+    def test_independent_compressed_payload_and_per_byte_cursors(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['byte_streams']),18)
+            for p in n['byte_streams']:
+                bit=p['initial_bit'];pairs=[];bits=[]
+                for value in p['symbols']:
+                    code=self.words[value&255];bits+=code;bit+=len(code);pairs.append([bit,bit//8+1])
+                payload=[0]*(bit//8+1)
+                for i,value in enumerate(bits):position=p['initial_bit']+i;payload[position//8]|=value<<(position%8)
+                self.assertEqual(p['payload_bytes'],payload);self.assertEqual(p['write_bit_size_pairs'],pairs);self.assertEqual(p['read_bit_count_pairs'],pairs)
+
+    def test_decoded_low_byte_and_diagnostic_counter_not_message_overflow(self):
+        for n in self.r['native'].values():
+            for p in n['byte_streams']:
+                self.assertEqual(p['decoded_words'],[value&255 for value in p['symbols']]);self.assertEqual(p['overflow_flag'],0)
+                self.assertEqual(p['range_overflow_counter'],sum(value>255 for value in p['symbols']));self.assertEqual(p['oldsize_word'],8*len(p['symbols']));self.assertEqual(p['canaries'],[0x7b]*2)
+
+    def test_exact_boundary_extra_zero_byte_is_in_size(self):
+        found=0
+        for n in self.r['native'].values():
+            for p in n['byte_streams']:
+                bit,size=p['write_bit_size_pairs'][-1]
+                if bit%8==0:
+                    found+=1;self.assertEqual(size,bit//8+1);self.assertEqual(p['payload_bytes'][-1],0)
+        self.assertGreater(found,0)
+
+    def test_transmit_requests_forward_message_bit_pointer_and_masked_symbol(self):
+        for n in self.r['native'].values():
+            for p in n['byte_streams']:
+                self.assertEqual(len(p['transmit_requests']),len(p['symbols']))
+                bases={q[0] for q in p['transmit_requests']};self.assertEqual(len(bases),1)
+                for value,q in zip(p['symbols'],p['transmit_requests']):self.assertEqual(q[1:],[value&255,0xa0000,0x90024])
+
+    def test_paired_guard_and_byte_results_match(self):
+        a,b=self.r['native'].values()
+        def clean(rows):return [{k:([q[1:] for q in v] if k=='transmit_requests' else v) for k,v in p.items() if k not in ['trace','trace_sha256']} for p in rows]
+        self.assertEqual(clean(a['writer_guard_probes']),clean(b['writer_guard_probes']));self.assertEqual(clean(a['byte_streams']),clean(b['byte_streams']))
+
+
+if __name__=='__main__':unittest.main()