Recover scalar message wrappers and data overflow continuation

docs/ASSET_PIPELINE_TODO.md

@@ -228,3 +228,5 @@ Codebook follow-up: [provisional modeled codewords](research/HUFFMAN_CODEBOOK.md
 Message-codec follow-up: [modeled-tree byte streams](research/MESSAGE_BYTE_CODEC.md) validates native message/helper compositions and byte accounting. Full native codebook, general widths and real packet/buffer contracts remain prerequisites for network/map/renderer integration (RE-368).
 
 Research progress: [message width codecs](research/MESSAGE_WIDTHS.md) adds bounded record/OOB/mixed-width contracts (RE-369–371). RE-372 leaves wrappers, lifecycle, real packet/runtime/platform and native seeded-codebook validation open; no network recoding/conversion is declared complete.
+
+Research progress: [message wrappers](research/MESSAGE_WRAPPERS.md) adds scalar OOB and data overflow contracts (RE-373–375). Broader codec/packet conversion milestones remain open; RE-376 tracks compressed/read/string/delta/snapshot and runtime/platform validation.

docs/FINDINGS.md

@@ -414,3 +414,9 @@ Bounded qagame paths recover bomb-plant/inactivity correction requests, direct f
 **F-137 — Observed OOB composition:** accepted 8/16/32-bit writes store low bytes little-endian and advance size/bit directly; selected negative widths normalize, while -32 is rejected by writer validation. Matching reads retain byte order and signed tails. Capacity/EOF/real-header/runtime coverage remains open. [WIDTH-002](research/MESSAGE_WIDTHS.md).
 
 **F-138 — Observed mixed-width composition:** compressed writes emit raw low-bit remainder then modeled-tree Huffman bytes. Independent packing and native reader compositions agree for every positive width 1–32 and selected negative widths; negative sign tails use the mutated remaining byte width. Native seed/codeword, legal-caller, buffer and real-packet/platform conformance remain open. [WIDTH-003](research/MESSAGE_WIDTHS.md).
+
+**F-139 — Observed scalar forwarding:** Byte/Char/Short/Long wrappers request positive widths 8/8/16/32 without separate local validation. Bounded OOB writer/read compositions verify little-endian low-byte results, guarded write failures and post-read EOF. Compressed wrappers/real-packet/runtime/platform coverage remains open. [WRAP-001](research/MESSAGE_WRAPPERS.md).
+
+**F-140 — Observed scalar reader tails:** Byte zero extends, Char/Short sign extend and Long preserves the word, gated by signed post-read count <= size. EOF returns 0xffffffff after reading; valid signed -1 values can share that word. Legal cursors, caller interpretation and access safety remain open. [WRAP-002](research/MESSAGE_WRAPPERS.md).
+
+**F-141 — Observed data overflow continuation:** bounded WriteData compositions request every positive-length source byte even after child capacity failure. Only the permitted prefix is stored; oldsize counts all requests. A pre-set overflow flag is retained but does not itself block ample-capacity stores. Compressed/allowoverflow/lifecycle/alias/real-packet coverage remains open. [WRAP-003](research/MESSAGE_WRAPPERS.md).

docs/PROVISIONAL_TODO.md

@@ -540,3 +540,5 @@ Research progress: [differential Huffman codebook](research/HUFFMAN_CODEBOOK.md)
 Research progress: [message byte codec](research/MESSAGE_BYTE_CODEC.md) completes bounded RE-365–367. Broad network/runtime milestones remain open; RE-368 tracks general message/snapshot codecs, overflow/truncation/padding and real native packet/platform validation.
 
 Research progress: [message width codecs](research/MESSAGE_WIDTHS.md) completes bounded RE-369–371. Broad network/runtime milestones remain open; RE-372 tracks wrappers/delta/snapshot codecs, message lifecycle and actual packet/platform validation.
+
+Research progress: [message wrappers](research/MESSAGE_WRAPPERS.md) completes bounded RE-373–375. RE-376 leaves ReadData/compressed/string/delta/snapshot compositions, sentinel/lifecycle/alias and actual packet/platform validation open.

docs/RESEARCH_LEDGER.md

@@ -233,3 +233,7 @@ Completed bounded RE-365–367 with `analyze_message_byte_codec.py`: ten verifie
 ## 2026-10-11 — Message initialization, OOB and mixed-width codecs
 
 Completed bounded RE-369–371 with `analyze_message_widths.py`: twenty verified graphs across the Linux ELF64 engines, 24 initialization probes, four clear probes, 78 OOB probes and 328 compressed writer/reader compositions. WIDTH-001–003/F-136–138 distinguish warm 40-byte record reset from the cold first Huff_Init boundary, clear's overflow/size/bit reset from retained readcount/mode/data/maxsize, and direct OOB byte accounting from compressed size/readcount. Accepted OOB widths emit low bytes little-endian; all positive compressed widths 1–32 and selected negative widths compose raw-bit remainder followed by modeled-tree Huffman bytes. Independent packing verifies exact payloads and cursors; generated writer bytes confirm the reader's mutated-width signed tail. The seeded tree remains MODEL-DERIVED. Prologue/epilogue omissions, stopped CALL boundaries and explicit caller-register restoration are documented; no original process or native callee executes. Eight tests verify initialization/cold boundary/canaries, clear retention, OOB errors/endian/sign/cursors, raw/symbol requests, independent packing, signed tails/diagnostics/accounting, paired outcomes and narrow MOVW/BX/implicit-one shift evaluator support. All 350 tests pass; new and preceding reports regenerate byte-identically, with storage/link/whitespace checks passing. RE-368 remains open; RE-372 tracks wrappers/string/float/angle/delta/entity/player/snapshot codecs, capacity/overflow/allowoverflow/reset/truncation/EOF/prefix/padding/legal-width invariants, native seeded codewords and actual packet/runtime/all-platform/renderer contracts.
+
+## 2026-10-11 — Scalar message wrappers and data overflow
+
+Completed bounded RE-373–375 with `analyze_message_wrappers.py`: 22 verified graphs across the Linux ELF64 engines, 64 scalar OOB compositions, 1,280 supplied reader-tail probes and 300 data OOB compositions. WRAP-001–003/F-139–141 distinguish fixed positive wrapper widths from result signedness, post-read signed cursor/size comparisons from pre-read safety, and valid signed -1 words from EOF sentinels. Scalar compositions evaluate actual OOB writer/reader instructions against ample or guarded capacities; separate reader-tail probes supply return/count effects explicitly. WriteData composes each child OOB byte write, continuing all requests after space failure, retaining sticky flags even when ample writes proceed and counting every request in oldsize. Source/buffer bytes and canaries verify exact prefixes and unchanged tails; disjoint supplied wrapper/child frames and explicit caller-register retention preserve loop state. PUSH/POP and save/restore frames remain excluded/stopped; no original process or native callee executes. Eight independent tests verify forwarding/diagnostics, OOB bytes/cursors/post-read EOF, signed conversion/count comparisons, every data request, stored prefixes/cursors, overflow continuation/flag retention/canaries, paired outcomes and restricted AX/MOVSBL/MOVSWL evaluator support. All 358 tests pass; new and preceding reports regenerate byte-identically and storage/link/whitespace checks pass. RE-372 remains open; RE-376 tracks ReadData/compressed/string/float/angle/delta/entity/player/snapshot compositions, caller sentinel/alias/capacity/allowoverflow/reset/truncation/EOF/padding/legal cursor/length invariants, native seeded codewords and actual packet/runtime/all-platform/renderer contracts.

docs/REVERSE_ENGINEERING_TODO.md

@@ -447,3 +447,8 @@ Purpose: turn the supplied installation into an evidence-backed specification of
 - [x] RE-370 Compose bounded little-endian OOB writes/reads, width dispatch/errors and byte/bit cursors ([WIDTH-002](research/MESSAGE_WIDTHS.md)).
 - [x] RE-371 Compose positive widths 1–32 and selected signed compressed writes/reads through raw-bit and modeled-tree Huffman helpers ([WIDTH-003](research/MESSAGE_WIDTHS.md)).
 - [ ] RE-372 Recover message byte/short/long/data/read/string/float/angle/delta/entity/player wrappers and snapshot codecs; validate capacity/overflow/allowoverflow/reset/truncation/EOF/dirty-prefix/padding/legal-signed-width invariants, native seeded codewords and actual packet/runtime/all-platform/renderer contracts.
+
+- [x] RE-373 Recover scalar writer/reader fixed-width forwarding and compose bounded OOB writes/reads through the wrappers ([WRAP-001](research/MESSAGE_WRAPPERS.md)).
+- [x] RE-374 Recover scalar signed/unsigned conversion and signed post-read EOF comparisons across supplied return/count/size boundaries ([WRAP-002](research/MESSAGE_WRAPPERS.md)).
+- [x] RE-375 Compose native data-writer loops with OOB child writes, including exact stored prefixes, continued requests and sticky overflow/accounting ([WRAP-003](research/MESSAGE_WRAPPERS.md)).
+- [ ] RE-376 Compose ReadData/compressed scalar/data/string/float/angle/delta/entity/player/snapshot codecs; recover caller sentinel/alias/capacity/allowoverflow/reset/truncation/EOF/padding/legal cursor/length invariants, native seeded codewords and actual packet/runtime/all-platform/renderer contracts.

docs/research/MESSAGE_WIDTHS.md

@@ -65,3 +65,5 @@ These are recovered local behaviors, not corrected signed-codec specifications o
 Eight independent tests verify warm/cold record effects and canaries, exact clear retention, OOB width errors/little-endian/sign/cursors, raw-remainder and symbol arguments, independent modeled-codeword packing, signed-tail/diagnostic/accounting effects, paired engine results, and the restricted evaluator extension. The report preserves graph provenance and compact composition trace hashes.
 
 RE-372: recover MSG_WriteByte/Short/Long/Data and read wrappers, string/float/angle/delta/entity/player codecs and snapshot messages; validate capacity/overflow/allowoverflow/clear/read-reset lifecycle, truncation/EOF/dirty-prefix/padding and legal signed-width callers. Complete native seed/codeword validation, actual packets, original-runtime/all-platform and renderer contracts remain open.
+
+Follow-up: [message wrappers](MESSAGE_WRAPPERS.md) recovers bounded scalar forwarding/OOB composition, supplied-result signed/EOF tails and data-writer overflow continuation. Broader RE-372 remains open; RE-376 tracks remaining wrapper/lifecycle and actual packet/platform contracts.

docs/research/MESSAGE_WRAPPERS.md

@@ -0,0 +1,50 @@
+# Scalar message wrappers and data-writer overflow
+
+Reproduce with `python3 scripts/analyze_message_wrappers.py`; validate with `python3 scripts/test_message_wrappers.py`. [message-wrappers.json](message-wrappers.json) retains 22 source/disassembly-hash-verified graphs across the Linux ELF64 dedicated/client engines: four scalar writers, four scalar readers, WriteData, WriteBits and ReadBits per image. It contains 64 scalar OOB compositions, 1,280 supplied reader-tail probes and 300 data OOB compositions. Synthetic source and buffer contents are stored as hexadecimal strings to keep the ordinary Git report below the repository's 2 MiB limit.
+
+Original processes and native callees never execute. These are synthetic instruction compositions with explicitly supplied memory/registers, omitted/stopped PUSH/POP and register-save/restore frames, and explicit caller-register retention. Scalar EOF tail probes supply ReadBits return words and post-read counts. Separate scalar OOB and data OOB compositions evaluate underlying writer/reader instructions; padded backing buffers do not prove out-of-message reads are safe. No Huffman tree is needed for these OOB compositions. Compressed wrappers, real packets, complete lifecycle/alias/wrap/runtime/all-platform behavior remain open.
+
+## WRAP-001 — Scalar forwarding and bounded OOB composition (RE-373)
+
+MSG_WriteByte and MSG_WriteChar forward their unchanged message/value arguments to MSG_WriteBits with positive width eight. MSG_WriteShort forwards width sixteen; MSG_WriteLong forwards width thirty-two. Each is a two-instruction width assignment and tail jump. These local wrappers do not separately mask, validate or test overflow before forwarding. Character writes request positive width eight even though character reads later convert to signed eight-bit values.
+
+The scalar reader entry slices similarly request positive widths eight/eight/sixteen/thirty-two for Byte/Char/Short/Long. They retain the message pointer for the post-read size check. The native helper CALL boundary is stopped and composed explicitly; actual ABI execution is not claimed.
+
+Each image has 32 scalar OOB compositions: all four scalar types, input words zero/127/0x8196a5ff/0xffffffff, and capacities three/64. At capacity 64, the forwarded writer stores the value's low bytes little-endian and advances cursize and bit by the type width. Subsequent OOB ReadBits instructions consume those bytes; the wrapper then converts/checks the result. At capacity three, the writer's early space guard sets overflow without storing bytes or advancing size/bit. The supplied padded data remains 0xa5. The underlying reader still consumes its type's byte count, after which the wrapper returns 0xffffffff because readcount exceeds the zero cursize.
+
+The range diagnostic occurs only after the capacity guard; positive width thirty-two skips that diagnostic. Requested-bit oldsize increments by the positive fixed width even when the sampled write fails. Forwarding, payloads, read counts, bit counts, flags, diagnostics and canaries have independent test oracles.
+
+## WRAP-002 — Scalar conversion and post-read EOF (RE-374)
+
+For every sampled reader tail, native signed comparison tests post-read readcount at +32 against cursize at +28. It returns the converted ReadBits word when signed readcount is less than or equal to signed cursize; otherwise it returns 0xffffffff. The comparison occurs after the underlying read, not as a pre-read capacity guard. Error policy and access safety remain open.
+
+Conversion rules recovered from the tails are:
+
+| Wrapper | Successful local result |
+| --- | --- |
+| ReadByte | Low eight bits, zero extended |
+| ReadChar | Low eight bits, sign extended to a 32-bit word |
+| ReadShort | Low sixteen bits, sign extended to a 32-bit word |
+| ReadLong | Full 32-bit word |
+
+There are 640 independent tail probes per engine: four types, size words 0/1/32/0xffffffff, post-count words 0/1/33/0x80000000/0xffffffff and eight supplied return words. Signed-negative cursor words can satisfy the local comparison against nonnegative sizes. This characterizes the instruction tail; it does not establish that such cursors are reachable in legal packets.
+
+For supplied value 0x8196a5ff with an accepted count, Byte returns 0x000000ff, Char returns 0xffffffff, Short returns 0xffffa5ff and Long returns 0x8196a5ff. Successful signed -1 values can share the EOF sentinel word; the surrounding caller/error interpretation remains unverified. The evaluator adds narrowly scoped AX, MOVSBL and MOVSWL support with independent register/memory sign-extension checks. It does not add arbitrary signed/vector instructions or native ABI execution.
+
+## WRAP-003 — Data iteration continues after writer overflow (RE-375)
+
+MSG_WriteData uses a signed nonpositive-length early return. For each positive sampled length it loads source bytes as unsigned values and requests MSG_WriteBits(message, byte, 8) for every source byte. Its local unrolling covers the first byte, residual runs and eight-byte blocks. It does not inspect the overflow flag or stop issuing requests when child writes stop advancing size.
+
+Each image has 150 data compositions: lengths -1/0/1–9/16/17/31/32, capacities 0/3/4/8/40 and initial overflow flags zero/one. Synthetic source bytes follow `(129 + 37*i) & 255`. An explicit disjoint wrapper stack preserves loop state while the underlying OOB byte writer is composed with separate supplied frames; child memory effects are retained and caller registers restored.
+
+Starting at cursize zero, the early three-byte-reserve guard permits `min(max(length,0), max(capacity-3,0))` byte stores for these nonnegative capacities. The rest of the 64-byte backing buffer retains 0xa5. At capacity eight and length nine, the final stored prefix has five bytes; all nine requests still occur, and post-call sizes are 1/2/3/4/5/5/5/5/5. Bit starts at seven and advances only for successful stores, ending at 47 in this example. The initial bit value is deliberately distinct from the OOB byte offset.
+
+Overflow becomes one on the first failed guard and remains one. An initially set overflow flag does not itself prevent later stores when capacity is ample: at capacity forty, every sampled positive length writes completely while the flag remains one. Nonpositive lengths leave buffer/size/bit/flag unchanged and issue no requests. Oldsize, seeded at five, becomes `5 + 8*max(length,0)` even for requests rejected by the guard. No range diagnostics arise from these unsigned source bytes.
+
+These results do not determine allowoverflow recovery, actual maxsize/cursize legality, wrapping offsets/lengths, source-buffer ownership/aliasing, compressed expansion or complete networking callers. ReadData still requires real-reader composition; its earlier bounded [supplied-result analysis](CLIENT_MASKS.md) is not replaced by this writer report.
+
+## Validation and next work
+
+Eight tests verify scalar width/argument forwarding and diagnostics, OOB payloads/cursors and post-read EOF, signed size/count conversion, every data request/source byte, exact stored prefixes/tails/cursors, sticky flags and continuation/canaries, paired results and restricted signed-load evaluator behavior. All local sampled paths are bounded; trace hashes preserve the composed instruction sequences.
+
+RE-376: compose ReadData, compressed scalar/data wrappers and strings/float/angle/delta/entity/player/snapshot codecs; recover caller sentinel handling, source/destination aliasing and capacity/allowoverflow/reset/truncation/EOF/padding/legal cursor and length invariants. Native full-training/codeword validation, real packets, original-runtime/all-platform and renderer contracts remain open.

docs/research/README.md

@@ -141,3 +141,5 @@ The extension collector adds five focused tests for native tail calls, command-r
 [Message byte codec and writer guards](MESSAGE_BYTE_CODEC.md) adds RE-365–367: space/width/range priorities and native byte writer/reader compositions against model-derived trees. Reproduce with `python3 scripts/analyze_message_byte_codec.py`; check with `python3 scripts/test_message_byte_codec.py`. General codecs, padding/overflow/truncation, native codebook and actual packet/runtime/platform coverage remain open.
 
 [Message initialization, OOB and mixed-width codecs](MESSAGE_WIDTHS.md) adds RE-369–371: warm/cold record effects, exact clear retention and native writer/reader compositions for OOB and raw-remainder/Huffman paths. Reproduce with `python3 scripts/analyze_message_widths.py`; check with `python3 scripts/test_message_widths.py`. Wrappers, lifecycle, real packets and runtime/platform validation remain open.
+
+[Scalar message wrappers and data overflow](MESSAGE_WRAPPERS.md) adds RE-373–375: fixed-width forwarding, scalar conversion/post-read EOF, and data-writer continuation after child space failure. Reproduce with `python3 scripts/analyze_message_wrappers.py`; check with `python3 scripts/test_message_wrappers.py`. ReadData, compressed/higher-level codecs and actual packet/runtime/platform coverage remain open.

scripts/analyze_collision_composition.py

@@ -13,7 +13,7 @@ from analyze_vm_extensions import native_functions,verified
 
 
 def register(r):
-    aliases={'eax':('rax',32),'edi':('rdi',32),'esi':('rsi',32),'edx':('rdx',32),
+    aliases={'eax':('rax',32),'ax':('rax',16),'edi':('rdi',32),'esi':('rsi',32),'edx':('rdx',32),
              'ecx':('rcx',32),'ebp':('rbp',32),'ebx':('rbx',32),'bx':('rbx',16),'bl':('rbx',8),'al':('rax',8),'sil':('rsi',8),'dl':('rdx',8),'cl':('rcx',8),'dil':('rdi',8),'bpl':('rbp',8)}
     r=r.lstrip('%')
     if r in aliases:return aliases[r]
@@ -66,6 +66,9 @@ def evaluate(graph,start,stops,memory,registers,limit=128,capture_registers=Fals
             a,b=args.split(', ');width=8 if op in ['movq','movabsq'] else 1 if op=='movb' else 2 if op=='movw' else 4;write(b,read(a,width),width)
         elif op in ['movzbl','movzwl']:
             a,b=args.split(', ');write(b,read(a,1 if op=='movzbl' else 2),4)
+        elif op in ['movsbl','movswl']:
+            a,b=args.split(', ');width=1 if op=='movsbl' else 2;v=read(a,width)&((1<<(width*8))-1);sign=1<<(width*8-1)
+            write(b,(v^sign)-sign,4)
         elif op=='sbbl':
             if below is None:raise ValueError('unmodeled carry flag')
             a,b=args.split(', ');write(b,read(b,4)-read(a,4)-int(below),4)

scripts/analyze_message_wrappers.py

@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+"""Scalar wrapper boundaries and native OOB data overflow compositions."""
+import copy
+import gzip
+import itertools
+import json
+from pathlib import Path
+from analyze_area_queries import site
+from analyze_bsp_references import global_address, trace_fields
+from analyze_collision_composition import get_word, put_word
+from analyze_engine_control import control_flow
+from analyze_message_byte_codec import MSG, DATA
+from analyze_message_widths import reader, run, writer
+from analyze_qvm import write_json
+from analyze_vm_extensions import native_functions, verified
+
+SOURCE, WRAPPER_STACK=0xd0000,0xc0000
+
+
+def read_prefix(g):
+    g=copy.deepcopy(g)
+    for x in g['instructions']:
+        if x['opcode'] in ['pushq','popq']:x['opcode']='nop';x['operands']=''
+    rows=g['instructions'];i=next(i for i,x in enumerate(rows) if '<MSG_ReadBits>' in x['operands'])
+    t=run(g,rows[0]['address'],[rows[i]['address']],{}, {'%rdi':MSG})
+    return g,i,t
+
+
+def read_tail(g,i,registers,m,value):
+    regs=dict(registers);regs['rax']=value
+    t=run(g,g['instructions'][i+1]['address'],[x['address'] for x in g['instructions'] if x['opcode']=='retq'],m,regs)
+    return t['registers']['rax'],t['trace']
+
+
+def message_memory(capacity,flag=0):
+    m={MSG+i:0 for i in range(40)}
+    for i in range(64):m[DATA+i]=0xa5
+    m[DATA-1]=m[DATA+64]=0x7b
+    put_word(m,MSG+4,flag);put_word(m,MSG+8,1);put_word(m,MSG+16,DATA,8);put_word(m,MSG+24,capacity);put_word(m,MSG+36,7)
+    return m
+
+
+def main():
+    out={'native':{},'limitations':'Synthetic Linux ELF64 instruction compositions. Original processes and native callees never execute. Scalar reader tails use supplied ReadBits return/count; separate ample/overflow scalar OOB compositions and data OOB compositions evaluate underlying writer/reader instructions. PUSH/POP and frame saves/restores omitted/stopped, caller registers explicitly retained. Real packets/compressed data loops/EOF access safety/legal cursor/allowoverflow/runtime/all-platform conformance remain open.'}
+    widths={'Byte':8,'Char':8,'Short':16,'Long':32}
+    for label in ['linux--Quake3-UrT-Ded.x86_64','linux--Quake3-UrT.x86_64']:
+        r=json.loads(Path(f'docs/research/native/{label}.json').read_text());verified((Path('.cache/UrbanTerror43')/r['source']['path']).read_bytes(),r['source']['sha256']);a=r['artifacts']['disassembly.txt'];fs=native_functions(verified(gzip.open(a['path'],'rb').read(),a['uncompressed_sha256']).decode());names=['MSG_Write'+s for s in widths]+['MSG_Read'+s for s in widths]+['MSG_WriteData','MSG_WriteBits','MSG_ReadBits'];graphs={n:control_flow(fs[n]) for n in names}
+        g=graphs['MSG_WriteBits'];old=global_address(next(x for x in g['instructions'] if x['opcode']=='addl' and '(%rip)' in x['operands']));over=global_address(next(x for x in g['instructions'] if x['opcode']=='addl' and '$0x1,' in x['operands'] and '(%rip)' in x['operands']))
+        scalar=[];tails=[]
+        for suffix,width in widths.items():
+            wg=graphs['MSG_Write'+suffix];jump=next(x for x in wg['instructions'] if x['opcode']=='jmp');rg,ci,prefix=read_prefix(graphs['MSG_Read'+suffix]);args=[prefix['registers'][k] for k in ['rdi','rsi']]
+            for value,capacity in itertools.product([0,127,0x8196a5ff,0xffffffff],[3,64]):
+                forward=run(wg,wg['instructions'][0]['address'],[jump['address']],{}, {'%rdi':MSG,'%rsi':value});wargs=[forward['registers'][k] for k in ['rdi','rsi','rdx']]
+                m=message_memory(capacity);put_word(m,old,0);put_word(m,over,0)
+                m,requests,error,wt=writer(graphs,m,wargs[2],wargs[1]);size=get_word(m,MSG+28);bit=get_word(m,MSG+36);flag=get_word(m,MSG+4);counter=get_word(m,over);account=get_word(m,old)
+                payload=[m[DATA+i] for i in range(size)];put_word(m,MSG+32,0);put_word(m,MSG+36,7)
+                # Padded backing bytes allow observing the post-read EOF guard,
+                # not claiming that an out-of-message read is memory safe.
+                m,raw,read_error,rt=reader(graphs,m,args[1]);result,tail=read_tail(rg,ci,prefix['registers'],m,raw)
+                scalar.append({'suffix':suffix,'value_word':value,'capacity':capacity,'write_arguments':wargs,'read_arguments':args,'write_error':error,'read_error':read_error,'write_requests':requests,'payload_bytes':payload,'write_size':size,'write_bit':bit,'read_count':get_word(m,MSG+32),'read_bit':get_word(m,MSG+36),'raw_read_word':raw,'result_word':result,'overflow_flag':flag,'range_counter':counter,'oldsize_word':account,'canaries':[m[DATA-1],m[DATA+64]],**trace_fields({'trace':forward['trace']+wt+prefix['trace']+rt+tail},False)})
+            for size,count,value in itertools.product([0,1,32,0xffffffff],[0,1,33,0x80000000,0xffffffff],[0,127,128,255,32767,32768,0x8196a5ff,0xffffffff]):
+                m={};put_word(m,MSG+28,size);put_word(m,MSG+32,count);result,trace=read_tail(rg,ci,prefix['registers'],m,value)
+                tails.append({'suffix':suffix,'size_word':size,'post_count_word':count,'supplied_read_word':value,'result_word':result,'read_arguments':args,**trace_fields({'trace':trace},False)})
+        data=[];dg=graphs['MSG_WriteData'];rows=dg['instructions'];start=site(dg,'testl','%edx, %edx');end=site(dg,'addq','$0x8, %rsp');calls={x['address']:i for i,x in enumerate(rows) if x['opcode']=='callq'}
+        for length,capacity,flag in itertools.product([-1,0,1,2,3,4,5,6,7,8,9,16,17,31,32],[0,3,4,8,40],[0,1]):
+            m=message_memory(capacity,flag);source=[(129+37*i)&255 for i in range(32)]
+            for i,v in enumerate(source):m[SOURCE+i]=v
+            m[SOURCE-1]=m[SOURCE+32]=0x7b;put_word(m,old,5);put_word(m,over,0)
+            regs={'%r15':length&0xffffffff,'%r14':SOURCE,'%r12':MSG,'%rdx':length&0xffffffff,'%rsi':SOURCE,'%rdi':MSG,'%rsp':WRAPPER_STACK};pc=start;trace=[];requests=[];sizes=[];flags=[]
+            while True:
+                t=run(dg,pc,list(calls)+[end],m,regs);m=t['memory'];regs=t['registers'];trace+=t['trace']
+                if t['stop']==end:break
+                i=calls[t['stop']]
+                if '<MSG_WriteBits>' not in rows[i]['operands']:raise ValueError('unexpected data helper')
+                request=[regs[k] for k in ['rdi','rsi','rdx']];requests.append(request)
+                m,child_requests,error,child=writer(graphs,m,request[2],request[1]);trace+=child
+                if child_requests or error:raise ValueError('unexpected OOB byte write outcome')
+                sizes.append(get_word(m,MSG+28));flags.append(get_word(m,MSG+4));pc=rows[i+1]['address']
+                if len(requests)>32:raise ValueError('data fixture bound')
+            size=get_word(m,MSG+28)
+            data.append({'length':length,'capacity':capacity,'initial_overflow':flag,'source_hex':bytes(source).hex(),'requests':requests,'post_write_sizes':sizes,'post_overflow_flags':flags,'size':size,'bit':get_word(m,MSG+36),'overflow_flag':get_word(m,MSG+4),'oldsize_word':get_word(m,old),'range_counter':get_word(m,over),'buffer_hex':bytes(m[DATA+i] for i in range(64)).hex(),'source_canaries':[m[SOURCE-1],m[SOURCE+32]],'buffer_canaries':[m[DATA-1],m[DATA+64]],**trace_fields({'trace':trace},False)})
+        out['native'][label]={'source':r['source'],'disassembly_sha256':a['uncompressed_sha256'],'graphs':graphs,'scalar_oob':scalar,'reader_tails':tails,'data_oob':data}
+    write_json(Path('docs/research/message-wrappers.json'),out)
+    print('Recovered scalar OOB wrappers, supplied reader tails and data overflow compositions')
+
+
+if __name__=='__main__':main()

scripts/test_message_wrappers.py

@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+"""Independent scalar conversion, EOF and bounded data overflow oracles."""
+import json
+from pathlib import Path
+import unittest
+from analyze_collision_composition import evaluate
+from test_bsp_planes import graph
+
+
+def signed(word,bits=32):
+    mask=(1<<bits)-1;word&=mask;return (word^(1<<(bits-1)))-(1<<(bits-1))
+
+
+def converted(suffix,value):
+    if suffix=='Byte':return value&255
+    if suffix=='Char':return signed(value,8)&0xffffffff
+    if suffix=='Short':return signed(value,16)&0xffffffff
+    return value&0xffffffff
+
+
+class MessageWrapperTests(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):cls.r=json.loads(Path('docs/research/message-wrappers.json').read_text())
+
+    def test_scalar_fixed_width_forwarding(self):
+        widths={'Byte':8,'Char':8,'Short':16,'Long':32}
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['scalar_oob']),32)
+            for p in n['scalar_oob']:
+                width=widths[p['suffix']];self.assertEqual(p['write_arguments'],[0x90000,p['value_word'],width]);self.assertEqual(p['read_arguments'],[0x90000,width]);self.assertEqual(p['oldsize_word'],width);self.assertEqual(p['write_requests'],[]);self.assertIsNone(p['write_error']);self.assertIsNone(p['read_error'])
+                diagnostic=p['capacity']>3 and width!=32 and (signed(p['value_word'])<0 or signed(p['value_word'])>(1<<width)-1)
+                self.assertEqual(p['range_counter'],int(diagnostic))
+
+    def test_scalar_oob_bytes_and_post_read_eof(self):
+        widths={'Byte':8,'Char':8,'Short':16,'Long':32}
+        for n in self.r['native'].values():
+            for p in n['scalar_oob']:
+                width=widths[p['suffix']];amount=width//8;accepted=p['capacity']>3;value=p['value_word']&((1<<width)-1)
+                self.assertEqual(p['payload_bytes'],list(value.to_bytes(amount,'little')) if accepted else [])
+                self.assertEqual(p['write_size'],amount if accepted else 0);self.assertEqual(p['write_bit'],7+width if accepted else 7)
+                self.assertEqual(p['read_count'],amount);self.assertEqual(p['read_bit'],7+width)
+                self.assertEqual(p['raw_read_word'],value if accepted else int.from_bytes(bytes([0xa5])*amount,'little'))
+                self.assertEqual(p['result_word'],converted(p['suffix'],value) if accepted else 0xffffffff);self.assertEqual(p['overflow_flag'],int(not accepted));self.assertEqual(p['canaries'],[0x7b]*2)
+
+    def test_reader_tail_signed_count_comparison_and_conversion(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['reader_tails']),640)
+            for p in n['reader_tails']:
+                expected=converted(p['suffix'],p['supplied_read_word']) if signed(p['post_count_word'])<=signed(p['size_word']) else 0xffffffff
+                self.assertEqual(p['result_word'],expected)
+
+    def test_data_requests_include_every_source_byte_after_overflow(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['data_oob']),150)
+            for p in n['data_oob']:
+                count=max(0,p['length']);source=[(129+37*i)&255 for i in range(32)]
+                self.assertEqual(list(bytes.fromhex(p['source_hex'])),source);self.assertEqual(p['requests'],[[0x90000,v,8] for v in source[:count]])
+                self.assertEqual(p['oldsize_word'],5+8*count);self.assertEqual(p['range_counter'],0)
+
+    def test_data_space_guard_exact_prefix_and_cursor(self):
+        for n in self.r['native'].values():
+            for p in n['data_oob']:
+                count=max(0,p['length']);accepted=min(count,max(0,p['capacity']-3));source=bytes.fromhex(p['source_hex'])
+                expected=source[:accepted]+bytes([0xa5])*(64-accepted)
+                self.assertEqual(bytes.fromhex(p['buffer_hex']),expected);self.assertEqual(p['size'],accepted);self.assertEqual(p['bit'],7+8*accepted)
+                self.assertEqual(p['post_write_sizes'],[min(i+1,accepted) for i in range(count)])
+
+    def test_data_flag_sticky_but_does_not_prevent_ample_writes(self):
+        for n in self.r['native'].values():
+            for p in n['data_oob']:
+                count=max(0,p['length']);accepted=min(count,max(0,p['capacity']-3));flags=[int(bool(p['initial_overflow']) or i>=accepted) for i in range(count)]
+                self.assertEqual(p['post_overflow_flags'],flags);self.assertEqual(p['overflow_flag'],int(bool(p['initial_overflow']) or count>accepted));self.assertEqual(p['source_canaries'],[0x7b]*2);self.assertEqual(p['buffer_canaries'],[0x7b]*2)
+                if p['length']<=0:self.assertEqual(p['requests'],[]);self.assertEqual(p['size'],0)
+
+    def test_paired_results(self):
+        a,b=self.r['native'].values()
+        for key in ['scalar_oob','reader_tails','data_oob']:
+            clean=lambda rows:[{k:v for k,v in p.items() if k not in ['trace','trace_sha256']} for p in rows]
+            self.assertEqual(clean(a[key]),clean(b[key]))
+
+    def test_restricted_signed_loads_and_ax(self):
+        g=graph([('movsbl','%al, %ecx'),('movswl','%ax, %edx'),('movsbl','(%rdi), %esi'),('movswl','0x1(%rdi), %eax'),('nop','')])
+        t=evaluate(g,0,[4],{100:0x80,101:0xff,102:0x7f},{'%rax':0x123480ff,'%rdi':100},capture_registers=True)
+        self.assertEqual(t['registers']['rcx'],0xffffffff);self.assertEqual(t['registers']['rdx'],0xffff80ff);self.assertEqual(t['registers']['rsi'],0xffffff80);self.assertEqual(t['registers']['rax'],0x7fff)
+
+
+if __name__=='__main__':unittest.main()