Recover OOB data reads and message string codecs

docs/ASSET_PIPELINE_TODO.md

@@ -230,3 +230,5 @@ Message-codec follow-up: [modeled-tree byte streams](research/MESSAGE_BYTE_CODEC
 Research progress: [message width codecs](research/MESSAGE_WIDTHS.md) adds bounded record/OOB/mixed-width contracts (RE-369–371). RE-372 leaves wrappers, lifecycle, real packet/runtime/platform and native seeded-codebook validation open; no network recoding/conversion is declared complete.
 
 Research progress: [message wrappers](research/MESSAGE_WRAPPERS.md) adds scalar OOB and data overflow contracts (RE-373–375). Broader codec/packet conversion milestones remain open; RE-376 tracks compressed/read/string/delta/snapshot and runtime/platform validation.
+
+Research progress: [message strings and data reads](research/MESSAGE_STRINGS.md) adds OOB reader/cursor/string contracts (RE-377–379). RE-380 retains compressed/library/caller/higher-level packet/runtime/platform work; no broad network conversion milestone is complete.

docs/FINDINGS.md

@@ -420,3 +420,9 @@ Bounded qagame paths recover bomb-plant/inactivity correction requests, direct f
 **F-140 — Observed scalar reader tails:** Byte zero extends, Char/Short sign extend and Long preserves the word, gated by signed post-read count <= size. EOF returns 0xffffffff after reading; valid signed -1 values can share that word. Legal cursors, caller interpretation and access safety remain open. [WRAP-002](research/MESSAGE_WRAPPERS.md).
 
 **F-141 — Observed data overflow continuation:** bounded WriteData compositions request every positive-length source byte even after child capacity failure. Only the permitted prefix is stored; oldsize counts all requests. A pre-set overflow flag is retained but does not itself block ample-capacity stores. Compressed/allowoverflow/lifecycle/alias/real-packet coverage remains open. [WRAP-003](research/MESSAGE_WRAPPERS.md).
+
+**F-142 — Observed OOB ReadData EOF continuation:** actual OOB read instructions advance cursors for every positive-length request; ReadData stores valid prefix bytes then 0xff after post-read size failure. Requests continue and destination tail/canaries retain supplied contents. Truncated backing safety, compressed/alias/lifecycle/runtime behavior remains open. [TEXT-001](research/MESSAGE_STRINGS.md).
+
+**F-143 — Observed string readers:** ordinary/line/big readers terminate on consumed NUL/EOF, with line also consuming newline; capacities store at most 1023/1023/8191 bytes. They return recovered static buffers and replace percent/high bytes with periods. Caller lifetime/compressed/malformed/platform coverage remains open. [TEXT-002](research/MESSAGE_STRINGS.md).
+
+**F-144 — Observed string writers:** signed low-word length guards use limits 1024/8192; null/over-limit paths request an empty byte. Small supplied-copy fixtures sanitize high bytes, retain percent and request every byte plus NUL even after child space failure. Native library-copy/strlen, complete maximum emission and real compressed/caller/runtime behavior remain open. [TEXT-003](research/MESSAGE_STRINGS.md).

docs/PROVISIONAL_TODO.md

@@ -542,3 +542,5 @@ Research progress: [message byte codec](research/MESSAGE_BYTE_CODEC.md) complete
 Research progress: [message width codecs](research/MESSAGE_WIDTHS.md) completes bounded RE-369–371. Broad network/runtime milestones remain open; RE-372 tracks wrappers/delta/snapshot codecs, message lifecycle and actual packet/platform validation.
 
 Research progress: [message wrappers](research/MESSAGE_WRAPPERS.md) completes bounded RE-373–375. RE-376 leaves ReadData/compressed/string/delta/snapshot compositions, sentinel/lifecycle/alias and actual packet/platform validation open.
+
+Research progress: [message strings and data reads](research/MESSAGE_STRINGS.md) completes bounded RE-377–379. RE-380 leaves compressed/library/caller-lifetime/alias/lifecycle and higher-level packet/platform contracts open.

docs/RESEARCH_LEDGER.md

@@ -237,3 +237,7 @@ Completed bounded RE-369–371 with `analyze_message_widths.py`: twenty verified
 ## 2026-10-11 — Scalar message wrappers and data overflow
 
 Completed bounded RE-373–375 with `analyze_message_wrappers.py`: 22 verified graphs across the Linux ELF64 engines, 64 scalar OOB compositions, 1,280 supplied reader-tail probes and 300 data OOB compositions. WRAP-001–003/F-139–141 distinguish fixed positive wrapper widths from result signedness, post-read signed cursor/size comparisons from pre-read safety, and valid signed -1 words from EOF sentinels. Scalar compositions evaluate actual OOB writer/reader instructions against ample or guarded capacities; separate reader-tail probes supply return/count effects explicitly. WriteData composes each child OOB byte write, continuing all requests after space failure, retaining sticky flags even when ample writes proceed and counting every request in oldsize. Source/buffer bytes and canaries verify exact prefixes and unchanged tails; disjoint supplied wrapper/child frames and explicit caller-register retention preserve loop state. PUSH/POP and save/restore frames remain excluded/stopped; no original process or native callee executes. Eight independent tests verify forwarding/diagnostics, OOB bytes/cursors/post-read EOF, signed conversion/count comparisons, every data request, stored prefixes/cursors, overflow continuation/flag retention/canaries, paired outcomes and restricted AX/MOVSBL/MOVSWL evaluator support. All 358 tests pass; new and preceding reports regenerate byte-identically and storage/link/whitespace checks pass. RE-372 remains open; RE-376 tracks ReadData/compressed/string/float/angle/delta/entity/player/snapshot compositions, caller sentinel/alias/capacity/allowoverflow/reset/truncation/EOF/padding/legal cursor/length invariants, native seeded codewords and actual packet/runtime/all-platform/renderer contracts.
+
+## 2026-10-11 — OOB data reads and message strings
+
+Completed bounded RE-377–379 with `analyze_message_strings.py`: sixteen verified graphs across the Linux ELF64 engines, 106 actual OOB data-read compositions, ninety string-read compositions, 120 string-write compositions and 24 supplied writer length-guard probes. TEXT-001–003/F-142–144 distinguish continued EOF data requests and 0xff stores from read safety, consumed NUL/newline/EOF termination from capacity termination, static output addresses from caller lifetime, and reader percent/high-byte replacement from writer high-byte-only replacement. Complete OOB capacity fixtures recover 1023/1023/8191-byte ordinary/line/big outputs, terminators, unread following bytes and canaries. Writer strlen returns and Q_strncpyz copy/NUL effects are supplied explicitly; native sanitation and byte requests compose OOB child stores, including overflow continuation. File-backed empty-byte offsets are retained; over-limit and signed low-word guard probes do not claim legal huge strings or library conformance. Save/restore/PUSH/POP omissions, disjoint virtual frames and explicit caller-register retention are documented; no original process or native callee executes. Eight independent tests verify data requests/cursors/EOF prefixes and tails, string termination/sanitation/capacity/pointers/control bytes, writer copy/NUL/capacity effects, signed length/empty-byte boundaries and paired outcomes. All 366 tests pass; new and preceding reports regenerate byte-identically, with storage/link/whitespace checks passing. RE-376 remains open; RE-380 tracks compressed/library/caller lifetime/alias/nonterminated/overflow/reset/truncation/cursor contracts, float/angle/delta/entity/player/snapshot codecs, native seeded codewords and actual packet/runtime/all-platform/renderer validation.

docs/REVERSE_ENGINEERING_TODO.md

@@ -452,3 +452,8 @@ Purpose: turn the supplied installation into an evidence-backed specification of
 - [x] RE-374 Recover scalar signed/unsigned conversion and signed post-read EOF comparisons across supplied return/count/size boundaries ([WRAP-002](research/MESSAGE_WRAPPERS.md)).
 - [x] RE-375 Compose native data-writer loops with OOB child writes, including exact stored prefixes, continued requests and sticky overflow/accounting ([WRAP-003](research/MESSAGE_WRAPPERS.md)).
 - [ ] RE-376 Compose ReadData/compressed scalar/data/string/float/angle/delta/entity/player/snapshot codecs; recover caller sentinel/alias/capacity/allowoverflow/reset/truncation/EOF/padding/legal cursor/length invariants, native seeded codewords and actual packet/runtime/all-platform/renderer contracts.
+
+- [x] RE-377 Compose bounded OOB ReadData with actual reader cursor effects, EOF stores and continued requests ([TEXT-001](research/MESSAGE_STRINGS.md)).
+- [x] RE-378 Recover bounded ordinary/big/line string termination, percent/high-byte sanitation, static output and full capacity fixtures ([TEXT-002](research/MESSAGE_STRINGS.md)).
+- [x] RE-379 Recover writer length guards/null/over-limit paths and compose small supplied-copy sanitation/NUL OOB emission ([TEXT-003](research/MESSAGE_STRINGS.md)).
+- [ ] RE-380 Compose compressed scalar/data/string codecs and real library-copy/length behavior; recover string caller lifetime/limits/alias/nonterminated/overflow/reset/truncation/cursor invariants, float/angle/delta/entity/player/snapshot codecs, native seeded codewords and actual packet/runtime/all-platform/renderer contracts.

docs/research/MESSAGE_STRINGS.md

@@ -0,0 +1,43 @@
+# OOB data reads and message strings
+
+Reproduce with `python3 scripts/analyze_message_strings.py`; validate with `python3 scripts/test_message_strings.py`. [message-strings.json](message-strings.json) retains sixteen source/disassembly-hash-verified graphs across both Linux ELF64 engines, 106 data-read compositions, ninety string-read compositions, 120 string-write compositions and 24 writer length-guard probes. Synthetic source/result buffers use hexadecimal strings to keep this ordinary Git report below 2 MiB.
+
+Original processes and native callees never execute. OOB ReadBits/WriteBits instructions are composed at explicit stopped CALL boundaries, with caller registers retained and child memory effects applied. Virtual backing storage is padded and distinct from cursize. String strlen returns and Q_strncpyz copy/NUL effects are supplied; those library routines and logging callees remain unexecuted. PUSH/POP and save/restore instructions are omitted/stopped explicitly, including the line reader's restores before its final terminator store. Compressed strings/data, actual packets, aliases, full lifecycle/runtime/all-platform and ABI contracts remain open.
+
+## TEXT-001 — ReadData with actual OOB reader effects (RE-377)
+
+Each image has 53 ReadData compositions across lengths -1/0/1–9/16/17/31/32 and distinct size values from zero, one, length minus one and length. This follows the earlier [supplied-result ReadData probes](CLIENT_MASKS.md), now composing actual OOB ReadBits(width eight) instructions for each request. Source bytes follow `(129 + 37*i) &255`; the destination starts as forty 0xa5 bytes with outside canaries.
+
+Signed nonpositive lengths issue no reads or stores. Positive lengths issue exactly that many eight-bit requests, regardless of EOF. After each read, signed readcount is compared with cursize: accepted bytes are stored as low eight bits; out-of-message results become 0xff. For these nonnegative sizes, the destination becomes the valid source prefix, followed by 0xff through the requested length, followed by unchanged 0xa5 tail bytes. Requests continue after the first failed size check.
+
+Readcount advances once per request; the OOB bit cursor starts at seven and advances by eight per request, including EOF reads. The underlying reader itself accesses supplied backing bytes beyond cursize before the ReadData size check. This is local behavior over padded memory, not evidence that original truncated buffers are safe or that EOF recovery is complete. Legal wrapping/signed cursors, destination lengths, aliasing and compressed input remain open.
+
+## TEXT-002 — String read termination, sanitation and capacity (RE-378)
+
+MSG_ReadString and MSG_ReadStringLine use distinct recovered static 1024-byte output buffers. MSG_ReadBigString uses a recovered static 8192-byte buffer. All return the corresponding buffer address; each writes a NUL terminator after the result and retains the unused 0xa5 tail in the supplied fixtures. This recovers local buffer addresses/bytes, not complete ownership, reentrancy or threading behavior.
+
+The reader consumes eight-bit requests until a post-read size failure, a NUL byte, or the output's capacity minus one. ReadStringLine additionally stops at newline 0x0a; carriage return 0x0d is retained. NUL/newline/EOF termination consumes the request that triggered it. Capacity termination stops after filling the permitted output bytes, leaving the following source byte unread. High source byte 0xff is a valid byte when the post-read count is within size; it is not by itself an EOF sentinel.
+
+For accepted nonterminating bytes, all three readers replace percent 0x25 and every value above 127 with period 0x2e. Other sampled ASCII/control bytes retain their values. No multibyte text decoding is composed.
+
+Each reader is tested against empty/NUL input, ordinary ASCII, percent/high bytes/newline, carriage-return/newline, and all nonzero byte values ascending, with zero/truncated/full declared sizes. Complete capacity fixtures supply more than enough 'A' bytes: ordinary/line readers store 1023 bytes and terminate locally; BigString stores 8191. The following byte and final source NUL remain unread in those capacity cases. Independent oracles verify every result byte, consumed request count, actual OOB readcount/bit cursor, return pointer, terminator, retained tail and canaries.
+
+These are OOB and explicitly padded fixtures. Static-buffer reuse/caller lifetime, compressed cursor-driven EOF, malformed/wrapping sizes, real packet limits and actual newline parsing callers remain open.
+
+## TEXT-003 — String writer lengths, supplied copy and byte requests (RE-379)
+
+The writer first handles a null source pointer as one eight-bit request for a file-backed zero byte. MSG_WriteString otherwise requests strlen, compares its low word as signed against 1023, and requests Q_strncpyz with destination the supplied stack buffer, source pointer and capacity 1024. WriteBigString uses threshold 8191 and capacity 8192. Actual file offsets and zero-byte values are retained in the report.
+
+Lengths zero/limit minus one request copying. Lengths at/above the limit request Com_Printf, then the empty-byte path. The logger remains unexecuted. Word probes 0xffffffff and 0x80000000 satisfy the signed comparison and request copying; they are deliberately supplied library-return boundaries, not evidence of legal string sizes or actual huge allocations. Only copy request arguments are evaluated for these boundaries.
+
+Small valid fixtures supply strlen and copy/NUL effects explicitly, then evaluate the native stack-byte sanitation and emission instructions. Writer sanitation replaces bytes above 127 with period; percent is retained. The source remains unchanged because sanitation acts on the supplied copied stack bytes. Every sanitized byte plus the final NUL requests MSG_WriteBits(message, byte, eight). The requests compose actual OOB stores, including the underlying space guard. As with WriteData, emission continues after child overflow, so the stored prefix can omit the final NUL when capacity is insufficient.
+
+Each image has sixty compositions across the two writers: null/empty/percent-and-high-byte inputs, alternating ASCII/high-byte lengths 1–9/17/31, capacities four/64, and explicit over-limit supplied strlen cases that evaluate logging-request plus empty-byte emission. For the latter, backing source bytes are deliberately empty while the supplied strlen word is limit/limit plus one; these are control-flow probes, not native strlen conformance.
+
+The normal reader and writer differ: the writer retains percent, while the reader replaces it. A percent-containing string therefore need not round-trip byte-for-byte through these recovered paths. Complete maximum-length emission, real Q_strncpyz/strlen implementations, source/destination aliasing, nonterminated source memory, compressed strings, overflow recovery and game caller text conventions remain open.
+
+## Validation and next work
+
+Eight independent tests verify actual data-read requests/cursors and EOF prefixes; continuation/unchanged tails/canaries; string termination/sanitation; full capacity/return-pointer behavior; percent/high-byte/CR/newline cases; writer copied-byte/NUL/capacity effects; signed length guards/file-backed empty bytes; and paired engine outcomes. No evaluator extension was needed.
+
+RE-380: compose compressed scalar/data/string codecs and real library-copy/length behavior; recover string caller buffer lifetime/limits, alias/nonterminated/overflow/allowoverflow/reset/truncation/cursor invariants, float/angle/delta/entity/player/snapshot codecs, native full-seed codewords and actual packet/runtime/all-platform/renderer contracts.

docs/research/MESSAGE_WRAPPERS.md

@@ -48,3 +48,5 @@ These results do not determine allowoverflow recovery, actual maxsize/cursize le
 Eight tests verify scalar width/argument forwarding and diagnostics, OOB payloads/cursors and post-read EOF, signed size/count conversion, every data request/source byte, exact stored prefixes/tails/cursors, sticky flags and continuation/canaries, paired results and restricted signed-load evaluator behavior. All local sampled paths are bounded; trace hashes preserve the composed instruction sequences.
 
 RE-376: compose ReadData, compressed scalar/data wrappers and strings/float/angle/delta/entity/player/snapshot codecs; recover caller sentinel handling, source/destination aliasing and capacity/allowoverflow/reset/truncation/EOF/padding/legal cursor and length invariants. Native full-training/codeword validation, real packets, original-runtime/all-platform and renderer contracts remain open.
+
+Follow-up: [message strings and data reads](MESSAGE_STRINGS.md) composes OOB ReadData and bounded ordinary/big/line strings. Broader RE-376 remains open; RE-380 tracks compressed/library/caller and complete packet/platform contracts.

docs/research/README.md

@@ -143,3 +143,5 @@ The extension collector adds five focused tests for native tail calls, command-r
 [Message initialization, OOB and mixed-width codecs](MESSAGE_WIDTHS.md) adds RE-369–371: warm/cold record effects, exact clear retention and native writer/reader compositions for OOB and raw-remainder/Huffman paths. Reproduce with `python3 scripts/analyze_message_widths.py`; check with `python3 scripts/test_message_widths.py`. Wrappers, lifecycle, real packets and runtime/platform validation remain open.
 
 [Scalar message wrappers and data overflow](MESSAGE_WRAPPERS.md) adds RE-373–375: fixed-width forwarding, scalar conversion/post-read EOF, and data-writer continuation after child space failure. Reproduce with `python3 scripts/analyze_message_wrappers.py`; check with `python3 scripts/test_message_wrappers.py`. ReadData, compressed/higher-level codecs and actual packet/runtime/platform coverage remain open.
+
+[OOB data reads and message strings](MESSAGE_STRINGS.md) adds RE-377–379: actual ReadData cursor/EOF composition, ordinary/big/line string read limits/sanitation, and bounded supplied-copy string writing. Reproduce with `python3 scripts/analyze_message_strings.py`; check with `python3 scripts/test_message_strings.py`. Compressed/library/caller/higher-level and real packet/runtime/platform contracts remain open.

scripts/analyze_message_strings.py

@@ -0,0 +1,129 @@
+#!/usr/bin/env python3
+"""Bounded OOB ReadData and string codec instruction compositions."""
+import copy
+import gzip
+import itertools
+import json
+from pathlib import Path
+import re
+from analyze_area_queries import site
+from analyze_bsp_loading import read_address
+from analyze_bsp_references import global_address, trace_fields
+from analyze_collision_composition import get_word, put_word
+from analyze_engine_control import control_flow
+from analyze_message_byte_codec import MSG, DATA
+from analyze_message_widths import reader, run, writer
+from analyze_message_wrappers import message_memory, SOURCE, WRAPPER_STACK
+from analyze_native import elf
+from analyze_qvm import write_json
+from analyze_vm_extensions import native_functions, verified
+
+DEST=0xe0000
+
+
+def string_graph(g):
+    g=copy.deepcopy(g);omitted=[]
+    for x in g['instructions']:
+        # Explicit frame-save/restore omission, including the line reader's
+        # tail restores which precede its final terminator store.
+        if x['opcode'] in ['pushq','popq'] or (x['opcode']=='movq' and '(%rsp)' in x['operands']):
+            omitted.append(x['address']);x['opcode']='nop';x['operands']=''
+    return g,omitted
+
+
+def read_string(graphs,name,source,size):
+    original=graphs[name];g,omitted=string_graph(original);rows=g['instructions'];calls={x['address']:i for i,x in enumerate(rows) if x['opcode']=='callq'};end=next(x['address'] for x in rows if x['opcode']=='retq')
+    zero=next(x for x in original['instructions'] if x['opcode']=='movb' and x['operands'].startswith('$0x0,'));base=int(re.search(r', (0x[0-9a-f]+)',zero['operands'])[1],16);capacity=8192 if name=='MSG_ReadBigString' else 1024
+    m=message_memory(16384)
+    for i,v in enumerate(source):m[DATA+i]=v
+    m[DATA+len(source)]=0
+    for i in range(capacity):m[base+i]=0xa5
+    m[base-1]=m[base+capacity]=0x7b;put_word(m,MSG+28,size)
+    regs={'%rdi':MSG,'%rbx':MSG,'%rsp':WRAPPER_STACK};pc=rows[0]['address'];trace=[];count=0
+    while True:
+        t=run(g,pc,list(calls)+[end],m,regs);m=t['memory'];regs=t['registers'];trace+=t['trace']
+        if t['stop']==end:break
+        i=calls[t['stop']]
+        if '<MSG_ReadBits>' not in rows[i]['operands'] or [regs['rdi'],regs['rsi']]!=[MSG,8]:raise ValueError('unexpected string read request')
+        m,value,error,child=reader(graphs,m,8);trace+=child
+        if error:raise ValueError('unexpected OOB read error')
+        regs['rax']=value;count+=1;pc=rows[i+1]['address']
+        if count>capacity:raise ValueError('string fixture request bound')
+    result=bytes(m[base+i] for i in range(capacity));length=result.index(0)
+    return {'function':name,'source_hex':bytes(source).hex(),'size':size,'capacity':capacity,'read_requests':count,'read_count':get_word(m,MSG+32),'bit':get_word(m,MSG+36),'result_hex':result[:length+1].hex(),'tail_unchanged':all(v==0xa5 for v in result[length+1:]),'returned_pointer':regs['rax'],'buffer_address':base,'canaries':[m[base-1],m[base+capacity]],'omitted_frame_instructions':omitted,**trace_fields({'trace':trace},False)}
+
+
+def write_string(graphs,name,source,capacity,constant,strlen_override=None):
+    g=graphs[name];rows=g['instructions'];start=site(g,'testq','%rsi, %rsi');end=next(x['address'] for x in rows if x['opcode']=='movq' and re.match(r'0x(?:400|2000)\(%rsp\), %rbx',x['operands']));calls={x['address']:i for i,x in enumerate(rows) if x['opcode']=='callq'}
+    m=message_memory(capacity);put_word(m,constant[0],constant[1],1)
+    if source is not None:
+        for i,v in enumerate(source+[0]):m[SOURCE+i]=v
+    regs={'%rsi':SOURCE if source is not None else 0,'%rbx':SOURCE if source is not None else 0,'%r12':MSG,'%rsp':WRAPPER_STACK};pc=start;trace=[];values=[];copy_requests=[];prints=0
+    for x in graphs['MSG_WriteBits']['instructions']:
+        if x['opcode']=='addl' and '(%rip)' in x['operands']:put_word(m,global_address(x),0)
+    while True:
+        t=run(g,pc,list(calls)+[end],m,regs);m=t['memory'];regs=t['registers'];trace+=t['trace']
+        if t['stop']==end:break
+        i=calls[t['stop']];target=rows[i]['operands']
+        if '<strlen@plt>' in target:
+            if regs['rdi']!=SOURCE:raise ValueError('strlen pointer')
+            regs['rax']=len(source) if strlen_override is None else strlen_override  # Explicit supplied library return.
+        elif '<Q_strncpyz>' in target:
+            args=[regs[k] for k in ['rdi','rsi','rdx']];copy_requests.append(args)
+            if args[:2]!=[WRAPPER_STACK,SOURCE] or len(source)>=args[2]:raise ValueError('copy fixture')
+            for j,v in enumerate(source+[0]):m[WRAPPER_STACK+j]=v  # Supplied copy/NUL effects.
+        elif '<Com_Printf>' in target:prints+=1  # Logging callee remains unexecuted.
+        elif '<MSG_WriteBits>' in target:
+            if [regs['rdi'],regs['rdx']]!=[MSG,8]:raise ValueError('string writer request')
+            values.append(regs['rsi']);m,requests,error,child=writer(graphs,m,8,regs['rsi']);trace+=child
+            if requests or error:raise ValueError('unexpected OOB write')
+        else:raise ValueError('unexpected string helper '+target)
+        pc=rows[i+1]['address']
+        if len(values)>256:raise ValueError('string writer fixture bound')
+    size=get_word(m,MSG+28)
+    return {'function':name,'source_hex':None if source is None else bytes(source).hex(),'capacity':capacity,'supplied_strlen_word':None if source is None else len(source) if strlen_override is None else strlen_override,'requested_hex':bytes(values).hex(),'copy_requests':copy_requests,'printf_requests':prints,'payload_hex':bytes(m[DATA+i] for i in range(size)).hex(),'size':size,'bit':get_word(m,MSG+36),'overflow_flag':get_word(m,MSG+4),'source_unchanged':source is None or all(m[SOURCE+i]==v for i,v in enumerate(source+[0])),'canaries':[m[DATA-1],m[DATA+64]],**trace_fields({'trace':trace},False)}
+
+
+def main():
+    out={'native':{},'limitations':'Synthetic Linux ELF64 OOB instruction compositions; original processes and native callees never execute. ReadData and strings compose ReadBits/WriteBits with explicit caller-register retention and virtual frames. String strlen return and Q_strncpyz copy/NUL effects are supplied, not native library execution. Frame saves/restores/PUSH/POP excluded/stopped. Padded backing storage is distinct from cursize. No compressed strings/data, actual packets, alias/lifecycle/capacity-wrap/ABI/runtime/all-platform conformance.'}
+    for label in ['linux--Quake3-UrT-Ded.x86_64','linux--Quake3-UrT.x86_64']:
+        r=json.loads(Path(f'docs/research/native/{label}.json').read_text());data=verified((Path('.cache/UrbanTerror43')/r['source']['path']).read_bytes(),r['source']['sha256']);meta,_=elf(data);a=r['artifacts']['disassembly.txt'];fs=native_functions(verified(gzip.open(a['path'],'rb').read(),a['uncompressed_sha256']).decode());names=['MSG_ReadData','MSG_ReadString','MSG_ReadBigString','MSG_ReadStringLine','MSG_WriteString','MSG_WriteBigString','MSG_ReadBits','MSG_WriteBits'];graphs={n:control_flow(fs[n]) for n in names}
+        data_probes=[];g=graphs['MSG_ReadData'];rows=g['instructions'];start=site(g,'testl','%edx, %edx');end=site(g,'addq','$0x8, %rsp');calls={x['address']:i for i,x in enumerate(rows) if x['opcode']=='callq'}
+        for length in [-1,0,1,2,3,4,5,6,7,8,9,16,17,31,32]:
+            for size in sorted({0,1,max(0,length-1),max(0,length)}):
+                source=[(129+37*i)&255 for i in range(40)];m=message_memory(64)
+                for i,v in enumerate(source):m[DATA+i]=v
+                for i in range(40):m[DEST+i]=0xa5
+                m[DEST-1]=m[DEST+40]=0x7b;put_word(m,MSG+28,size)
+                regs={'%r15':length&0xffffffff,'%r14':DEST,'%rbp':MSG,'%rdx':length&0xffffffff,'%rdi':MSG,'%rsp':WRAPPER_STACK};pc=start;trace=[];count=0
+                while True:
+                    t=run(g,pc,list(calls)+[end],m,regs);m=t['memory'];regs=t['registers'];trace+=t['trace']
+                    if t['stop']==end:break
+                    i=calls[t['stop']]
+                    if '<MSG_ReadBits>' not in rows[i]['operands'] or [regs['rdi'],regs['rsi']]!=[MSG,8]:raise ValueError('data read request')
+                    m,value,error,child=reader(graphs,m,8);trace+=child
+                    if error:raise ValueError('data OOB read')
+                    regs['rax']=value;count+=1;pc=rows[i+1]['address']
+                    if count>32:raise ValueError('data read fixture bound')
+                data_probes.append({'length':length,'size':size,'read_requests':count,'read_count':get_word(m,MSG+32),'bit':get_word(m,MSG+36),'source_hex':bytes(source).hex(),'destination_hex':bytes(m[DEST+i] for i in range(40)).hex(),'canaries':[m[DEST-1],m[DEST+40]],**trace_fields({'trace':trace},False)})
+        reads=[]
+        sequences=[[0],[65,66,67,0],[37,128,255,65,10,0],[13,10,65,0],list(range(1,256))+[0]]
+        for name,source in itertools.product(['MSG_ReadString','MSG_ReadBigString','MSG_ReadStringLine'],sequences):
+            for size in sorted({0,len(source)-1,len(source)}):reads.append(read_string(graphs,name,source,size))
+        for name in ['MSG_ReadString','MSG_ReadBigString','MSG_ReadStringLine']:
+            capacity=8192 if name=='MSG_ReadBigString' else 1024;source=[65]*capacity+[0];reads.append(read_string(graphs,name,source,len(source)))
+        writes=[];guards=[];constants={}
+        patterns=[None,[],[37,128,255,65,10]]+[[128 if i%2 else 65 for i in range(length)] for length in [1,2,3,4,5,6,7,8,9,17,31]]
+        for name in ['MSG_WriteString','MSG_WriteBigString']:
+            g=graphs[name];rows=g['instructions'];x=next(x for x in rows if x['opcode']=='movzbl' and '(%rip)' in x['operands']);address=global_address(x);value,offset=read_address(data,meta['sections'],address,1);constant=(address,value[0]);constants[name]={'address':address,'file_offset':offset,'byte':value[0]}
+            for source,capacity in itertools.product(patterns,[4,64]):writes.append(write_string(graphs,name,source,capacity,constant))
+            ci=next(i for i,x in enumerate(rows) if '<strlen@plt>' in x['operands']);stops=[x['address'] for x in rows if x['opcode']=='callq' and ('<Q_strncpyz>' in x['operands'] or '<Com_Printf>' in x['operands'])];limit=8192 if name=='MSG_WriteBigString' else 1024
+            for length in [limit,limit+1]:writes.append(write_string(graphs,name,[],64,constant,strlen_override=length))
+            for length in [0,limit-1,limit,limit+1,0xffffffff,0x80000000]:
+                t=run(g,rows[ci+1]['address'],stops,{}, {'%rax':length,'%rbx':SOURCE,'%rsp':WRAPPER_STACK});target=next(x['operands'] for x in rows if x['address']==t['stop']);guards.append({'function':name,'supplied_strlen_word':length,'outcome':'copy_request' if '<Q_strncpyz>' in target else 'printf_request','copy_arguments':[t['registers'][k] for k in ['rdi','rsi','rdx']] if '<Q_strncpyz>' in target else None,**trace_fields(t,False)})
+        out['native'][label]={'source':r['source'],'disassembly_sha256':a['uncompressed_sha256'],'graphs':graphs,'data_reads':data_probes,'string_reads':reads,'string_writes':writes,'writer_length_guards':guards,'writer_empty_constants':constants}
+    write_json(Path('docs/research/message-strings.json'),out)
+    print('Recovered OOB data reads and string read/write boundary compositions')
+
+
+if __name__=='__main__':main()

scripts/test_message_strings.py

@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+"""Independent OOB read, string sanitation and length boundary oracles."""
+import json
+from pathlib import Path
+import unittest
+
+
+def read_oracle(p):
+    source=bytes.fromhex(p['source_hex']);output=[];count=0
+    for value in source:
+        count+=1
+        if count>p['size'] or value==0 or (p['function']=='MSG_ReadStringLine' and value==10):break
+        output.append(46 if value==37 or value>127 else value)
+        if len(output)==p['capacity']-1:break
+    return bytes(output+[0]),count
+
+
+class MessageStringTests(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):cls.r=json.loads(Path('docs/research/message-strings.json').read_text())
+
+    def test_data_actual_oob_read_counts_and_eof_prefix(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['data_reads']),53)
+            for p in n['data_reads']:
+                count=max(0,p['length']);source=bytes((129+37*i)&255 for i in range(40));valid=min(count,p['size'])
+                self.assertEqual(bytes.fromhex(p['source_hex']),source);self.assertEqual(p['read_requests'],count);self.assertEqual(p['read_count'],count);self.assertEqual(p['bit'],7+8*count)
+                expected=source[:valid]+bytes([255])*(count-valid)+bytes([0xa5])*(40-count)
+                self.assertEqual(bytes.fromhex(p['destination_hex']),expected)
+
+    def test_data_continues_after_eof_and_retains_tail_canaries(self):
+        for n in self.r['native'].values():
+            for p in n['data_reads']:
+                self.assertEqual(p['canaries'],[0x7b]*2)
+                if p['size']==0 and p['length']>0:self.assertEqual(bytes.fromhex(p['destination_hex'])[:p['length']],bytes([255])*p['length'])
+                if p['length']<=0:self.assertEqual(p['read_requests'],0);self.assertEqual(bytes.fromhex(p['destination_hex']),bytes([0xa5])*40)
+
+    def test_string_independent_termination_and_sanitization(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['string_reads']),45)
+            for p in n['string_reads']:
+                result,count=read_oracle(p)
+                self.assertEqual(bytes.fromhex(p['result_hex']),result);self.assertEqual(p['read_requests'],count);self.assertEqual(p['read_count'],count);self.assertEqual(p['bit'],7+8*count)
+
+    def test_string_limits_static_pointer_and_canaries(self):
+        for n in self.r['native'].values():
+            found=0
+            for p in n['string_reads']:
+                self.assertEqual(p['returned_pointer'],p['buffer_address']);self.assertEqual(p['canaries'],[0x7b]*2);self.assertTrue(p['tail_unchanged'])
+                if len(bytes.fromhex(p['source_hex']))>p['capacity']:
+                    found+=1;self.assertEqual(p['read_requests'],p['capacity']-1);self.assertEqual(len(bytes.fromhex(p['result_hex'])),p['capacity']);self.assertEqual(p['size'],p['capacity']+1)
+            self.assertEqual(found,3)
+
+    def test_percent_high_bytes_carriage_return_and_newline(self):
+        for n in self.r['native'].values():
+            for p in n['string_reads']:
+                source=bytes.fromhex(p['source_hex'])
+                if p['size']!=len(source):continue
+                if source==bytes([37,128,255,65,10,0]):self.assertEqual(bytes.fromhex(p['result_hex']),b'...A\0' if p['function'].endswith('Line') else b'...A\n\0')
+                if source==bytes([13,10,65,0]):self.assertEqual(bytes.fromhex(p['result_hex']),b'\r\0' if p['function'].endswith('Line') else b'\r\nA\0')
+
+    def test_writer_supplied_copy_sanitation_nul_and_capacity(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['string_writes']),60)
+            for p in n['string_writes']:
+                limit=8192 if p['function']=='MSG_WriteBigString' else 1024;source=None if p['source_hex'] is None else bytes.fromhex(p['source_hex']);oversize=p['supplied_strlen_word'] is not None and p['supplied_strlen_word']>=limit
+                expected=b'\0' if source is None or oversize else bytes(46 if v>127 else v for v in source)+b'\0'
+                self.assertEqual(bytes.fromhex(p['requested_hex']),expected);self.assertEqual(p['printf_requests'],int(oversize));self.assertTrue(p['source_unchanged']);self.assertEqual(p['canaries'],[0x7b]*2)
+                accepted=min(len(expected),max(0,p['capacity']-3));self.assertEqual(bytes.fromhex(p['payload_hex']),expected[:accepted]);self.assertEqual(p['size'],accepted);self.assertEqual(p['bit'],7+8*accepted);self.assertEqual(p['overflow_flag'],int(accepted<len(expected)))
+                self.assertEqual(p['copy_requests'],[] if source is None or oversize else [[0xc0000,0xd0000,limit]])
+
+    def test_writer_signed_length_guard_and_file_backed_empty_byte(self):
+        for n in self.r['native'].values():
+            for p in n['writer_length_guards']:
+                limit=8192 if p['function']=='MSG_WriteBigString' else 1024;word=p['supplied_strlen_word'];value=word if word<2**31 else word-2**32;accepted=value<=limit-1
+                self.assertEqual(p['outcome'],'copy_request' if accepted else 'printf_request');self.assertEqual(p['copy_arguments'],[0xc0000,0xd0000,limit] if accepted else None)
+            for value in n['writer_empty_constants'].values():self.assertEqual(value['byte'],0);self.assertGreaterEqual(value['file_offset'],0)
+
+    def test_paired_results(self):
+        a,b=self.r['native'].values()
+        for key in ['data_reads','string_reads','string_writes','writer_length_guards']:
+            def clean(rows):return [{k:v for k,v in p.items() if k not in ['trace','trace_sha256','returned_pointer','buffer_address','omitted_frame_instructions']} for p in rows]
+            self.assertEqual(clean(a[key]),clean(b[key]))
+
+
+if __name__=='__main__':unittest.main()