Recover leaf metadata loops and query output contracts

docs/ASSET_PIPELINE_TODO.md

@@ -10,6 +10,8 @@ The [world-sector/link collector](research/WORLD_LINKS.md) adds topology, placem
 
 The [entity-bound collector](research/LINK_BOUNDS.md) adds rounded radius/expansion and BSP request fixtures for future collision sidecars; actual leaf/cluster outputs remain open. Reproduce with `python3 scripts/analyze_link_bounds.py`.
 
+The [leaf-metadata collector](research/LEAF_METADATA.md) adds native stride/area/cluster and count/overflow/last-leaf evidence for future BSP collision sidecars; actual traversal and native/on-disk correspondence remain open. Reproduce with `python3 scripts/analyze_leaf_metadata.py`.
+
 ## Planned tool catalogue
 
 Names and output formats below are proposals to refine after the relevant research gates. They may become Python scripts, Rust CLI subcommands, or both. Runtime-critical parsers and shared schemas should have one authoritative implementation; orchestration scripts must not diverge from it.

docs/FINDINGS.md

@@ -324,3 +324,9 @@ Bounded qagame paths recover bomb-plant/inactivity correction requests, direct f
 **F-092 — Observed leaf-query gates:** metadata resets precede a capacity-128 bound query request. Supplied zero count exits, positive counts request first leaf area, and negative count requests fallback cluster from the extra output. Actual BSP returns and full area/cluster behavior remain unverified. [BOUNDS-002](research/LINK_BOUNDS.md).
 
 **F-093 — Observed inline removal precondition:** owned-record fixtures agree with standalone unlink on repairs/clears/stale next/warnings. A NULL-owner inline precheck skips the linked-word clear that standalone unlink performs. Membership/reset/ABI consistency and full relink/runtime behavior remain open. [BOUNDS-003](research/LINK_BOUNDS.md).
+
+**F-094 — Observed area accumulation:** -1 is skipped; the first area persists while later distinct areas can replace the second. A third distinct area requests a diagnostic only for raw sv state word one. All area lanes are exercised with supplied returns; actual map ordering/state/consumers remain open. [META-001](research/LEAF_METADATA.md).
+
+**F-095 — Observed cluster capacity:** qualifying IDs append without deduplication; new count exactly sixteen requests the retained-leaf fallback and preserves all sixteen entries. Area visitation completes first, and remaining cluster calls stop. Real geometry, output ownership and runtime behavior remain unverified. [META-002](research/LEAF_METADATA.md).
+
+**F-096 — Observed leaf/query outputs:** guarded native leaf access uses stride 24 with cluster +0 and area +4. The store callback updates the latest leaf with cluster != -1 before capacity checking; the wrapper returns stored count and publishes that leaf separately, rather than returning overflow as a negative count. Traversal/layout/loading/runtime invariants remain open. [META-003](research/LEAF_METADATA.md).

docs/PROVISIONAL_TODO.md

@@ -33,7 +33,7 @@ For implementation items, the minimum done condition is a documented contract, a
 
 ### First research batch
 
-Progress: [RE-232–234 / TIME-001–003](research/FRAME_TIMING.md) now cover Linux ELF64 elapsed modification, event sources and pause predicates. The [RE-237–239 follow-up](research/EVENT_QUEUES.md) resolves the pause-field address/bot chain and adds queue/journal evidence. The [RE-241–243 announcement pass](research/AUTH_ANNOUNCEMENTS.md) recovers label tables and recipient paths; The [RE-245–247 cvar pass](research/GAME_CVARS.md) adds all 126 main registration rows and resolves auth_verbosity/callers. The [RE-249–251 change pass](research/CVAR_CHANGES.md) adds cache/notification behavior, four numeric correction branches and direct registration coverage. The [RE-253–255 query pass](research/AUTH_QUERIES.md) adds selector helpers, command-role routing and internal authentication modes. The [RE-257–259 bounds/entry pass](research/SELECTOR_BOUNDS.md) adds integer parsing, selector storage/count evidence and native/QVM/chat route connections. The [RE-261–263 chat pass](research/CHAT_PATHS.md) adds native command metadata/filter evidence and bounded message/recipient paths, including the documented argument-mapping correction. The [RE-265–267 delivery pass](research/CHAT_DELIVERY.md) adds bounded dead-chat predicates, formatter argument routing and echo/fanout calls. The [RE-269–271 expansion pass](research/CHAT_EXPANSION.md) adds marker states, twelve key comparisons and health/momo branches. The [RE-273–275 item/time pass](research/CHAT_ITEMS_TIME.md) adds packed ammunition fields, sampled names and countdown branches. The [RE-277–279 location pass](research/CHAT_LOCATIONS.md) adds leader selection, cache/format boundaries and location-list/spatial evidence. The [RE-281–283 crosshair pass](research/CHAT_CROSSHAIR.md) adds finite geometry fixtures, trace arguments and result routing. The [RE-285–287 native spatial pass](research/NATIVE_SPATIAL.md) confirms Linux trig/spatial services, VM pointer branches and collision/query evidence. The [RE-289–291 collision pass](research/NATIVE_COLLISION.md) adds native candidate filters, result decisions and model/field evidence. The [RE-293–295 composition pass](research/COLLISION_COMPOSITION.md) adds three-lane byte-copy/status agreement and world-result routing. The [RE-297–299 loop pass](research/COLLISION_LOOP.md) adds signed count/status gates, parity scheduling and related sentinel normalization. The [RE-301–303 area pass](research/AREA_QUERIES.md) adds registered address arithmetic, inclusive overlap and count/split ordering boundaries. The [RE-305–307 world-link pass](research/WORLD_LINKS.md) adds bounded sector construction, strict placement/head insertion and unlink state/repair behavior. The [RE-309–311 bounds pass](research/LINK_BOUNDS.md) adds radius/padding composition, leaf-return gates and inline removal evidence. PROV-002 remains open for full output/reset coverage. PROV-001 remains open for complete scheduling and platform coverage.
+Progress: [RE-232–234 / TIME-001–003](research/FRAME_TIMING.md) now cover Linux ELF64 elapsed modification, event sources and pause predicates. The [RE-237–239 follow-up](research/EVENT_QUEUES.md) resolves the pause-field address/bot chain and adds queue/journal evidence. The [RE-241–243 announcement pass](research/AUTH_ANNOUNCEMENTS.md) recovers label tables and recipient paths; The [RE-245–247 cvar pass](research/GAME_CVARS.md) adds all 126 main registration rows and resolves auth_verbosity/callers. The [RE-249–251 change pass](research/CVAR_CHANGES.md) adds cache/notification behavior, four numeric correction branches and direct registration coverage. The [RE-253–255 query pass](research/AUTH_QUERIES.md) adds selector helpers, command-role routing and internal authentication modes. The [RE-257–259 bounds/entry pass](research/SELECTOR_BOUNDS.md) adds integer parsing, selector storage/count evidence and native/QVM/chat route connections. The [RE-261–263 chat pass](research/CHAT_PATHS.md) adds native command metadata/filter evidence and bounded message/recipient paths, including the documented argument-mapping correction. The [RE-265–267 delivery pass](research/CHAT_DELIVERY.md) adds bounded dead-chat predicates, formatter argument routing and echo/fanout calls. The [RE-269–271 expansion pass](research/CHAT_EXPANSION.md) adds marker states, twelve key comparisons and health/momo branches. The [RE-273–275 item/time pass](research/CHAT_ITEMS_TIME.md) adds packed ammunition fields, sampled names and countdown branches. The [RE-277–279 location pass](research/CHAT_LOCATIONS.md) adds leader selection, cache/format boundaries and location-list/spatial evidence. The [RE-281–283 crosshair pass](research/CHAT_CROSSHAIR.md) adds finite geometry fixtures, trace arguments and result routing. The [RE-285–287 native spatial pass](research/NATIVE_SPATIAL.md) confirms Linux trig/spatial services, VM pointer branches and collision/query evidence. The [RE-289–291 collision pass](research/NATIVE_COLLISION.md) adds native candidate filters, result decisions and model/field evidence. The [RE-293–295 composition pass](research/COLLISION_COMPOSITION.md) adds three-lane byte-copy/status agreement and world-result routing. The [RE-297–299 loop pass](research/COLLISION_LOOP.md) adds signed count/status gates, parity scheduling and related sentinel normalization. The [RE-301–303 area pass](research/AREA_QUERIES.md) adds registered address arithmetic, inclusive overlap and count/split ordering boundaries. The [RE-305–307 world-link pass](research/WORLD_LINKS.md) adds bounded sector construction, strict placement/head insertion and unlink state/repair behavior. The [RE-309–311 bounds pass](research/LINK_BOUNDS.md) adds radius/padding composition, leaf-return gates and inline removal evidence. The [RE-313–315 metadata pass](research/LEAF_METADATA.md) adds area-slot/cluster loops and guarded leaf/query output contracts. PROV-002 remains open for full output/reset coverage. PROV-001 remains open for complete scheduling and platform coverage.
 
 Start with the open questions raised by the current binary evidence. These tasks depend on the linked reports, not on a working rewrite.
 

docs/RESEARCH_LEDGER.md

@@ -173,3 +173,7 @@ Completed bounded/static RE-305–307 with `analyze_world_links.py`: eight verif
 ## 2026-10-11 — Entity bounds, leaf-return gates and inline removal
 
 Completed bounded RE-309–311 with `analyze_link_bounds.py`: six verified graphs, file-backed margin/mask constants, 180 finite bounds/radius compositions, twelve supplied leaf-return slices and fourteen inline removals. BOUNDS-001–003/F-091–093 distinguish scalar radius composition from geometry execution, capacity requests from actual return bounds, and owned-removal agreement from the NULL-owner precheck difference. Eight tests check independent rounded bounds, model/angle gates, metadata/request/gate evidence, paired outcomes, removal state and scalar vector/float failure domains. The evaluator adds lower-lane ANDPS/MAXSS/SQRTSS/register MOVAPS and selected 32-bit logical/subtraction operations. The full suite passes 230 tests; new and prior collision/area/tree reports regenerate byte-identically, and storage/documentation link checks pass. RE-308 remains open; RE-312 tracks full metadata/BSP/packed fields, producers/lifetime/relink/float environment and whole runtime/platform behavior. No original process or recorded native callee executes.
+
+## 2026-10-11 — Leaf metadata loops and output contracts
+
+Completed bounded/static RE-313–315 with `analyze_leaf_metadata.py`: ten verified graphs, 208 complete area/cluster compositions, sixty guarded accessors and 72 callback probes. META-001–003/F-094–096 distinguish first/second replacement from a set, cluster duplicates from unique IDs, and stored count/overflow/last-leaf outputs. All observed unrolled service sites are exercised; diagnostic/callee returns are supplied and accessor PUSH/POP omission is explicit. Eight tests verify independent loop/guard/store/order oracles, paired results, signed comparison/sign/NOT/NOP semantics and fail-closed operations. The full suite passes 238 tests; new and prior reports regenerate byte-identically, and storage/documentation link checks pass. RE-312 remains open; RE-316 tracks full traversal/plane/layout/loading/output ownership, consumers and runtime/platform behavior. No original process or native callee executes.

docs/REVERSE_ENGINEERING_TODO.md

@@ -372,3 +372,8 @@ Purpose: turn the supplied installation into an evidence-backed specification of
 - [x] RE-310 Recover metadata reset writes, CM_BoxLeafnums request arguments and isolated zero/positive/negative return classification ([BOUNDS-002](research/LINK_BOUNDS.md)).
 - [x] RE-311 Recover inline owned-removal agreement and NULL-owner precheck difference against standalone unlink fixtures ([BOUNDS-003](research/LINK_BOUNDS.md)).
 - [ ] RE-312 Recover full area/cluster accumulation and overflow/fallback/packed-field behavior, actual BSP output bounds, object/angle/layout producers, membership/reset/relink/ABI invariants, exceptional float environment and whole-query/collision/original-runtime/all-platform fixtures.
+
+- [x] RE-313 Recover bounded complete area-loop compositions, first/second slot replacement and raw state-word diagnostic gates across all unrolled lanes ([META-001](research/LEAF_METADATA.md)).
+- [x] RE-314 Recover cluster-loop append/duplicate/skip behavior, exact sixteen-entry fallback and area-before-cluster call ordering ([META-002](research/LEAF_METADATA.md)).
+- [x] RE-315 Recover guarded 24-byte native leaf accesses and callback count/overflow/last-leaf output relationships ([META-003](research/LEAF_METADATA.md)).
+- [ ] RE-316 Recover full BSP traversal/plane predicates and leaf-array loading/ownership, actual query output bounds and diagnostics/state lifecycle, metadata/PVS/snapshot consumers, composed collision fixtures and original-runtime/all-platform conformance.

docs/research/LEAF_METADATA.md

@@ -0,0 +1,43 @@
+# Leaf areas, cluster accumulation and query output contracts
+
+Static/bounded pass, 2026-10-11. Reproduce with `python3 scripts/analyze_leaf_metadata.py`; check with `python3 scripts/test_leaf_metadata.py`. [Generated evidence](leaf-metadata.json) verifies both Linux ELF64 originals/disassemblies and retains five graphs per engine. It records **208 metadata compositions, sixty guarded accessor probes and 72 leaf-store probes**. This extends [bound/query-return evidence](LINK_BOUNDS.md).
+
+Every CM_LeafArea/CM_LeafCluster return is supplied, and diagnostic blocks are skipped with explicit post-call continuation. No original process, BSP traversal or native callee executes. Metadata slices stop before placement/cleanup. Accessor probes explicitly replace their RBX PUSH/POP with NOPs, retaining original instruction addresses; stack traffic and frame unwinding are excluded, and error calls/returns are stopped. Actual maps, ABI clobbers/lifetimes and runtime/platform conformance remain open.
+
+## META-001: two area slots and diagnostic gating
+
+The area loop visits all supplied leaves before starting cluster accumulation. It ignores returned area -1. A qualifying area writes server record +0x128 when that slot is -1 or already equal to the new area. Other areas write +0x12c. Thus the first qualifying area stays in the first slot, while a later distinct area can replace the second slot; this is not a set of every touched area.
+
+When the second slot is neither -1 nor the incoming area, the native global word at the base of `sv` is compared with **one**. Equality requests Com_DPrintf before the second slot is replaced; other sampled global values skip the diagnostic. The report preserves the original diagnostic text. That raw state-word predicate does not establish a complete named server-state contract, rendered output or permission policy.
+
+Each engine has 104 compositions: leaf counts 0–8, 15, 16, 17 and 128, four supplied return patterns (all absent, repeated IDs, mixed -1/repeated/distinct IDs, and unique IDs), and state word 0/1. Original unrolled instructions are followed between each stopped service/diagnostic boundary. All eight observed area-call lanes are exercised. Call requests retain leaf identity, supplied return and native address; traces through count three are retained in full, larger traces have deterministic hashes/instruction counts.
+
+Independent fixtures verify slot replacement, repeated first/second values, diagnostic counts, complete area visitation and paired engine outputs. Actual leaf enumeration order, area connectivity and metadata consumers remain open.
+
+## META-002: cluster append, duplicates and capacity fallback
+
+The cluster phase skips only return -1 in the tested sequences. Other IDs are appended at record +0xe4 +4*old_count, and count at +0xe0 increments. Duplicates remain: sixteen repeated sevens produce sixteen entries, not a single unique cluster.
+
+After storing/incrementing, **new count ==16** requests CM_LeafCluster for the extra output word from the bound query; its supplied return is stored at +0x124 before placement. The sixteen stored entries remain intact and count stays sixteen. This occurs even when the sixteenth qualifying entry is also the final leaf. Remaining cluster calls are skipped, while the preceding area phase has already visited every supplied leaf. Fewer qualifying clusters keep the initialized fallback word zero.
+
+All observed normal cluster-call lanes and the fallback call are exercised. Fixtures verify source visitation order, -1 filtering, duplicate preservation, sixteen-entry canaries/unwritten markers, exact fallback timing and all-area-before-cluster ordering. Here the extra word is supplied as leaf 1000 and fallback return as 777, not produced by actual geometry. The earlier signed-negative-count path remains a separate supplied-return slice; normal leaf-query count ownership is addressed below.
+
+## META-003: guarded leaf access and actual store callback
+
+CM_LeafArea and CM_LeafCluster each reject signed-negative index or index >= native leaf count by requesting Com_Error with code one and respectively `CM_LeafArea: bad number` / `CM_LeafCluster: bad number`. The probes stop before that error call; they do not infer recovery or nonreturn behavior.
+
+Valid accesses share the same native leaf-array pointer/count globals and use a **24-byte native leaf stride**. Cluster reads record +0, area reads +4. This is the shipped native representation, not automatically the BSP on-disk record stride. Fifteen cases per accessor/engine combine native counts 0/1/3 with indices -1/0/1/2/3 and distinct synthetic field values. PUSH/POP omission is recorded explicitly in the report; error arguments and indexed reads are still evaluated from original instructions.
+
+CM_BoxLeafnums installs **CM_StoreLeafs** as its leaf callback; the collector verifies that immediate function pointer against the decoded callback entry. CM_StoreLeafs complements the supplied negative child encoding to obtain leaf index. It reads that leaf's cluster through the same 24-byte representation, then:
+
+- If cluster !=-1, it writes leaf index to list record +48 **before** testing capacity.
+- If signed count < capacity, it appends the leaf index to output +4*count and increments count. Leaves with cluster -1 are still appended.
+- Otherwise it sets overflow word +8 to one, leaves count unchanged and performs no output write. The latest leaf with cluster != -1 can still update +48 when capacity is full.
+
+The callback reads its leaf directly, without using the guarded accessors; index provenance and tree bounds remain open. Thirty-six callback probes per engine cover three indices, cluster -1/7 and capacity boundaries 0/1/2/128. The wrapper initializes count/overflow/+48 to zero, and its retained return path reads count into EAX and writes +48 through the extra pointer. It does not return the overflow word as a negative count. Consequently the extra output is a retained leaf index, not a tree-node pointer. Under supplied valid initial count/capacity and these callback rules, count stays nonnegative and bounded; complete traversal/mutation ownership still needs verification. The previous negative EAX fixtures tested a caller branch, not an observed normal query return.
+
+## Reproduction and remaining work
+
+The evaluator adds signed JL/JS with separately modeled comparison/sign predicates, NOTL preserving flags, and NOPL/NOPW handling. Signed branches after unmodeled arithmetic fail closed; tests cover comparison overflow cases and unknown calls/registers. Accessor stack omission and diagnostic/callee-return injection remain explicit limitations.
+
+Eight tests check independent area/cluster/guard/store oracles, all unrolled service-site coverage, paired outputs and failure domains. For Rust, specify stable leaf ordering, area-slot replacement rather than set accumulation, cluster duplicates/overflow timing, and the distinct count/overflow/last-leaf outputs. RE-316 tracks full BSP traversal/plane predicates, leaf-array loading/layout/ownership, actual output bounds and diagnostics/state lifecycle, metadata/PVS/snapshot consumers, complete query/collision fixtures and original-runtime/all-platform conformance. Broad collision and port tasks remain open.

docs/research/LINK_BOUNDS.md

@@ -62,3 +62,5 @@ The scalar byte evaluator adds low-lane ANDPS/MAXSS/SQRTSS, register-only low-la
 Eight tests cover independent finite bounds/radius oracles, raw model/signed-zero gates, margin/metadata/request identity, leaf-return classification, paired engines, owned-removal agreement and NULL-owner difference, and evaluator failure boundaries. Earlier collision/area/tree reports regenerate unchanged.
 
 For Rust, keep absolute-bound preparation separate from narrow-phase transforms, specify expansion and rounding fixtures, distinguish membership preconditions from pointer presence, and treat BSP outputs as an explicit bounded contract. RE-312 tracks full leaf/area/cluster and packed-field behavior, real query outputs, object/angle/layout producers, relink/reset/ABI invariants, exceptional floats and whole-query/collision/original-runtime/platform conformance. Broad collision and port tasks remain open.
+
+The [leaf-metadata follow-up](LEAF_METADATA.md) recovers the store callback and clarifies the extra output as a retained leaf index; its wrapper returns stored count rather than a negative overflow encoding. The negative-count slice above remains a supplied caller-branch fixture.

docs/research/README.md

@@ -111,3 +111,5 @@ The extension collector adds five focused tests for native tail calls, command-r
 [World-sector construction and entity linking](WORLD_LINKS.md) adds RE-305–307: bounded sector topology/allocation, strict placement/head insertion and standalone unlink repairs/state. Reproduce with `python3 scripts/analyze_world_links.py`; check with `python3 scripts/test_world_links.py`. Complete bounds/BSP metadata, reset/relink/list ownership and whole-query/runtime/platform conformance remain open.
 
 [Entity bounds, inline removal and leaf-return gates](LINK_BOUNDS.md) adds RE-309–311: finite radius/padding compositions, metadata/query/return gates and owned-removal agreement with a NULL-owner precheck distinction. Reproduce with `python3 scripts/analyze_link_bounds.py`; check with `python3 scripts/test_link_bounds.py`. Full BSP metadata, producer/lifetime/relink/float and runtime/platform contracts remain open.
+
+[Leaf areas, clusters and query outputs](LEAF_METADATA.md) adds RE-313–315: all-lane area/cluster compositions, exact overflow fallback and guarded leaf/store callback contracts. Reproduce with `python3 scripts/analyze_leaf_metadata.py`; check with `python3 scripts/test_leaf_metadata.py`. Full BSP traversal/layout/output ownership, consumers and original-runtime/platform conformance remain open.

scripts/analyze_collision_composition.py

@@ -34,7 +34,7 @@ def get_word(m,address,width=4):
 
 def evaluate(graph,start,stops,memory,registers,limit=128,capture_registers=False):
     rows=graph['instructions'];indices={x['address']:i for i,x in enumerate(rows)}
-    regs={register(k)[0]:v for k,v in registers.items()};m=dict(memory);equal=None;ge=greater=below=None;signed_le=None;pc=start;trace=[];writes=[]
+    regs={register(k)[0]:v for k,v in registers.items()};m=dict(memory);equal=None;ge=greater=below=None;signed_le=signed_lt=negative=None;pc=start;trace=[];writes=[]
     def read_reg(name):
         key,size=register(name)
         if key not in regs:raise ValueError('unknown register')
@@ -60,13 +60,15 @@ def evaluate(graph,start,stops,memory,registers,limit=128,capture_registers=Fals
         x=rows[indices[pc]];trace.append(pc);op=x['opcode'];args=x['operands'].split('#')[0].strip()
         i=indices[pc];pc=rows[i+1]['address'] if i+1<len(rows) else None
         regs['rip']=pc
-        if op=='nop':pass
+        if op in ['nop','nopl','nopw']:pass
         elif op in ['movl','movq','movss','movabsq']:
             a,b=args.split(', ');width=8 if op in ['movq','movabsq'] else 4;write(b,read(a,width),width)
         elif op=='movaps':
             a,b=args.split(', ')
             if not a.startswith('%xmm') or not b.startswith('%xmm'):raise ValueError('unsupported vector move')
             write(b,read(a,4),4)  # Scalar low lane only.
+        elif op=='notl':
+            write(args,~read(args,4),4)  # NOT preserves flags.
         elif op=='cltq':
             v=read_reg('%eax');write('%rax',v if v<2**31 else v-2**32,8)
         elif op=='movslq':
@@ -81,25 +83,27 @@ def evaluate(graph,start,stops,memory,registers,limit=128,capture_registers=Fals
                 return (value^sign)-sign
             equal=v==w if op in ['cmpl','cmpq'] else (v&w)==0
             signed_le=signed(w)<=signed(v) if op in ['cmpl','cmpq'] else signed(v&w)<=0
+            signed_lt=signed(w)<signed(v) if op in ['cmpl','cmpq'] else signed(v&w)<0
+            negative=signed((w-v)&((1<<(width*8))-1))<0 if op in ['cmpl','cmpq'] else signed(v&w)<0
             ge=greater=below=None
         elif op in ['imull','imulq']:
             width=8 if op=='imulq' else 4;parts=args.split(', ')
             if len(parts)==2:a,b=parts;v,w=read(a,width),read(b,width);dest=b
             elif len(parts)==3:a,b,dest=parts;v,w=read(a,width),read(b,width)
             else:raise ValueError('unsupported multiply operands')
-            write(dest,v*w,width);equal=None;signed_le=None;ge=greater=below=None
+            write(dest,v*w,width);equal=None;signed_le=signed_lt=negative=None;ge=greater=below=None
         elif op=='shlq':
             a,b=args.split(', ');write(b,read(b,8)<<(read(a,8)&63),8)
-            equal=None;signed_le=None;ge=greater=below=None
+            equal=None;signed_le=signed_lt=negative=None;ge=greater=below=None
         elif op=='sarq':
             a,b=args.split(', ');v=read(b,8);v=v if v<2**63 else v-2**64
-            write(b,v>>(read(a,8)&63),8);equal=None;signed_le=None;ge=greater=below=None
+            write(b,v>>(read(a,8)&63),8);equal=None;signed_le=signed_lt=negative=None;ge=greater=below=None
         elif op in ['xorl','orl','andl','subl','addl','addq','subq']:
             width=8 if op in ['addq','subq'] else 4
             a,b=args.split(', ');v,w=read(a,width),read(b,width)
             result=v^w if op=='xorl' else v|w if op=='orl' else v&w if op=='andl' else w-v if op in ['subl','subq'] else v+w
             write(b,result,width);equal=(result&((1<<(width*8))-1))==0
-            signed_le=None;ge=greater=below=None
+            signed_le=signed_lt=negative=None;ge=greater=below=None
         elif op=='xorps':
             a,b=args.split(', ')
             if a!=b:raise ValueError('unsupported SIMD xor')
@@ -115,16 +119,16 @@ def evaluate(graph,start,stops,memory,registers,limit=128,capture_registers=Fals
             a,b=args.split(', ');v,w=number(read(a,4)),number(read(b,4))
             write(b,bits(w+v if op=='addss' else w-v if op=='subss' else w*v),4)
         elif op=='comiss':
-            a,b=args.split(', ');v,w=number(read(a,4)),number(read(b,4));equal=v==w;ge=w>=v;greater=w>v;below=w<v;signed_le=None
+            a,b=args.split(', ');v,w=number(read(a,4)),number(read(b,4));equal=v==w;ge=w>=v;greater=w>v;below=w<v;signed_le=signed_lt=negative=None
         elif op in ['sete','setbe']:
             if op=='sete' and equal is None:raise ValueError('unmodeled zero flag')
             if op=='setbe' and below is None:raise ValueError('unmodeled unsigned flags')
             write(args,int(equal if op=='sete' else below or equal),1)
-        elif op in ['je','jne','jae','ja','jb','jbe','jle','jmp']:
+        elif op in ['je','jne','jae','ja','jb','jbe','jle','jl','js','jmp']:
             if op in ['je','jne'] and equal is None:raise ValueError('unmodeled zero flag')
-            if op=='jle' and signed_le is None:raise ValueError('unmodeled signed flags')
+            if op in ['jle','jl','js'] and signed_le is None:raise ValueError('unmodeled signed flags')
             if op in ['jae','ja','jb','jbe'] and ge is None:raise ValueError('unmodeled unsigned flags')
-            taken=op=='jmp' or (equal if op=='je' else not equal if op=='jne' else signed_le if op=='jle' else greater if op=='ja' else below if op=='jb' else below or equal if op=='jbe' else ge)
+            taken=op=='jmp' or (equal if op=='je' else not equal if op=='jne' else signed_le if op=='jle' else signed_lt if op=='jl' else negative if op=='js' else greater if op=='ja' else below if op=='jb' else below or equal if op=='jbe' else ge)
             if taken:pc=int(args.split()[0],16)
         else:raise ValueError('unsupported composition opcode '+op)
     result={'trace':trace,'stop':pc,'memory':m,'writes':writes}

scripts/analyze_leaf_metadata.py

@@ -0,0 +1,115 @@
+#!/usr/bin/env python3
+"""Compose stopped native area/cluster loops and probe leaf access guards."""
+import gzip
+import hashlib
+import json
+from pathlib import Path
+import re
+from analyze_area_queries import run,site
+from analyze_chat_paths import native_string
+from analyze_collision_composition import get_word,put_word
+from analyze_engine_control import control_flow
+from analyze_native import elf
+from analyze_qvm import write_json
+from analyze_vm_extensions import native_functions,verified
+
+
+def sequence(pattern,count):
+    if pattern=='absent':return [-1]*count,[-1]*count
+    if pattern=='duplicates':return [1]*count,[7]*count
+    if pattern=='mixed':return [[1,2,3,-1][i%4] for i in range(count)],[[-1,7,7,9][i%4] for i in range(count)]
+    if pattern=='unique':return list(range(1,count+1)),list(range(100,100+count))
+    raise ValueError('unknown metadata pattern')
+
+
+def main():
+    out={'native':{},'limitations':'Synthetic stopped native instruction compositions. Every area/cluster '
+         'return and diagnostic continuation is supplied; original callees, BSP traversal and original '
+         'runtime never execute. Leaf accessor probes omit PUSH/POP stack traffic explicitly and stop '
+         'before error calls/returns. ABI/clobber/lifetime, actual maps/counts, complete metadata consumers '
+         'and all-platform conformance remain open.'}
+    record,stack,obj=0x20000,0x40000,0x10000
+    for label in ['linux--Quake3-UrT-Ded.x86_64','linux--Quake3-UrT.x86_64']:
+        r=json.loads(Path(f'docs/research/native/{label}.json').read_text());data=verified((Path('.cache/UrbanTerror43')/r['source']['path']).read_bytes(),r['source']['sha256'])
+        a=r['artifacts']['disassembly.txt'];fs=native_functions(verified(gzip.open(a['path'],'rb').read(),a['uncompressed_sha256']).decode());metadata,_=elf(data)
+        graphs={n:control_flow(fs[n]) for n in ['SV_LinkEntity','CM_LeafArea','CM_LeafCluster','CM_BoxLeafnums','CM_StoreLeafs']};g=graphs['SV_LinkEntity'];rows=g['instructions']
+        call_index=next(i for i,x in enumerate(rows) if '<CM_BoxLeafnums>' in x['operands']);start=rows[call_index+1]['address'];placement=site(g,'addl','$0x1, 0x1a4(%rbx)');cleanup=site(g,'addq','$0x228, %rsp')
+        calls={};debug={}
+        for i,x in enumerate(rows):
+            for service in ['CM_LeafArea','CM_LeafCluster']:
+                if x['opcode']=='callq' and '<'+service+'>' in x['operands']:calls[x['address']]={'service':service,'after':rows[i+1]['address']}
+            if x['opcode']=='callq' and '<Com_DPrintf>' in x['operands']:
+                # Each diagnostic block begins at its first CVTSS2SD, not its nearby previous block.
+                begin=next(k for k in range(max(0,i-8),i) if rows[k]['opcode']=='cvtss2sd')
+                debug[rows[begin]['address']]={'call':x['address'],'after':rows[i+1]['address']}
+        if len(debug)!=8:raise ValueError('diagnostic lane count changed')
+        sv_ins=next(x for x in rows if x['opcode']=='cmpl' and x['operands'].startswith('$0x1,') and '(%rip)' in x['operands']);sv_address=int(re.search(r'#\s*0x([0-9a-f]+)',sv_ins['operands'])[1],16)
+        stops=list(calls)+list(debug)+[placement,cleanup];compositions=[]
+        for count in [0,1,2,3,4,5,6,7,8,15,16,17,128]:
+            for pattern in ['absent','duplicates','mixed','unique']:
+                for sv_state in [0,1]:
+                    areas,clusters=sequence(pattern,count);m={};put_word(m,sv_address,sv_state);put_word(m,stack+0x21c,1000)
+                    for off,val in [(0xe0,0),(0x124,0),(0x128,-1),(0x12c,-1)]:put_word(m,record+off,val)
+                    for i in range(16):put_word(m,record+0xe4+4*i,0xaabbccdd)
+                    for i in range(count):put_word(m,stack+0x10+4*i,100+i)
+                    regs={'%rax':count,'%rsp':stack,'%r12':record,'%rbx':obj,'%rcx':0};pc=start;trace=[];requests=[];segments=0
+                    while pc not in [placement,cleanup]:
+                        if segments>500:raise ValueError('metadata composition bound')
+                        t=run(g,pc,stops,m,regs);m=t['memory'];regs=t['registers'];pc=t['stop'];trace+=t['trace'];segments+=1
+                        if pc in calls:
+                            c=calls[pc];leaf=regs['rdi']&0xffffffff
+                            if leaf==1000 and c['service']=='CM_LeafCluster':value=777
+                            elif 100<=leaf<100+count:value=(areas if c['service']=='CM_LeafArea' else clusters)[leaf-100]
+                            else:raise ValueError('unexpected requested leaf')
+                            requests.append([c['service'],leaf,value,pc]);regs['rax']=value&0xffffffff;pc=c['after']
+                        elif pc in debug:
+                            requests.append(['Com_DPrintf',None,None,debug[pc]['call']]);pc=debug[pc]['after']
+                    compositions.append({'count':count,'pattern':pattern,'sv_state_word':sv_state,'supplied_areas':areas,'supplied_clusters':clusters,
+                        'area_words':[get_word(m,record+0x128),get_word(m,record+0x12c)],'cluster_count':get_word(m,record+0xe0),
+                        'cluster_words':[get_word(m,record+0xe4+4*i) for i in range(16)],'fallback_word':get_word(m,record+0x124),
+                        'requests':requests,'reaches_placement':pc==placement,'trace':trace if count<=3 else None,
+                        'trace_instruction_count':len(trace),'trace_sha256':hashlib.sha256(json.dumps(trace,separators=(',',':')).encode()).hexdigest()})
+        accessors={}
+        for name in ['CM_LeafArea','CM_LeafCluster']:
+            original=graphs[name];model=dict(original);model['instructions']=[dict(x) for x in original['instructions']];omitted=[]
+            for x in model['instructions']:
+                if x['opcode'] in ['pushq','popq']:
+                    if x['operands']!='%rbx':raise ValueError('unexpected accessor stack traffic')
+                    omitted.append(x['address']);x['opcode']='nop';x['operands']=''
+            if len(omitted)!=2:raise ValueError('accessor frame changed')
+            rows2=original['instructions'];cmp=next(x for x in rows2 if x['opcode']=='cmpl' and '(%rip)' in x['operands']);count_address=int(re.search(r'#\s*0x([0-9a-f]+)',cmp['operands'])[1],16)
+            base_load=next(x for x in rows2 if x['opcode']=='movq' and '(%rip)' in x['operands']);base_address=int(re.search(r'#\s*0x([0-9a-f]+)',base_load['operands'])[1],16)
+            error=next(x['address'] for x in rows2 if '<Com_Error>' in x['operands']);end=next(x['address'] for x in rows2 if x['opcode']=='retq');probes=[]
+            for leaf_count in [0,1,3]:
+                for index in [-1,0,1,2,3]:
+                    m={};put_word(m,count_address,leaf_count);put_word(m,base_address,0x80000,8)
+                    for i in range(3):put_word(m,0x80000+24*i,100+i);put_word(m,0x80004+24*i,200+i)
+                    t=run(model,model['entry'],[error,end],m,{'%rdi':index,'%rax':0})
+                    probes.append({'leaf_count':leaf_count,'index':index,'error_requested':t['stop']==error,'result_bits':None if t['stop']==error else t['registers']['rax'],'error_code':t['registers']['rdi'] if t['stop']==error else None,'trace':t['trace']})
+            msg=next(x for x in rows2 if x['opcode']=='movl' and x['operands'].split('#')[0].strip().endswith(', %esi'));address=int(msg['operands'].split(',')[0][1:],0)
+            accessors[name]={'count_address':count_address,'base_address':base_address,'omitted_stack_instructions':omitted,
+                'error_text':native_string(data,metadata['sections'],address),'probes':probes}
+        store=graphs['CM_StoreLeafs'];ret_stops=[x['address'] for x in store['instructions'] if x['opcode']=='retq']
+        callback=next(x for x in graphs['CM_BoxLeafnums']['instructions'] if x['opcode']=='movq' and x['operands'].split('#')[0].strip().endswith(', 0x38(%rsp)'))
+        if int(callback['operands'].split(',')[0][1:],0)!=store['entry']:raise ValueError('leaf callback changed')
+        list_record,output=0x60000,0x90000;store_traces=[];base_slot=accessors['CM_LeafCluster']['base_address']
+        for index in [0,1,2]:
+            for cluster in [-1,7]:
+                for count,capacity in [(0,0),(0,1),(1,1),(1,2),(127,128),(128,128)]:
+                    m={};put_word(m,base_slot,0x80000,8);put_word(m,0x80000+24*index,cluster)
+                    for off,val,width in [(0,count,4),(4,capacity,4),(8,0,4),(16,output,8),(48,999,4)]:put_word(m,list_record+off,val,width)
+                    t=run(store,store['entry'],ret_stops,m,{'%rdi':list_record,'%rsi':(~index)&0xffffffff})
+                    store_traces.append({'index':index,'cluster':cluster,'count_before':count,'capacity':capacity,
+                        'count_after':get_word(t['memory'],list_record),'overflow_word':get_word(t['memory'],list_record+8),
+                        'last_leaf_word':get_word(t['memory'],list_record+48),'output_writes':[{'address':w['address'],'value':get_word(t['memory'],w['address'],w['bytes']),'bytes':w['bytes']} for w in t['writes'] if w['address']>=output], 'trace':t['trace']})
+        first_debug_call=next(x for x in rows if x['address']==next(iter(debug.values()))['call'])
+        first_debug_index=next(i for i,x in enumerate(rows) if x['address']==first_debug_call['address'])
+        fmt=next(x for x in rows[first_debug_index-8:first_debug_index] if x['opcode']=='movl' and x['operands'].split('#')[0].strip().endswith(', %edi'))
+        fmt_address=int(fmt['operands'].split(',')[0][1:],0)
+        out['native'][label]={'source':r['source'],'disassembly_sha256':a['uncompressed_sha256'],'graphs':graphs,
+            'sv_state_address':sv_address,'service_sites':calls,'diagnostic_sites':debug,'compositions':compositions,'accessors':accessors,'store_leaf_traces':store_traces,'diagnostic_text':native_string(data,metadata['sections'],fmt_address)}
+    write_json(Path('docs/research/leaf-metadata.json'),out)
+    print('Recovered 208 area/cluster compositions, 60 guarded accessors and 72 leaf-store probes')
+
+
+if __name__=='__main__':main()

scripts/test_leaf_metadata.py

@@ -0,0 +1,115 @@
+#!/usr/bin/env python3
+"""Independent metadata, guard, capacity and ordering oracles."""
+import json
+from pathlib import Path
+import unittest
+from analyze_collision_composition import evaluate
+from test_collision_composition import graph
+
+
+class LeafMetadataTests(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):cls.r=json.loads(Path('docs/research/leaf-metadata.json').read_text())
+
+    def test_first_area_retained_second_replaced_and_diagnostic_gate(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['compositions']),104)
+            for x in n['compositions']:
+                first=second=-1;diagnostics=0
+                for area in x['supplied_areas']:
+                    if area==-1:continue
+                    if first==-1 or area==first:first=area
+                    else:
+                        if second!=-1 and second!=area and x['sv_state_word']==1:diagnostics+=1
+                        second=area
+                self.assertEqual(x['area_words'],[first&0xffffffff,second&0xffffffff])
+                self.assertEqual(sum(c[0]=='Com_DPrintf' for c in x['requests']),diagnostics)
+                self.assertEqual(x['reaches_placement'],x['count']>0)
+
+    def test_cluster_append_duplicates_absent_and_exact_sixteen_fallback(self):
+        for n in self.r['native'].values():
+            for x in n['compositions']:
+                qualifying=[v for v in x['supplied_clusters'] if v!=-1];expected=qualifying[:16]
+                self.assertEqual(x['cluster_count'],len(expected))
+                self.assertEqual(x['cluster_words'],expected+[0xaabbccdd]*(16-len(expected)))
+                self.assertEqual(x['fallback_word'],777 if len(qualifying)>=16 else 0)
+                fallback=[c for c in x['requests'] if c[0]=='CM_LeafCluster' and c[1]==1000]
+                self.assertEqual(len(fallback),int(len(qualifying)>=16))
+                if x['pattern']=='duplicates' and x['count']>=16:self.assertEqual(x['cluster_words'],[7]*16)
+
+    def test_call_order_coverage_and_cluster_early_stop(self):
+        for n in self.r['native'].values():
+            observed={'CM_LeafArea':set(),'CM_LeafCluster':set()}
+            for x in n['compositions']:
+                calls=[c for c in x['requests'] if c[0]!='Com_DPrintf']
+                area=[c for c in calls if c[0]=='CM_LeafArea'];cluster=[c for c in calls if c[0]=='CM_LeafCluster']
+                self.assertEqual([c[1] for c in area],list(range(100,100+x['count'])))
+                self.assertEqual(calls,area+cluster)
+                stop=x['count'];seen=0
+                for i,v in enumerate(x['supplied_clusters']):
+                    seen+=v!=-1
+                    if seen==16:stop=i+1;break
+                self.assertEqual([c[1] for c in cluster if c[1]!=1000],list(range(100,100+stop)))
+                for service,_,_,address in calls:observed[service].add(address)
+                if x['trace'] is not None:self.assertEqual(len(x['trace']),x['trace_instruction_count'])
+            for service,addresses in observed.items():
+                self.assertEqual(addresses,{int(a) for a,c in n['service_sites'].items() if c['service']==service})
+
+    def test_leaf_accessors_signed_guards_stride_and_field_offsets(self):
+        for n in self.r['native'].values():
+            a=n['accessors']['CM_LeafArea'];b=n['accessors']['CM_LeafCluster']
+            self.assertEqual(a['base_address'],b['base_address']);self.assertEqual(a['count_address'],b['count_address'])
+            for name,info in n['accessors'].items():
+                self.assertEqual(len(info['omitted_stack_instructions']),2)
+                self.assertEqual(len(info['probes']),15)
+                self.assertEqual(info['error_text'],name+': bad number')
+                for x in info['probes']:
+                    valid=0<=x['index']<x['leaf_count']
+                    self.assertEqual(x['error_requested'],not valid)
+                    self.assertEqual(x['error_code'],None if valid else 1)
+                    self.assertEqual(x['result_bits'],(200 if name=='CM_LeafArea' else 100)+x['index'] if valid else None)
+
+    def test_leaf_store_last_nonnegative_cluster_before_capacity(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['store_leaf_traces']),36)
+            for x in n['store_leaf_traces']:
+                room=x['count_before']<x['capacity']
+                self.assertEqual(x['count_after'],x['count_before']+int(room))
+                self.assertEqual(x['overflow_word'],int(not room))
+                self.assertEqual(x['last_leaf_word'],x['index'] if x['cluster']!=-1 else 999)
+                self.assertEqual(x['output_writes'],[{'address':0x90000+4*x['count_before'],'value':x['index'],'bytes':4}] if room else [])
+            box=n['graphs']['CM_BoxLeafnums']['instructions']
+            self.assertTrue(any(x['operands']=='0x30(%rsp), %edx' for x in box))
+            self.assertTrue(any(x['operands']=='(%rsp), %eax' for x in box))
+            self.assertFalse(any(x['opcode']=='movl' and x['operands']=='0x8(%rsp), %eax' for x in box))
+
+    def test_paired_outputs_match_with_call_addresses_removed(self):
+        a,b=self.r['native'].values()
+        for key in ['compositions','store_leaf_traces']:
+            def outputs(n):
+                result=[]
+                for x in n[key]:
+                    y={k:v for k,v in x.items() if k not in ['trace','trace_sha256','requests']}
+                    if 'requests' in x:y['requests']=[c[:3] for c in x['requests']]
+                    result.append(y)
+                return result
+            self.assertEqual(outputs(a),outputs(b))
+
+    def test_signed_comparison_overflow_and_negative_flag(self):
+        g=graph(('cmpl','$0x7fffffff, %edi'),('jl','0x4'),('movl','$0x1, %eax'))
+        for word,expected in [(0x80000000,9),(0x7fffffff,1),(0xffffffff,9)]:
+            t=evaluate(g,1,[4],{}, {'%rdi':word,'%rax':9},capture_registers=True)
+            self.assertEqual(t['registers']['rax'],expected)
+        g=graph(('testl','%edi, %edi'),('notl','%eax'),('nopl','(%rax)'),('js','0x6'),('movl','$0x1, %eax'))
+        for word in [0,1,0xffffffff,0x80000000]:
+            t=evaluate(g,1,[6],{}, {'%rdi':word,'%rax':0},capture_registers=True)
+            self.assertEqual(t['registers']['rax'],0xffffffff if word>=0x80000000 else 1)
+
+    def test_unmodeled_signed_flags_and_calls_rejected(self):
+        with self.assertRaises(ValueError):evaluate(graph(('subl','$0x1, %eax'),('jl','0x3')),1,[3],{}, {'%rax':0})
+        with self.assertRaises(ValueError):evaluate(graph(('js','0x2')),1,[2],{}, {})
+        with self.assertRaises(ValueError):evaluate(graph(('callq','0x1234')),1,[2],{}, {})
+        with self.assertRaises(ValueError):evaluate(graph(('notl','%eax')),1,[2],{}, {})
+
+
+if __name__=='__main__':unittest.main()