Recover world sector topology and entity link ownership

docs/ASSET_PIPELINE_TODO.md

@@ -6,6 +6,8 @@ The [entity-clipping loop collector](research/COLLISION_LOOP.md) extends binary
 
 The [area-query collector](research/AREA_QUERIES.md) adds inclusive-overlap and strict-split fixtures for future converted collision validation; full node placement and map query equivalence remain open. Reproduce with `python3 scripts/analyze_area_queries.py`.
 
+The [world-sector/link collector](research/WORLD_LINKS.md) adds topology, placement and list-order evidence for future converted collision fixtures; original bound producers and full query conformance remain open. Reproduce with `python3 scripts/analyze_world_links.py`.
+
 ## Planned tool catalogue
 
 Names and output formats below are proposals to refine after the relevant research gates. They may become Python scripts, Rust CLI subcommands, or both. Runtime-critical parsers and shared schemas should have one authoritative implementation; orchestration scripts must not diverge from it.

docs/FINDINGS.md

@@ -312,3 +312,9 @@ Bounded qagame paths recover bomb-plant/inactivity correction requests, direct f
 **F-086 — Observed area overlap/capacity boundaries:** finite candidate-box comparisons include touching faces/edges/corners. Accepted entries test count == capacity before a four-byte index write and count increment; deliberately invalid counts expose the guard's equality-only nature. Count/array ownership and runtime safety remain unverified. [AREA-002](research/AREA_QUERIES.md).
 
 **F-087 — Observed area ordering and split rules:** local linked records precede children; maximum > split requests child 0 before minimum < split tail-selects child 1. Axis -1 returns. Plane equality excludes each corresponding child, so completeness depends on node placement; full trees/callees/runtime remain open. [AREA-003](research/AREA_QUERIES.md).
+
+**F-088 — Observed sector construction:** depth ==4 ends construction; X extent > Y selects X, otherwise Y, ignoring Z. Explicit depth-zero compositions yield 31 preorder nodes. Synthetic count 64 aliases the adjacent counter, so valid reset/count ownership remains essential and unverified at runtime. [LINK-001](research/WORLD_LINKS.md).
+
+**F-089 — Observed placement and list order:** strictly one-sided bounds descend; touching/crossing bounds stay at the parent. Link tails prepend the server record and set owner/linked words; repeated distinct-record insertions reverse insertion order. Full bounds/metadata/relink/query/runtime composition remains open. [LINK-002](research/WORLD_LINKS.md).
+
+**F-090 — Observed unlink state:** NULL owner skips removal; nonzero owner clears before head/predecessor search. Missing/empty cases select a warning, and removed-record next remains unchanged. Lifecycle/list validity and full runtime/platform behavior remain open. [LINK-003](research/WORLD_LINKS.md).

docs/PROVISIONAL_TODO.md

@@ -33,7 +33,7 @@ For implementation items, the minimum done condition is a documented contract, a
 
 ### First research batch
 
-Progress: [RE-232–234 / TIME-001–003](research/FRAME_TIMING.md) now cover Linux ELF64 elapsed modification, event sources and pause predicates. The [RE-237–239 follow-up](research/EVENT_QUEUES.md) resolves the pause-field address/bot chain and adds queue/journal evidence. The [RE-241–243 announcement pass](research/AUTH_ANNOUNCEMENTS.md) recovers label tables and recipient paths; The [RE-245–247 cvar pass](research/GAME_CVARS.md) adds all 126 main registration rows and resolves auth_verbosity/callers. The [RE-249–251 change pass](research/CVAR_CHANGES.md) adds cache/notification behavior, four numeric correction branches and direct registration coverage. The [RE-253–255 query pass](research/AUTH_QUERIES.md) adds selector helpers, command-role routing and internal authentication modes. The [RE-257–259 bounds/entry pass](research/SELECTOR_BOUNDS.md) adds integer parsing, selector storage/count evidence and native/QVM/chat route connections. The [RE-261–263 chat pass](research/CHAT_PATHS.md) adds native command metadata/filter evidence and bounded message/recipient paths, including the documented argument-mapping correction. The [RE-265–267 delivery pass](research/CHAT_DELIVERY.md) adds bounded dead-chat predicates, formatter argument routing and echo/fanout calls. The [RE-269–271 expansion pass](research/CHAT_EXPANSION.md) adds marker states, twelve key comparisons and health/momo branches. The [RE-273–275 item/time pass](research/CHAT_ITEMS_TIME.md) adds packed ammunition fields, sampled names and countdown branches. The [RE-277–279 location pass](research/CHAT_LOCATIONS.md) adds leader selection, cache/format boundaries and location-list/spatial evidence. The [RE-281–283 crosshair pass](research/CHAT_CROSSHAIR.md) adds finite geometry fixtures, trace arguments and result routing. The [RE-285–287 native spatial pass](research/NATIVE_SPATIAL.md) confirms Linux trig/spatial services, VM pointer branches and collision/query evidence. The [RE-289–291 collision pass](research/NATIVE_COLLISION.md) adds native candidate filters, result decisions and model/field evidence. The [RE-293–295 composition pass](research/COLLISION_COMPOSITION.md) adds three-lane byte-copy/status agreement and world-result routing. The [RE-297–299 loop pass](research/COLLISION_LOOP.md) adds signed count/status gates, parity scheduling and related sentinel normalization. The [RE-301–303 area pass](research/AREA_QUERIES.md) adds registered address arithmetic, inclusive overlap and count/split ordering boundaries. PROV-002 remains open for full output/reset coverage. PROV-001 remains open for complete scheduling and platform coverage.
+Progress: [RE-232–234 / TIME-001–003](research/FRAME_TIMING.md) now cover Linux ELF64 elapsed modification, event sources and pause predicates. The [RE-237–239 follow-up](research/EVENT_QUEUES.md) resolves the pause-field address/bot chain and adds queue/journal evidence. The [RE-241–243 announcement pass](research/AUTH_ANNOUNCEMENTS.md) recovers label tables and recipient paths; The [RE-245–247 cvar pass](research/GAME_CVARS.md) adds all 126 main registration rows and resolves auth_verbosity/callers. The [RE-249–251 change pass](research/CVAR_CHANGES.md) adds cache/notification behavior, four numeric correction branches and direct registration coverage. The [RE-253–255 query pass](research/AUTH_QUERIES.md) adds selector helpers, command-role routing and internal authentication modes. The [RE-257–259 bounds/entry pass](research/SELECTOR_BOUNDS.md) adds integer parsing, selector storage/count evidence and native/QVM/chat route connections. The [RE-261–263 chat pass](research/CHAT_PATHS.md) adds native command metadata/filter evidence and bounded message/recipient paths, including the documented argument-mapping correction. The [RE-265–267 delivery pass](research/CHAT_DELIVERY.md) adds bounded dead-chat predicates, formatter argument routing and echo/fanout calls. The [RE-269–271 expansion pass](research/CHAT_EXPANSION.md) adds marker states, twelve key comparisons and health/momo branches. The [RE-273–275 item/time pass](research/CHAT_ITEMS_TIME.md) adds packed ammunition fields, sampled names and countdown branches. The [RE-277–279 location pass](research/CHAT_LOCATIONS.md) adds leader selection, cache/format boundaries and location-list/spatial evidence. The [RE-281–283 crosshair pass](research/CHAT_CROSSHAIR.md) adds finite geometry fixtures, trace arguments and result routing. The [RE-285–287 native spatial pass](research/NATIVE_SPATIAL.md) confirms Linux trig/spatial services, VM pointer branches and collision/query evidence. The [RE-289–291 collision pass](research/NATIVE_COLLISION.md) adds native candidate filters, result decisions and model/field evidence. The [RE-293–295 composition pass](research/COLLISION_COMPOSITION.md) adds three-lane byte-copy/status agreement and world-result routing. The [RE-297–299 loop pass](research/COLLISION_LOOP.md) adds signed count/status gates, parity scheduling and related sentinel normalization. The [RE-301–303 area pass](research/AREA_QUERIES.md) adds registered address arithmetic, inclusive overlap and count/split ordering boundaries. The [RE-305–307 world-link pass](research/WORLD_LINKS.md) adds bounded sector construction, strict placement/head insertion and unlink state/repair behavior. PROV-002 remains open for full output/reset coverage. PROV-001 remains open for complete scheduling and platform coverage.
 
 Start with the open questions raised by the current binary evidence. These tasks depend on the linked reports, not on a working rewrite.
 

docs/RESEARCH_LEDGER.md

@@ -165,3 +165,7 @@ Completed bounded RE-297–299 with `analyze_collision_loop.py`: paired verified
 ## 2026-10-11 — Entity addressing and area-query boundaries
 
 Completed bounded/static RE-301–303 with `analyze_area_queries.py`: ten verified native graphs, 64 direct/24 reverse address traces, two registration slices, 432 overlap/48 insertion slices and 48 split compositions. AREA-001–003/F-085–087 distinguish registered game stride from fixed server stride, closed box overlap from strict tree splits, and equality guards from established safety. Eight tests check independent arithmetic/geometry/capacity/order oracles, registration widths, paired results and unknown/exceptional flag/address domains. The byte evaluator adds RIP/scaled addresses, multiply/sign-extension/shift/subtract/qword-test and finite comparison branches, with unmodeled flags rejected. The full suite passes 214 tests; new and prior collision reports regenerate byte-identically, and storage policy/documentation link checks pass. RE-300 remains open; RE-304 tracks complete registration/node/list/callee/count ownership, geometry/runtime and platform behavior. No original process, recursive query or recorded native callee executes.
+
+## 2026-10-11 — Sector construction and entity link ownership
+
+Completed bounded/static RE-305–307 with `analyze_world_links.py`: eight verified graphs, six explicit constructor compositions, 32 local allocation/depth slices, 128 link and 14 unlink slices. LINK-001–003/F-088–090 distinguish normal depth-zero topology from supplied out-of-range counter aliasing, strict parent placement from inclusive overlap, and stale next links from membership. Eight tests check independent topology/list/order/state oracles, paired outputs, float rounding/flag preservation and bounded cycle failures. The byte evaluator adds CMPQ/CLTQ/SHLQ and finite scalar SSE arithmetic/SETBE/JBE. The full suite passes 222 tests; new and prior collision/area reports regenerate byte-identically, and storage/documentation links pass validation. RE-304 remains open; RE-308 tracks bounds/BSP metadata, reset/lifetime/relink ownership, full queries/collision order and original-runtime/platform behavior. No original process or native callee executes.

docs/REVERSE_ENGINEERING_TODO.md

@@ -362,3 +362,8 @@ Purpose: turn the supplied installation into an evidence-backed specification of
 - [x] RE-302 Recover finite inclusive candidate-box overlap and equality-only capacity/index insertion slices ([AREA-002](research/AREA_QUERIES.md)).
 - [x] RE-303 Recover static local-list-before-children order and bounded strict split/leaf/child-order compositions ([AREA-003](research/AREA_QUERIES.md)).
 - [ ] RE-304 Recover registered ABI callers/sizes/counts/lifetime and pointer/index validation, area node/list construction/relinking/order/duplicates, full recursive count/capacity propagation and callee state effects, exceptional floats and original geometry/runtime/all-platform query fixtures.
+
+- [x] RE-305 Recover paired sector reset/allocation/depth/axis rules and explicit bounded constructor compositions ([LINK-001](research/WORLD_LINKS.md)).
+- [x] RE-306 Recover strict entity placement, parent retention at plane equality and head-insertion/link-word writes ([LINK-002](research/WORLD_LINKS.md)).
+- [x] RE-307 Recover standalone unlink owner/flag clearing, head/predecessor repair, missing-list warning and retained-next behavior ([LINK-003](research/WORLD_LINKS.md)).
+- [ ] RE-308 Recover complete rotated/padded bounds and BSP leaf/area/cluster metadata, sector/reset/registration and record lifetime invariants, inlined relink equivalence, composed whole-tree query/collision ordering, malformed/exceptional states and original-runtime/all-platform fixtures.

docs/research/README.md

@@ -107,3 +107,5 @@ The extension collector adds five focused tests for native tail calls, command-r
 [Entity-clipping loop and sentinel normalization](COLLISION_LOOP.md) adds RE-297–299: signed count/status entry gates, parity-selected lanes, bounded continuation/scheduling compositions and pass/related sentinel normalization. Reproduce with `python3 scripts/analyze_collision_loop.py`; check with `python3 scripts/test_collision_loop.py`. Count/order/lookup ownership, candidate/callee/geometry behavior and runtime/platform conformance remain open.
 
 [Entity addressing and area-query boundaries](AREA_QUERIES.md) adds RE-301–303: registered low-width addressing, inclusive overlap/capacity insertion and static list/strict split ordering. Reproduce with `python3 scripts/analyze_area_queries.py`; check with `python3 scripts/test_area_queries.py`. Complete node/list/count/ABI ownership, query composition and runtime/platform conformance remain open.
+
+[World-sector construction and entity linking](WORLD_LINKS.md) adds RE-305–307: bounded sector topology/allocation, strict placement/head insertion and standalone unlink repairs/state. Reproduce with `python3 scripts/analyze_world_links.py`; check with `python3 scripts/test_world_links.py`. Complete bounds/BSP metadata, reset/relink/list ownership and whole-query/runtime/platform conformance remain open.

docs/research/WORLD_LINKS.md

@@ -0,0 +1,62 @@
+# World-sector construction, entity placement and unlinking
+
+Static/bounded pass, 2026-10-11. Reproduce with `python3 scripts/analyze_world_links.py`; check with `python3 scripts/test_world_links.py`. [Generated evidence](world-links.json) verifies original/disassembly hashes for both Linux ELF64 engines, retains four graphs per engine and records **six tree compositions, 32 local constructor slices, 128 placement/link slices and 14 unlink slices**. This extends [area-query boundary and ordering evidence](AREA_QUERIES.md).
+
+No original process, memset, model-bound query, entity lookup or recursive native callee executes. Tree compositions drive stopped constructor slices and explicitly substitute each child return; they supply zeroed storage and finite bounds. Placement starts after the unrecovered bounds/leaf/cluster metadata pipeline. Unlink starts after a supplied server-record lookup return. Frame/ABI/callee effects, exceptional floats and full runtime/platform conformance remain open.
+
+## LINK-001: cleared storage and depth-four sector construction
+
+SV_ClearWorld statically requests memset of **2048 bytes** at the world-sector root, resets sv_numworldSectors to zero, requests inline model zero and its bounds, then requests SV_CreateworldSector with depth zero. Those calls are retained as evidence rather than executed. The constructor uses **32-byte records**:
+
+| Offset | Observed role |
+| --- | --- |
+| +0 | 32-bit split axis, or 0xffffffff at a leaf |
+| +4 | Single-precision split coordinate |
+| +8 | Child 0 pointer |
+| +16 | Child 1 pointer |
+| +24 | Linked server-record head |
+
+Allocation reads the counter, increments it, and selects root +32*sign_extend(old_count). A leaf is selected by **depth ==4**, not depth >=4. Leaf code writes axis -1 and zero child pointers; head and split rely on the supplied clear state. Non-leaf code compares X and Y extents, selects X only when X extent > Y extent, otherwise Y, and computes `(maximum[axis] + minimum[axis]) * 0.5` with binary32 rounding at each arithmetic instruction. Z does not select a split axis.
+
+Child 0 receives the upper half (selected minimum becomes split); child 1 receives the lower half (selected maximum becomes split). Child 0 is requested and stored before child 1. The explicit compositions use three bound pairs: [0,0,0]–[16,8,4], [-8,-8,-100]–[8,8,100], and zero extents. Each yields **31 preorder-allocated nodes**, including sixteen depth-four leaves, with the same topology/values in both engines after address normalization. Large Z and equal extents confirm the observed tie/axis rule; degenerate bounds still create the fixed topology.
+
+Sixteen additional local cases per engine combine depths 0/3/4/5 and supplied counts 0/30/63/64. Depth 5 requests further construction rather than returning a leaf; that invalid starting depth is stopped before recursion. No local capacity guard occurs before allocation. Crucially, **root +2048 equals the counter address** in both engines: supplied count 64 selects the counter as the new record, and its axis write overwrites the incremented counter (zero for tested non-leaf X selection, 0xffffffff for a leaf). This demonstrates raw out-of-range aliasing, not normal runtime reachability or a security conclusion. Depth-zero compositions stay at 31 nodes; startup/reset callers and allocation invariants still require complete verification.
+
+## LINK-002: strict placement and head insertion
+
+The bounded placement slices start at the observed object +0x1a4 increment after leaf/cluster work, supply an existing constructed tree and game-object absolute bounds at +0x1d0/+0x1dc, and stop before frame cleanup. The selected server record is in r12.
+
+At each node:
+
+- Axis -1 stops descent.
+- Object minimum[axis] > split descends to child 0.
+- Otherwise object maximum[axis] < split descends to child 1.
+- Otherwise it stays at the current node. Touching or crossing the plane therefore stays at the parent.
+
+The 64 cases per engine combine eight X intervals and eight Y intervals, with Z [0,1]. They cover split equality, crossings, points and boxes outside the supplied root bounds. These are supplied absolute bounds; radius/rotation/padding and BSP leaf validation were not executed.
+
+Insertion then writes selected node into record +0, old node head into record +8, record into node +24, and one into game object +0x1a0. The sampled object +0x1a4 changes from 41 to 42. Each fixture uses an opaque supplied old-head pointer; the placement slice does not dereference that pointer. A separate three-insertion tail composition verifies the resulting local list order is reverse insertion order.
+
+Combined with the prior query's local-list-first/child-0-first ordering, this explains why candidate order is sensitive to linking history rather than a numeric entity sort. Touching-plane placement at the parent also explains how a query exactly on a split can skip both children while still checking touching records stored locally. Complete query correctness requires actual bound producers, linked-list validity, and node placement/query composition across the whole tree.
+
+SV_LinkEntity's retained prelude checks an existing owner pointer and has an inlined removal path before metadata production. The collector does not execute that prelude or claim full relink equivalence. Linking the same record twice by invoking only the isolated tail would bypass that guard; those tail compositions use distinct records.
+
+## LINK-003: standalone unlink state and list repairs
+
+After a supplied SV_SvEntityForGentity return, SV_UnlinkEntity first clears game object +0x1a0. If record +0 is NULL, it returns without touching the node/list. Otherwise it clears record +0 **before** searching the recorded node's list.
+
+- Target is head: node +24 becomes target +8.
+- Target is later in the list: predecessor +8 becomes target +8.
+- Target is missing or the node list is empty: the warning path is selected, without a list repair.
+
+The retained warning is `WARNING: SV_UnlinkEntity: not found in worldSector\n`. Fourteen paired fixtures cover NULL owner, head, middle, tail, sole member, missing record and empty list. The linked word ends zero, and supplied nonzero owner is cleared even on missing/empty failure. **Record +8 is not cleared**; its stale next value survives. Consumers must use owner/linked state rather than treating a nonzero next pointer as membership.
+
+A cyclic synthetic list that cannot reach the target fails at the evaluator's step limit. This does not establish a native cycle detector; the original loop has no such observed guard. Corrupt ownership, stale pointers, concurrent mutation and actual lifecycle/reset invariants remain unresolved.
+
+## Evaluation and rewrite implications
+
+The byte evaluator adds qword CMP, CLTQ, qword shift-left, finite binary32 ADDSS/SUBSS/MULSS, SETBE and JBE. SSE arithmetic preserves integer comparison flags; scalar arithmetic results round through binary32 and reject nonfinite inputs/results. Multiplication/shift flag knowledge remains invalidated, and unsupported addressing/reads/operations/step escape fail closed. Prior collision/area reports regenerate unchanged.
+
+Eight focused tests use independent tree/placement/list oracles and cover allocation/depth alias boundaries, exact insertion writes, unlink state including retained next, a three-record insertion order composition, paired results, rounding/flag/sign-extension rules and cyclic-list limits. This remains synthetic instruction evidence rather than original-runtime conformance.
+
+For Rust, specify bounded sector storage and depth ownership, retain strict placement versus inclusive overlap distinctions, preserve candidate-order fixtures and model membership independently from stale next links. RE-308 tracks complete rotated/padded bounds, BSP leaf/area/cluster metadata, registration/reset/relink ownership, composed whole-tree queries/collision ordering, exceptional floats and original-runtime/all-platform equivalence. Broad collision/game/port tasks stay open.

scripts/analyze_collision_composition.py

@@ -62,18 +62,20 @@ def evaluate(graph,start,stops,memory,registers,limit=128,capture_registers=Fals
         if op=='nop':pass
         elif op in ['movl','movq','movss','movabsq']:
             a,b=args.split(', ');width=8 if op in ['movq','movabsq'] else 4;write(b,read(a,width),width)
+        elif op=='cltq':
+            v=read_reg('%eax');write('%rax',v if v<2**31 else v-2**32,8)
         elif op=='movslq':
             a,b=args.split(', ');v=read(a,4);write(b,v if v<2**31 else v-2**32,8)
         elif op in ['leal','leaq']:
             a,b=args.split(', ');write(b,address(a),4 if op=='leal' else 8)
-        elif op in ['testl','testb','testq','cmpl']:
-            width=1 if op=='testb' else 8 if op=='testq' else 4
+        elif op in ['testl','testb','testq','cmpl','cmpq']:
+            width=1 if op=='testb' else 8 if op in ['testq','cmpq'] else 4
             a,b=args.split(', ');v,w=read(a,width),read(b,width)
             def signed(value):
                 sign=1<<(width*8-1)
                 return (value^sign)-sign
-            equal=v==w if op=='cmpl' else (v&w)==0
-            signed_le=signed(w)<=signed(v) if op=='cmpl' else signed(v&w)<=0
+            equal=v==w if op in ['cmpl','cmpq'] else (v&w)==0
+            signed_le=signed(w)<=signed(v) if op in ['cmpl','cmpq'] else signed(v&w)<=0
             ge=greater=below=None
         elif op in ['imull','imulq']:
             width=8 if op=='imulq' else 4;parts=args.split(', ')
@@ -81,6 +83,9 @@ def evaluate(graph,start,stops,memory,registers,limit=128,capture_registers=Fals
             elif len(parts)==3:a,b,dest=parts;v,w=read(a,width),read(b,width)
             else:raise ValueError('unsupported multiply operands')
             write(dest,v*w,width);equal=None;signed_le=None;ge=greater=below=None
+        elif op=='shlq':
+            a,b=args.split(', ');write(b,read(b,8)<<(read(a,8)&63),8)
+            equal=None;signed_le=None;ge=greater=below=None
         elif op=='sarq':
             a,b=args.split(', ');v=read(b,8);v=v if v<2**63 else v-2**64
             write(b,v>>(read(a,8)&63),8);equal=None;signed_le=None;ge=greater=below=None
@@ -94,16 +99,20 @@ def evaluate(graph,start,stops,memory,registers,limit=128,capture_registers=Fals
             a,b=args.split(', ')
             if a!=b:raise ValueError('unsupported SIMD xor')
             write(b,0,4)  # Does not change integer flags.
+        elif op in ['addss','subss','mulss']:
+            a,b=args.split(', ');v,w=number(read(a,4)),number(read(b,4))
+            write(b,bits(w+v if op=='addss' else w-v if op=='subss' else w*v),4)
         elif op=='comiss':
             a,b=args.split(', ');v,w=number(read(a,4)),number(read(b,4));equal=v==w;ge=w>=v;greater=w>v;below=w<v;signed_le=None
-        elif op=='sete':
-            if equal is None:raise ValueError('unmodeled zero flag')
-            write(args,int(equal),1)
-        elif op in ['je','jne','jae','ja','jb','jle','jmp']:
+        elif op in ['sete','setbe']:
+            if op=='sete' and equal is None:raise ValueError('unmodeled zero flag')
+            if op=='setbe' and below is None:raise ValueError('unmodeled unsigned flags')
+            write(args,int(equal if op=='sete' else below or equal),1)
+        elif op in ['je','jne','jae','ja','jb','jbe','jle','jmp']:
             if op in ['je','jne'] and equal is None:raise ValueError('unmodeled zero flag')
             if op=='jle' and signed_le is None:raise ValueError('unmodeled signed flags')
-            if op in ['jae','ja','jb'] and ge is None:raise ValueError('unmodeled unsigned flags')
-            taken=op=='jmp' or (equal if op=='je' else not equal if op=='jne' else signed_le if op=='jle' else greater if op=='ja' else below if op=='jb' else ge)
+            if op in ['jae','ja','jb','jbe'] and ge is None:raise ValueError('unmodeled unsigned flags')
+            taken=op=='jmp' or (equal if op=='je' else not equal if op=='jne' else signed_le if op=='jle' else greater if op=='ja' else below if op=='jb' else below or equal if op=='jbe' else ge)
             if taken:pc=int(args.split()[0],16)
         else:raise ValueError('unsupported composition opcode '+op)
     result={'trace':trace,'stop':pc,'memory':m,'writes':writes}

scripts/analyze_world_links.py

@@ -0,0 +1,121 @@
+#!/usr/bin/env python3
+"""Synthetic local slices/compositions for world-sector creation and entity lists."""
+import gzip
+import itertools
+import json
+from pathlib import Path
+import re
+from analyze_area_queries import site,run
+from analyze_chat_crosshair import bits,number
+from analyze_chat_paths import native_string
+from analyze_collision_composition import get_word,put_word
+from analyze_engine_control import control_flow
+from analyze_frame_timing import read_address
+from analyze_native import elf
+from analyze_qvm import write_json
+from analyze_vm_extensions import native_functions,verified
+
+
+def vector(m,p):return [number(get_word(m,p+4*i)) for i in range(3)]
+
+
+def build_tree(g,s,half_address,half_bits,minimum,maximum):
+    """Compose stopped constructor slices, explicitly substituting child returns."""
+    m={};put_word(m,s['counter'],0);put_word(m,half_address,half_bits)
+    for i in range(2048):m[s['root']+i]=0  # Supplied clear state, not executed memset.
+    nodes=[]
+    def build(depth,low,high):
+        stack=0x40000+depth*0x1000;lo=stack+0x800;hi=stack+0x820
+        for i in range(3):put_word(m,lo+4*i,bits(low[i]));put_word(m,hi+4*i,bits(high[i]))
+        t=run(g,s['allocation'],[s['first_call'],s['leaf_cleanup']],m,
+              {'%rsp':stack,'%rdi':depth,'%rsi':lo,'%r9':hi});m.update(t['memory']);regs=t['registers'];node=regs['rbp'];traces=[t['trace']]
+        if t['stop']==s['first_call']:
+            child0=build(regs['rdi'],vector(m,regs['rsi']),vector(m,regs['rdx']))
+            regs['rax']=child0
+            t=run(g,s['after_first_call'],[s['second_call']],m,regs);m.update(t['memory']);regs=t['registers'];traces.append(t['trace'])
+            child1=build(regs['rdi'],vector(m,regs['rsi']),vector(m,regs['rdx']))
+            regs['rax']=child1
+            t=run(g,s['after_second_call'],[s['internal_cleanup']],m,regs);m.update(t['memory']);traces.append(t['trace'])
+        nodes.append({'address':node,'depth':depth,'input_min':low,'input_max':high,'axis_bits':get_word(m,node),
+                      'split':number(get_word(m,node+4)),'children':[get_word(m,node+8,8),get_word(m,node+16,8)],'head':get_word(m,node+24,8),'traces':traces})
+        return node
+    root=build(0,minimum,maximum)
+    return m,{'root':root,'allocated_count':get_word(m,s['counter']),'nodes':sorted(nodes,key=lambda x:x['address'])}
+
+
+def main():
+    out={'native':{},'limitations':'Synthetic bounded native slices and explicit child-return compositions. '
+         'No original process/memset/model-bound query/entity lookup/recursive native callee executes. '
+         'Constructor starts after prologue with supplied zeroed storage and finite bounds; link probes '
+         'start after metadata/bounds production. Ownership/cycles/ABI lifetimes, exceptional floats, '
+         'full map/query/runtime and other-platform conformance remain open.'}
+    for label in ['linux--Quake3-UrT-Ded.x86_64','linux--Quake3-UrT.x86_64']:
+        r=json.loads(Path(f'docs/research/native/{label}.json').read_text());data=verified((Path('.cache/UrbanTerror43')/r['source']['path']).read_bytes(),r['source']['sha256'])
+        a=r['artifacts']['disassembly.txt'];fs=native_functions(verified(gzip.open(a['path'],'rb').read(),a['uncompressed_sha256']).decode())
+        graphs={n:control_flow(fs[n]) for n in ['SV_ClearWorld','SV_CreateworldSector','SV_LinkEntity','SV_UnlinkEntity']}
+        g=graphs['SV_CreateworldSector'];rows=g['instructions'];calls=[i for i,x in enumerate(rows) if '<SV_CreateworldSector>' in x['operands'] and x['opcode']=='callq']
+        if len(calls)!=2:raise ValueError('constructor calls changed')
+        alloc=next(x for x in rows if '<sv_numworldSectors>' in x['operands']);counter=int(re.search(r'#\s*0x([0-9a-f]+)',alloc['operands'])[1],16)
+        root=int(site_operand(g,'leaq','%rbp').split('(')[0],0)
+        half=next(x for x in rows if x['opcode']=='mulss');half_address=int(re.search(r'#\s*0x([0-9a-f]+)',half['operands'])[1],16)
+        meta,_=elf(data);raw,off=read_address(data,meta['sections'],half_address,4);half_bits=int.from_bytes(raw,'little')
+        s={'allocation':alloc['address'],'counter':counter,'root':root,'first_call':rows[calls[0]]['address'],
+           'after_first_call':rows[calls[0]+1]['address'],'second_call':rows[calls[1]]['address'],
+           'after_second_call':rows[calls[1]+1]['address'],'internal_cleanup':rows[calls[1]+2]['address'],
+           'leaf_cleanup':next(x['address'] for x in reversed(rows) if x['opcode']=='addq' and x['operands']=='$0x48, %rsp')}
+        tree_inputs=[([0,0,0],[16,8,4]),([-8,-8,-100],[8,8,100]),([0,0,0],[0,0,0])]
+        trees=[];tree_memory=None
+        for low,high in tree_inputs:
+            m,t=build_tree(g,s,half_address,half_bits,low,high);trees.append(t)
+            if tree_memory is None:tree_memory=m
+        local=[]
+        for depth,count in itertools.product([0,3,4,5],[0,30,63,64]):
+            stack,lo,hi=0x40000,0x40800,0x40820;m={};put_word(m,counter,count);put_word(m,half_address,half_bits)
+            for i in range(3):put_word(m,lo+4*i,bits(0));put_word(m,hi+4*i,bits([16,8,4][i]))
+            t=run(g,s['allocation'],[s['first_call'],s['leaf_cleanup']],m,{'%rsp':stack,'%rdi':depth,'%rsi':lo,'%r9':hi})
+            node=t['registers']['rbp'];local.append({'depth':depth,'count_before':count,'count_after':get_word(t['memory'],counter),
+                'node':node,'axis_bits':get_word(t['memory'],node),'leaf':t['stop']==s['leaf_cleanup'],'trace':t['trace']})
+        g=graphs['SV_LinkEntity'];start=site(g,'addl','$0x1, 0x1a4(%rbx)');stop=site(g,'addq','$0x228, %rsp')
+        obj,record=0x10000,0x20000;links=[]
+        intervals=[(-1,0),(0,0),(3,4),(7,8),(8,8),(8,9),(9,10),(16,17)]
+        for xbounds,ybounds in itertools.product(intervals,repeat=2):
+            m=dict(tree_memory);put_word(m,obj+0x1a4,41);put_word(m,obj+0x1a0,0)
+            for node in trees[0]['nodes']:
+                head=0x90000+312*((node['address']-root)//32);put_word(m,node['address']+24,head,8)
+            for i,pair in enumerate([xbounds,ybounds,(0,1)]):put_word(m,obj+0x1d0+4*i,bits(pair[0]));put_word(m,obj+0x1dc+4*i,bits(pair[1]))
+            t=run(g,start,[stop],m,{'%rbx':obj,'%r12':record});owner=get_word(t['memory'],record,8)
+            links.append({'min':[xbounds[0],ybounds[0],0],'max':[xbounds[1],ybounds[1],1],
+                'owner_node':owner,'next_record':get_word(t['memory'],record+8,8),'new_node_head':get_word(t['memory'],owner+24,8),
+                'linked_word':get_word(t['memory'],obj+0x1a0),'link_count':get_word(t['memory'],obj+0x1a4),'trace':t['trace'],'writes':t['writes']})
+        g=graphs['SV_UnlinkEntity'];start=site(g,'movl','$0x0, 0x1a0(%rbx)');stops=[x['address'] for x in g['instructions'] if x['opcode']=='popq' and x['operands']=='%rbx']
+        unlink=[];node=0x60000;records=[0x70000+i*312 for i in range(4)]
+        for name,order,target,owner in [('no_owner',[0,1,2],1,0),('head',[0,1,2],0,node),('middle',[0,1,2],1,node),('tail',[0,1,2],2,node),('only',[0],0,node),('missing',[0,1,2],3,node),('empty',[],3,node)]:
+            m={};put_word(m,obj+0x1a0,1);put_word(m,node+24,records[order[0]] if order else 0,8)
+            for i,p in enumerate(records):put_word(m,p,node,8);put_word(m,p+8,records[i+1] if i<3 else 0,8)
+            if order:put_word(m,records[order[-1]]+8,0,8)
+            put_word(m,records[target],owner,8)
+            t=run(g,start,stops,m,{'%rbx':obj,'%rax':records[target]})
+            cursor=get_word(t['memory'],node+24,8);remaining=[]
+            while cursor:
+                if cursor in remaining or len(remaining)>4:raise ValueError('fixture cycle')
+                remaining.append(cursor);cursor=get_word(t['memory'],cursor+8,8)
+            unlink.append({'case':name,'initial_indices':order,'target_index':target,'initial_owner':owner,
+                'remaining_indices':[records.index(p) for p in remaining],'linked_word':get_word(t['memory'],obj+0x1a0),
+                'owner_after':get_word(t['memory'],records[target],8),'next_after':get_word(t['memory'],records[target]+8,8),
+                'warning_path':t['stop']==stops[1],'trace':t['trace'],'writes':t['writes']})
+        warn=next(x for x in g['instructions'] if x['opcode']=='movl' and x['operands'].split('#')[0].strip().endswith('%edi') and '$0x' in x['operands']);warnaddr=int(warn['operands'].split(',')[0][1:],0)
+        out['native'][label]={'source':r['source'],'disassembly_sha256':a['uncompressed_sha256'],'graphs':graphs,'constructor_sites':s,
+            'half_constant':{'address':half_address,'file_offset':off,'bits':half_bits,'value':number(half_bits)},
+            'constructed_compositions':trees,'constructor_local_traces':local,'link_traces':links,'unlink_traces':unlink,
+            'unlink_warning':native_string(data,meta['sections'],warnaddr)}
+    write_json(Path('docs/research/world-links.json'),out)
+    print('Recovered 6 tree compositions, 32 local constructor, 128 link and 14 unlink traces')
+
+
+def site_operand(g,op,destination):
+    found=[x['operands'].split('#')[0].strip().split(', ')[0] for x in g['instructions'] if x['opcode']==op and x['operands'].split('#')[0].strip().endswith(', '+destination)]
+    if len(found)!=1:raise ValueError('ambiguous destination')
+    return found[0]
+
+
+if __name__=='__main__':main()

scripts/test_world_links.py

@@ -0,0 +1,126 @@
+#!/usr/bin/env python3
+"""Independent tree/list oracles and bounded evaluator regression checks."""
+import json
+from pathlib import Path
+import unittest
+from analyze_area_queries import site
+from analyze_chat_crosshair import bits,number
+from analyze_collision_composition import evaluate,put_word
+from test_collision_composition import graph
+
+
+class WorldLinkTests(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):cls.r=json.loads(Path('docs/research/world-links.json').read_text())
+
+    def test_constructor_preorder_depth_axes_midpoints_and_children(self):
+        for n in self.r['native'].values():
+            self.assertEqual(n['half_constant']['value'],.5)
+            for t in n['constructed_compositions']:
+                expected=[]
+                def build(depth,low,high):
+                    index=len(expected);row={'address':t['root']+32*index,'depth':depth,'input_min':low,'input_max':high,'axis_bits':0xffffffff,'split':0.0,'children':[0,0],'head':0};expected.append(row)
+                    if depth<4:
+                        axis=0 if high[0]-low[0]>high[1]-low[1] else 1;split=(high[axis]+low[axis])*.5
+                        row['axis_bits']=axis;row['split']=split
+                        above=list(low);above[axis]=split;below=list(high);below[axis]=split
+                        row['children']=[build(depth+1,above,list(high)),build(depth+1,list(low),below)]
+                    return row['address']
+                build(0,t['nodes'][0]['input_min'],t['nodes'][0]['input_max'])
+                self.assertEqual(t['allocated_count'],31)
+                self.assertEqual([{k:v for k,v in x.items() if k!='traces'} for x in t['nodes']],expected)
+                self.assertEqual(sum(x['axis_bits']==0xffffffff for x in t['nodes']),16)
+
+    def test_local_depth_equality_and_allocation_without_capacity_guard(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['constructor_local_traces']),16)
+            root=n['constructor_sites']['root']
+            self.assertEqual(n['constructor_sites']['counter'],root+2048)
+            for x in n['constructor_local_traces']:
+                self.assertEqual(x['leaf'],x['depth']==4)
+                self.assertEqual(x['count_after'],(0xffffffff if x['depth']==4 else 0) if x['count_before']==64 else x['count_before']+1)
+                self.assertEqual(x['node'],root+32*x['count_before'])
+                self.assertEqual(x['axis_bits'],0xffffffff if x['depth']==4 else 0)
+            clear=n['graphs']['SV_ClearWorld']['instructions']
+            self.assertTrue(any(x['operands'].split('#')[0].strip()=='$0x800, %edx' for x in clear))
+            self.assertTrue(any('<sv_numworldSectors>' in x['operands'] and x['operands'].startswith('$0x0,') for x in clear))
+
+    def test_strict_placement_and_prepend_write_contract(self):
+        for n in self.r['native'].values():
+            nodes={x['address']:x for x in n['constructed_compositions'][0]['nodes']};root=n['constructor_sites']['root']
+            self.assertEqual(len(n['link_traces']),64)
+            for x in n['link_traces']:
+                node=nodes[root]
+                while node['axis_bits']!=0xffffffff:
+                    axis=node['axis_bits'];split=node['split']
+                    if x['min'][axis]>split:node=nodes[node['children'][0]]
+                    elif x['max'][axis]<split:node=nodes[node['children'][1]]
+                    else:break
+                self.assertEqual(x['owner_node'],node['address'])
+                self.assertEqual(x['next_record'],0x90000+312*((node['address']-root)//32))
+                self.assertEqual(x['new_node_head'],0x20000)
+                self.assertEqual(x['linked_word'],1);self.assertEqual(x['link_count'],42)
+                self.assertEqual(x['writes'],[{'address':0x101a4,'bytes':4},{'address':0x20000,'bytes':8},
+                    {'address':0x20008,'bytes':8},{'address':node['address']+24,'bytes':8},{'address':0x101a0,'bytes':4}])
+            touch=next(x for x in n['link_traces'] if x['min']==[8,8,0] and x['max']==[8,8,1])
+            self.assertEqual(touch['owner_node'],root)
+
+    def test_unlink_positions_missing_owner_and_retained_next(self):
+        expected={'no_owner':[0,1,2],'head':[1,2],'middle':[0,2],'tail':[0,1],'only':[],'missing':[0,1,2],'empty':[]}
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['unlink_traces']),7)
+            for x in n['unlink_traces']:
+                self.assertEqual(x['remaining_indices'],expected[x['case']])
+                self.assertEqual(x['linked_word'],0);self.assertEqual(x['owner_after'],0)
+                self.assertEqual(x['warning_path'],x['case'] in ['missing','empty'])
+                target=x['target_index'];oldnext=0 if target==3 or (x['initial_indices'] and target==x['initial_indices'][-1]) else 0x70000+312*(target+1)
+                self.assertEqual(x['next_after'],oldnext)
+                self.assertNotIn({'address':0x70000+312*target+8,'bytes':8},x['writes'])
+            self.assertIn('SV_UnlinkEntity',n['unlink_warning'])
+
+    def test_link_history_changes_query_head_order(self):
+        for n in self.r['native'].values():
+            g=n['graphs']['SV_LinkEntity'];start=site(g,'movq','%rdx, (%r12)');stop=site(g,'addq','$0x228, %rsp')
+            node,obj=0x60000,0x10000;m={};put_word(m,node+24,0,8)
+            records=[0x20000+i*312 for i in range(3)]
+            for record in records:
+                t=evaluate(g,start,[stop],m,{'%rdx':node,'%r12':record,'%rbx':obj});m=t['memory']
+            from analyze_collision_composition import get_word
+            seen=[];cursor=get_word(m,node+24,8)
+            while cursor:seen.append(cursor);cursor=get_word(m,cursor+8,8)
+            self.assertEqual(seen,list(reversed(records)))
+
+    def test_paired_engines_agree_after_address_normalization(self):
+        a,b=self.r['native'].values()
+        for key in ['constructor_local_traces','link_traces','unlink_traces']:
+            def normalize(n):
+                root=n['constructor_sites']['root'];out=[]
+                for x in n[key]:
+                    y={k:v for k,v in x.items() if k not in ['trace','writes']}
+                    for k in ['node','owner_node']:
+                        if k in y:y[k]-=root
+                    out.append(y)
+                return out
+            self.assertEqual(normalize(a),normalize(b))
+
+    def test_finite_float_rounding_signed_conversion_and_flag_preservation(self):
+        g=graph(('cmpl','$0x1, %edi'),('movss','%xmm0, %xmm1'),('addss','%xmm2, %xmm1'),('sete','%al'),('cltq',''))
+        t=evaluate(g,1,[6],{}, {'%rdi':1,'%rax':0xffffff00,'%xmm0':bits(2**24),'%xmm2':bits(1)},capture_registers=True)
+        self.assertEqual(number(t['registers']['xmm1']),2**24)
+        self.assertEqual(t['registers']['rax'],0xffffffffffffff01)
+        for value in [-1,0,1]:
+            g=graph(('comiss','%xmm0, %xmm1'),('setbe','%al'))
+            t=evaluate(g,1,[3],{}, {'%rax':0,'%xmm0':bits(0),'%xmm1':bits(value)},capture_registers=True)
+            self.assertEqual(t['registers']['rax'],int(value<=0))
+        with self.assertRaises(ValueError):evaluate(graph(('setbe','%al')),1,[2],{}, {})
+        with self.assertRaises(ValueError):evaluate(graph(('mulss','%xmm0, %xmm1')),1,[2],{}, {'%xmm0':0x7f7fffff,'%xmm1':0x7f7fffff})
+
+    def test_cyclic_unlink_fixture_stops_at_step_bound(self):
+        for n in self.r['native'].values():
+            g=n['graphs']['SV_UnlinkEntity'];start=site(g,'movl','$0x0, 0x1a0(%rbx)');stops=[x['address'] for x in g['instructions'] if x['opcode']=='popq']
+            m={};obj,target,node,head=0x10000,0x20000,0x60000,0x70000
+            put_word(m,target,node,8);put_word(m,node+24,head,8);put_word(m,head+8,head,8)
+            with self.assertRaises(ValueError):evaluate(g,start,stops,m,{'%rbx':obj,'%rax':target},limit=64)
+
+
+if __name__=='__main__':unittest.main()