Recover leaf references and bounded area portal state

docs/ASSET_PIPELINE_TODO.md

@@ -204,3 +204,5 @@ Names and output formats below are proposals to refine after the relevant resear
 Native collision follow-up: [BSP planes](research/BSP_PLANES.md) records type/sign fields and 16-byte native node addressing. Connect these to file-lump conversion only after loading/type/sign producers and valid counts are recovered (RE-320); native in-memory offsets are not a disk-format specification.
 
 Native loading follow-up: [BSP payload loading](research/BSP_LOADING.md) connects plane/node/leaf descriptors 2/3/4 to native conversion. Preserve original disk fields and every map variant; native collision projection omits fields that rendering or other consumers may need. Complete range/reference validation and actual-map conversion remain RE-324 work.
+
+Reference follow-up: [leaf references and area state](research/BSP_REFERENCES.md) connects native leaf words to brush/surface index arrays and records standalone copy behavior. Validate every source reference and prove inline-loader equivalence before implementing conversion; retain disk bounds and every variant (RE-328).

docs/FINDINGS.md

@@ -342,3 +342,9 @@ Bounded qagame paths recover bomb-plant/inactivity correction requests, direct f
 **F-101 — Observed node reconstruction:** 36-byte disk records become 16-byte native nodes containing plane_base + 20 * signed index and unchanged child words. Sampled conversion performs no local plane-index guard and omits the six trailing disk words. Complete file validation and native ownership/runtime behavior remain open. [LOAD-002](research/BSP_LOADING.md).
 
 **F-102 — Observed leaf projection/state:** 48-byte disk records become 24-byte native leaves in word order 0,1,10,11,8,9. Zero-seeded running cluster/area words track tested maximum nonnegative IDs plus one; leaf storage reserves two spare records and later requests area-related storage. Reference semantics, initialization, count overflow and runtime conformance remain open. [LOAD-003](research/BSP_LOADING.md).
+
+**F-103 — Observed leaf reference consumers:** reference helpers preserve ordered four-byte IDs, reserving one spare brush word but no spare surface word. Bounded leaf address slices select 56-byte native brush records and surface pointer-table entries from signed IDs. Inline map loading, valid ranges and full collision bodies remain open. [REF-001](research/BSP_REFERENCES.md).
+
+**F-104 — Observed connectivity priority:** nonzero cm_noAreas returns true before index checks; either negative index returns false before upper bounds; valid areas compare labels in eight-byte records. Supplied labels establish local outcomes, not proven flood generation or runtime connectivity. [REF-002](research/BSP_REFERENCES.md).
+
+**F-105 — Observed portal references:** nonzero open increments both directed counters and zero decrements both; self-pairs mutate twice. Close-underflow is checked after writes, and successful updates increment the epoch before requesting flooding. Full flood/lifecycle/query composition and runtime safety remain open. [REF-003](research/BSP_REFERENCES.md).

docs/PROVISIONAL_TODO.md

@@ -516,3 +516,5 @@ Before marking a broad subsystem complete, reconcile it with the canonical check
 Research progress: [BSP plane boundaries and bounded node traversal](research/BSP_PLANES.md) completes bounded RE-317–319. Broad geometry/runtime/port milestones remain open; see RE-320 for full traversal and loading work.
 
 Research progress: [native BSP payload loading](research/BSP_LOADING.md) completes bounded RE-321–323. Complete asset conversion and geometry milestones remain open; RE-324 tracks actual variant audits, validation, reference ownership and runtime fixtures.
+
+Research progress: [leaf references and area portal state](research/BSP_REFERENCES.md) completes bounded RE-325–327. Broad geometry/networking/runtime milestones remain open; RE-328 tracks complete flooding, reference ownership and snapshot consumers.

docs/RESEARCH_LEDGER.md

@@ -185,3 +185,7 @@ Completed bounded RE-317–319 with `analyze_bsp_planes.py`: four hash-verified
 ## 2026-10-11 — Native BSP payload loading
 
 Completed bounded RE-321–323 with `analyze_bsp_loading.py`: eight verified graphs, six header descriptor-routing slices, 72 supplied-allocation plane/node/leaf payload conversions and corresponding allocation-size slices. LOAD-001–003/F-100–102 distinguish disk/native records, +1 type priority from unit-normal validation, signed plane-pointer construction from valid indices, and projected leaf words/maxima from complete ownership. Eight independent tests cover field copies/type/sign/padding, signed pointers/children, leaf projection/maxima, allocation/routing arithmetic, exact byte writes, paired outputs and restricted byte/conditional/signed-branch evaluator semantics. The full suite passes 254 tests; current and preceding reports regenerate byte-identically, storage and documentation checks pass. RE-320 remains open; RE-324 tracks complete validation/reset/allocator/spare-record/reference/portal contracts, actual BSP variants, full traversal and runtime/platform behavior. No original process, allocator or native callee executes.
+
+## 2026-10-11 — Leaf reference arrays and area portal state
+
+Completed bounded RE-325–327 with `analyze_bsp_references.py`: sixteen verified native graphs, 48 reference-copy and 48 consumer-address slices, 450 connectivity and 800 portal probes. REF-001–003/F-103–105 distinguish standalone copying from inline map loading, native signed address calculations from valid indices, supplied label equality from flood generation, and directed reference counters from Boolean portal state. Portal frame omission and all stopped callee boundaries are explicit. Eight independent tests verify copies/allocation/canaries, signed addressing, connectivity guard priority, full sampled matrix/write/error/epoch/request outcomes, self-pairs, prior-loader slots, paired outputs and low-byte aliases. The full suite passes 262 tests; new and preceding reports regenerate byte-identically, storage and documentation checks pass. RE-324 remains open; RE-328 tracks inline/range/layout/geometry/flood/lifecycle/snapshot and runtime/platform coverage. No original process, allocator or native callee executes.

docs/REVERSE_ENGINEERING_TODO.md

@@ -387,3 +387,8 @@ Purpose: turn the supplied installation into an evidence-backed specification of
 - [x] RE-322 Recover disk-to-native node reconstruction, signed stride-20 plane pointers, unchanged children and allocation sizes ([LOAD-002](research/BSP_LOADING.md)).
 - [x] RE-323 Recover leaf word projection, running cluster/area maxima and bounded leaf/area/portal allocation requests ([LOAD-003](research/BSP_LOADING.md)).
 - [ ] RE-324 Recover complete header/lump/error/endian/count validation, allocator/reset/spare-record initialization, brush/surface references and area/portal ownership; audit actual BSP variants and compose full loading/traversal/callback/runtime/all-platform fixtures.
+
+- [x] RE-325 Recover bounded brush/surface reference copies, allocation differences and signed leaf-consumer address relationships ([REF-001](research/BSP_REFERENCES.md)).
+- [x] RE-326 Recover area connectivity bypass/index-guard priority and eight-byte record label comparisons ([REF-002](research/BSP_REFERENCES.md)).
+- [x] RE-327 Recover paired portal counters, self-pair writes, close-underflow/error boundaries and epoch/first-flood requests ([REF-003](research/BSP_REFERENCES.md)).
+- [ ] RE-328 Recover inline reference-loader equivalence and actual map index ranges, complete brush/surface layouts/stamps/geometry, all unrolled/recursive flooding and epoch/revisit invariants, portal writers/reset/serialization, area-bit/snapshot consumers and original-runtime/all-platform conformance.

docs/research/BSP_LOADING.md

@@ -56,3 +56,5 @@ Twenty-four paired leaf payload probes cover odd/even peel and pair-loop paths.
 Eight independent tests cover copy/type/sign/padding behavior, signed node pointer arithmetic and child bits, leaf word reordering/maxima, allocation arithmetic, header descriptor routing, exact per-record byte writes, paired outcomes and restricted evaluator byte/conditional/signed-branch semantics. No probe writes into a spare record. Larger instruction traces retain counts/hashes; small traces retain addresses.
 
 RE-324: validate complete header/lump bounds, alignment/division/error paths, endian handling and count overflow; recover allocator/reset/zeroing and spare-record initialization; recover leaf brush/surface references, area/portal structures and lifetime; audit every actual BSP variant and compose loading with full recursive query/callback paths; obtain original-runtime and other-platform differential fixtures. Rendering geometry and Rust/Bevy/Solari conversion remain separate open work.
+
+Follow-up: [leaf references and area portal state](BSP_REFERENCES.md) connects projected reference words to consumers and bounds local connectivity/portal behavior. Complete actual loading/validation/ownership and recursive flooding remain open under RE-324/RE-328.

docs/research/BSP_REFERENCES.md

@@ -0,0 +1,54 @@
+# Leaf references, connectivity and portal counters
+
+Reproduce with `python3 scripts/analyze_bsp_references.py`; validate with `python3 scripts/test_bsp_references.py`. Evidence: [bsp-references.json](bsp-references.json), with eight hash-verified native graphs per Linux ELF64 engine. This extends [BSP loading](BSP_LOADING.md) and [leaf metadata](LEAF_METADATA.md).
+
+These are bounded synthetic instructions with supplied allocation returns, arrays, pointers and area labels. Original processes, allocations, errors, geometry callees and flood callees never execute. Portal PUSH instructions are explicitly replaced with NOPs at the same addresses; execution stops before POP/return, error calls or the first flood call. Connectivity starts after frame setup and stops before stack restoration. The retained flooding graphs are static evidence, not executed compositions.
+
+## REF-001 — Reference arrays and leaf consumers (RE-325)
+
+Forty-eight paired reference-loader probes cover counts 0–9, 16 and 17, including every copy remainder and eight-word loop paths. CMod_LoadLeafBrushes requests `4 * (count + 1)` bytes; CMod_LoadLeafSurfaces requests `4 * count`. Both publish supplied allocation pointers and counts, preserve all original words in order, and omit payload copies for zero count. Repeated IDs and -1 words are copied unchanged; these deliberately supplied IDs do not establish legal map values. The spare brush word is untouched under the supplied sentinel initialization. Alignment/error paths are retained but not evaluated.
+
+No direct calls to either standalone helper appear in the retained CM_LoadMap graph. Actual inline-loading equivalence and descriptor routing for these arrays remain open; standalone helper behavior is not evidence of complete original map loading.
+
+The leaf consumer slices connect the prior disk/native projection to index arithmetic:
+
+| Native leaf offsets | Observed consumer relationship |
+| --- | --- |
+| +8 first, +12 count | brush-reference array; signed referenced ID selects `brush_base + 56 * ID` |
+| +16 first, +20 count | surface-reference array; signed referenced ID selects an eight-byte pointer-table entry |
+
+Forty-eight paired address slices vary first index, ordinal and supplied IDs -1/0/7. Brush slices stop before the record stamp comparison. Surface slices stop before the selected pointer's NULL test. Both read `reference[first + ordinal]`; no local index bounds checks occur inside these selected arithmetic slices. Negative-ID fixtures use explicitly supplied memory and are not valid-map or safety demonstrations.
+
+The full retained consumer graphs show brush iteration before curve/surface iteration, reference-count gates, stamp checks, content tests and geometry calls. Those bodies/order/state effects are not composed in this report. The 56-byte brush record stride is native ELF64 layout, not a brush disk-record size or Bevy asset layout.
+
+## REF-002 — Area connectivity predicate (RE-326)
+
+Four hundred fifty paired probes cover supplied area counts 0/1/3, both indices -1 through 3, and cm_noAreas raw words 0/1/7. The observed priority is:
+
+1. Nonzero cm_noAreas value returns true before index validation.
+2. Either negative area returns false, before either upper bound is checked.
+3. A nonnegative area >= area count requests Com_Error with `area >= cm.numAreas`.
+4. Valid areas compare the first word of their eight-byte native records for equality.
+
+Supplied labels `[1,1,2]` establish connected versus disconnected outcomes. The record's second word is not used by this predicate. The base/count slots match the preceding leaf loader's area allocation/count state. This is label comparison, not a portal walk, line-of-sight test or independently verified connectivity generation. Label initialization/refresh must be established before applying this rule to real snapshots.
+
+## REF-003 — Portal mutation and reflood boundary (RE-327)
+
+Eight hundred paired probes use three areas, indices -1 through 3, open words 0/1/7/-1 and initial selected counter values -1/0/1/2. The matrix contains four-byte counters indexed `area1 * count + area2`; the reverse entry is addressed separately.
+
+- Either negative index returns without mutation, even if the other index exceeds the upper bound.
+- Nonnegative out-of-range indices request `CM_ChangeAreaPortalState: bad area number` before mutation.
+- Every tested nonzero open word increments both directed entries; zero decrements both.
+- Self-pairs write the same cell twice. Starting at one, open produces three and close produces -1.
+- Closing checks the reversed entry after both decrements. Negative results request `CM_AdjustAreaPortalState: negative reference count`; both writes have already occurred at the stopped error boundary.
+- Successful updates increment the flood epoch and begin a label rebuild. With epoch 40 and all area stamp words explicitly seeded to zero, the first request is CM_FloodArea_r(area 0, label 1) at epoch 41.
+
+The tests compare the whole matrix, write offsets/order, guard priority, stopped error text, epoch and requested arguments. Deliberately negative initial counters characterize local paths and do not establish reachable runtime states. Error handling itself is not executed; persistence after Com_Error is not claimed.
+
+The retained CM_FloodArea_r graph uses record +0 for the label and +4 for the epoch stamp; positive matrix entries select further visits. CM_FloodAreaConnections similarly increments the epoch and scans area indices in ascending order. Complete unrolled/recursive flooding, its revisit-label consistency errors, asymmetric matrices, stale/prestamped records, count/epoch wraparound and full update-to-query composition remain open.
+
+## Validation and next work
+
+Eight independent tests cover copy/allocation/canaries, signed consumer addressing, connectivity guard priority, complete sampled matrix mutations, self-pairs/nonzero open flags, prior-loader slot correspondence, paired outputs/trace lengths and low-byte register preservation. Small slices retain instruction addresses; larger collections retain trace lengths and hashes. The report does not copy any original asset into assets/.
+
+RE-328: prove inline/reference-loading equivalence and valid reference ranges across all BSP variants; recover complete brush/surface layouts, stamps and geometry; compose full flood generation/revisit/epoch behavior and update-to-connectivity/area-bit/snapshot consumers; connect server/QVM portal writers and lifetime/reset/serialization; obtain original-runtime and other-platform fixtures. Rust/Bevy/Solari collision and rendering implementation remain open.

docs/research/README.md

@@ -117,3 +117,5 @@ The extension collector adds five focused tests for native tail calls, command-r
 [Native BSP planes and bounded node traversal](BSP_PLANES.md) adds RE-317–319: axial/general equality rules, verified eight-case sign dispatch and one-node child order/addressing. Reproduce with `python3 scripts/analyze_bsp_planes.py`; check with `python3 scripts/test_bsp_planes.py`. Full traversal/loading/callback/float/runtime/platform contracts remain open.
 
 [Native BSP plane, node and leaf loading](BSP_LOADING.md) adds RE-321–323: descriptor routing, bounded payload conversion, derived plane fields, native node pointers, leaf projection/maxima and allocation requests. Reproduce with `python3 scripts/analyze_bsp_loading.py`; check with `python3 scripts/test_bsp_loading.py`. Complete validation/loading/ownership/recursive-query/runtime/platform contracts remain open.
+
+[Leaf references, connectivity and portal counters](BSP_REFERENCES.md) adds RE-325–327: bounded reference copying/consumer addressing, connectivity guard priority and portal mutation/epoch/flood-request boundaries. Reproduce with `python3 scripts/analyze_bsp_references.py`; check with `python3 scripts/test_bsp_references.py`. Full inline loading/flooding/lifecycle/geometry/runtime/platform contracts remain open.

scripts/analyze_bsp_references.py

@@ -0,0 +1,112 @@
+#!/usr/bin/env python3
+"""Bounded reference copies/addressing, area connectivity and portal mutations."""
+import gzip
+import hashlib
+import itertools
+import json
+from pathlib import Path
+import re
+from analyze_area_queries import site
+from analyze_chat_paths import native_string
+from analyze_collision_composition import evaluate,get_word,put_word
+from analyze_engine_control import control_flow
+from analyze_native import elf
+from analyze_qvm import write_json
+from analyze_vm_extensions import native_functions,verified
+
+
+def operand(x):return x['operands'].split('#')[0].strip()
+def global_address(x):return int(re.search(r'#\s*0x([0-9a-f]+)',x['operands'])[1],16)
+def trace_fields(t,full=False):return {'trace':t['trace'] if full else None,'trace_length':len(t['trace']),'trace_sha256':hashlib.sha256(json.dumps(t['trace'],separators=(',',':')).encode()).hexdigest()}
+def run(g,start,stops,m,regs):return evaluate(g,start,stops,m,regs,limit=2048,capture_registers=True)
+
+
+def errors(g,data,sections):
+    result=[];rows=g['instructions']
+    for i,x in enumerate(rows):
+        if '<Com_Error>' not in x['operands']:continue
+        fmt=next(y for y in reversed(rows[max(0,i-4):i]) if y['opcode']=='movl' and operand(y).endswith(', %esi'))
+        result.append({'call':x['address'],'text':native_string(data,sections,int(operand(fmt).split(',')[0][1:],0))})
+    return result
+
+
+def main():
+    out={'native':{},'limitations':'Synthetic supplied-allocation copies and stopped instruction slices. '
+         'Original processes, allocations, errors, collision callees and flooding never execute. '
+         'Reference array IDs, pointer validity, caller/inline loader equivalence, reset/zeroing, '
+         'full flood/portal lifecycle, snapshots and original-runtime/all-platform conformance remain open.'}
+    source,output,area_base,matrix,cvar=0x10000,0x30000,0x50000,0x60000,0x70000
+    names=['CMod_LoadLeafBrushes','CMod_LoadLeafSurfaces','CM_TraceThroughLeaf','CM_TestInLeaf','CM_AreasConnected','CM_AdjustAreaPortalState','CM_FloodAreaConnections','CM_FloodArea_r']
+    for label in ['linux--Quake3-UrT-Ded.x86_64','linux--Quake3-UrT.x86_64']:
+        r=json.loads(Path(f'docs/research/native/{label}.json').read_text());data=verified((Path('.cache/UrbanTerror43')/r['source']['path']).read_bytes(),r['source']['sha256'])
+        a=r['artifacts']['disassembly.txt'];fs=native_functions(verified(gzip.open(a['path'],'rb').read(),a['uncompressed_sha256']).decode());metadata,_=elf(data);graphs={n:control_flow(fs[n]) for n in names}
+        references={}
+        for name in names[:2]:
+            g=graphs[name];rows=g['instructions'];alloc=next(i for i,x in enumerate(rows) if '<Hunk_Alloc>' in x['operands']);start=rows[alloc+1]['address'];end=site(g,'addq','$0x8, %rsp')
+            allocation_start=site(g,'leal','0x1(%rbx), %edi') if name.endswith('Brushes') else site(g,'movslq','%ebx, %rdi')
+            count_slot=global_address(next(x for x in rows if x['opcode']=='movl' and '(%rip)' in x['operands'] and operand(x).startswith('%ebx,')))
+            base_slot=global_address(next(x for x in rows if x['opcode']=='movq' and '(%rip)' in x['operands'] and operand(x).startswith('%rax,')))
+            probes=[]
+            for count in list(range(10))+[16,17]:
+                m={};words=[[-1,0,7,7,2][i%5]&0xffffffff for i in range(count)]
+                for i,v in enumerate(words):put_word(m,source+4*i,v)
+                for i in range(4*(count+1)+2):m[output-1+i]=0xa5
+                allocation=run(g,allocation_start,[rows[alloc]['address']],m,{'%rbx':count})
+                t=run(g,start,[end],m,{'%rax':output,'%rbx':count,'%rbp':count,'%r13':0,'%r12':source})
+                probes.append({'count':count,'input_words':words,'output_words':[get_word(t['memory'],output+4*i) for i in range(count)],'allocation_bytes':allocation['registers']['rdi'],
+                    'count_word':get_word(t['memory'],count_slot),'base_word':get_word(t['memory'],base_slot,8),'canaries':[t['memory'][output-1],t['memory'][output+4*count]],'writes':[w for w in t['writes'] if output<=w['address']<output+4*(count+1)],**trace_fields(t,count<=3)})
+            references[name]={'count_slot':count_slot,'base_slot':base_slot,'probes':probes,'errors':errors(g,data,metadata['sections']),
+                'direct_CM_LoadMap_calls':[x['address'] for x in control_flow(fs['CM_LoadMap'])['instructions'] if '<'+name+'>' in x['operands']]}
+        # Connect leaf fields to the first indexed brush and surface address calculations.
+        consumers=[];leaf,brush_base,patch_table=0x80000,0x90000,0xa0000
+        g=graphs['CM_TraceThroughLeaf'];rows=g['instructions'];start=next(x['address'] for x in rows if x['opcode']=='movl' and operand(x)=='%ebp, %eax')
+        # Two MOVL EBP,EAX sites exist (brush and surface); select the first explicitly.
+        brush_stop=next(x['address'] for x in rows if x['opcode']=='cmpl' and operand(x)=='%eax, 0x30(%rbx)')
+        brush_base_slot=global_address(next(x for x in rows if x['opcode']=='addq' and '(%rip)' in x['operands'] and operand(x).endswith(', %rbx')))
+        brush_refs_slot=references['CMod_LoadLeafBrushes']['base_slot']
+        stamp_load=next(x for x in rows if x['opcode']=='movl' and '(%rip)' in x['operands'] and operand(x).endswith(', %eax'))
+        for first,index,value in itertools.product([0,2],[0,1],[-1,0,7]):
+            m={};put_word(m,leaf+8,first);put_word(m,brush_refs_slot,source,8);put_word(m,source+4*(first+index),value);put_word(m,brush_base_slot,brush_base,8);put_word(m,global_address(stamp_load),41)
+            t=run(g,start,[brush_stop],m,{'%rbp':index,'%r12':leaf})
+            consumers.append({'kind':'brush','first':first,'ordinal':index,'supplied_id':value,'pointer':t['registers']['rbx'],**trace_fields(t,True)})
+        g=graphs['CM_TestInLeaf'];rows=g['instructions'];start=site(g,'movl','%ebp, %eax');end=site(g,'testq','%rbx, %rbx')
+        patch_slot=global_address(next(x for x in rows if x['opcode']=='movq' and '(%rip)' in x['operands'] and operand(x).endswith(', %r9')))
+        for first,index,value in itertools.product([0,2],[0,1],[-1,0,7]):
+            m={};put_word(m,leaf+16,first);put_word(m,references['CMod_LoadLeafSurfaces']['base_slot'],source,8);put_word(m,source+4*(first+index),value);put_word(m,patch_slot,patch_table,8);put_word(m,patch_table+8*value,0xb0000+16*(value+1),8)
+            t=run(g,start,[end],m,{'%rbp':index,'%r13':leaf})
+            consumers.append({'kind':'surface','first':first,'ordinal':index,'supplied_id':value,'pointer':t['registers']['rbx'],**trace_fields(t,True)})
+        # Area predicate: start after frame setup, stop before stack restore or error call.
+        g=graphs['CM_AreasConnected'];rows=g['instructions'];start=next(x['address'] for x in rows if x['opcode']=='movq' and '(%rip)' in x['operands']);end=site(g,'movq','0x8(%rsp), %rbx');error=errors(g,data,metadata['sections'])
+        noareas_slot=global_address(next(x for x in rows if x['opcode']=='movq' and '(%rip)' in x['operands']));count_slot=global_address(next(x for x in rows if x['opcode']=='movl' and '(%rip)' in x['operands']));area_slot=global_address(next(x for x in rows if x['opcode']=='movq' and '(%rip)' in x['operands'] and operand(x).endswith(', %rsi')))
+        connected=[]
+        for count,a1,a2,disabled in itertools.product([0,1,3],[-1,0,1,2,3],[-1,0,1,2,3],[0,1,7]):
+            m={};put_word(m,noareas_slot,cvar,8);put_word(m,cvar+48,disabled);put_word(m,count_slot,count);put_word(m,area_slot,area_base,8)
+            for i,v in enumerate([1,1,2]):put_word(m,area_base+8*i,v);put_word(m,area_base+8*i+4,999)
+            t=run(g,start,[end]+[e['call'] for e in error],m,{'%rdi':a1,'%rsi':a2})
+            connected.append({'area_count':count,'area1':a1,'area2':a2,'disabled_word':disabled,'error':t['stop']!=end,'result_word':t['registers']['rdx'] if t['stop']==end else None,**trace_fields(t,count==3 and disabled==0 and a1==0)})
+        # Portal updates: replace only PUSH frame traffic; stop before POP, errors or flooding.
+        g=graphs['CM_AdjustAreaPortalState'];rows=g['instructions'];model=dict(g);model['instructions']=[dict(x) for x in rows];omitted=[]
+        for x in model['instructions']:
+            if x['opcode']=='pushq':omitted.append(x['address']);x['opcode']='nop';x['operands']=''
+        if len(omitted)!=3:raise ValueError('portal frame changed')
+        end=next(x['address'] for x in rows if x['opcode']=='popq');flood=next(x['address'] for x in rows if '<CM_FloodArea_r>' in x['operands']);error2=errors(g,data,metadata['sections']);matrix_slot=global_address(next(x for x in rows if x['opcode']=='addq' and '(%rip)' in x['operands']));epoch_slot=global_address(next(x for x in rows if x['opcode']=='addl' and '(%rip)' in x['operands']))
+        portal=[]
+        for a1,a2,opened,initial in itertools.product([-1,0,1,2,3],[-1,0,1,2,3],[0,1,7,-1],[-1,0,1,2]):
+            count=3;m={};put_word(m,count_slot,count);put_word(m,matrix_slot,matrix,8);put_word(m,area_slot,area_base,8);put_word(m,epoch_slot,40)
+            for i in range(9):put_word(m,matrix+4*i,0)
+            for i in range(3):put_word(m,area_base+8*i,0);put_word(m,area_base+8*i+4,0)
+            if 0<=a1<count and 0<=a2<count:
+                for i in [a1*count+a2,a2*count+a1]:put_word(m,matrix+4*i,initial)
+            t=run(model,model['entry'],[end,flood]+[e['call'] for e in error2],m,{'%rdi':a1,'%rsi':a2,'%rdx':opened})
+            outcome='return' if t['stop']==end else 'flood_request' if t['stop']==flood else 'error'
+            portal.append({'area1':a1,'area2':a2,'open_word':opened,'initial_counter':initial,'outcome':outcome,'error_text':next((e['text'] for e in error2 if e['call']==t['stop']),None),
+                'matrix_words':[get_word(t['memory'],matrix+4*i) for i in range(9)],'matrix_writes':[w['address']-matrix for w in t['writes'] if matrix<=w['address']<matrix+36], 'epoch_word':get_word(t['memory'],epoch_slot),
+                'flood_arguments':[t['registers']['rdi'],t['registers']['rsi']] if outcome=='flood_request' else None,**trace_fields(t,a1==0 and a2==1 and opened in [0,1] and initial==1)})
+        out['native'][label]={'source':r['source'],'disassembly_sha256':a['uncompressed_sha256'],'graphs':graphs,'references':references,'consumer_probes':consumers,
+            'area_slots':{'count':count_slot,'base':area_slot,'noareas_cvar_pointer':noareas_slot,'matrix':matrix_slot,'epoch':epoch_slot},'connectivity_errors':error,'connectivity_probes':connected,
+            'portal_errors':error2,'portal_omitted_pushes':omitted,'portal_probes':portal}
+    write_json(Path('docs/research/bsp-references.json'),out)
+    print('Recovered 48 reference copies, 48 consumer-address, 450 connectivity and 800 portal probes')
+
+
+if __name__=='__main__':main()

scripts/analyze_collision_composition.py

@@ -14,7 +14,7 @@ from analyze_vm_extensions import native_functions,verified
 
 def register(r):
     aliases={'eax':('rax',32),'edi':('rdi',32),'esi':('rsi',32),'edx':('rdx',32),
-             'ecx':('rcx',32),'ebp':('rbp',32),'ebx':('rbx',32),'al':('rax',8),'sil':('rsi',8),'dl':('rdx',8)}
+             'ecx':('rcx',32),'ebp':('rbp',32),'ebx':('rbx',32),'al':('rax',8),'sil':('rsi',8),'dl':('rdx',8),'cl':('rcx',8),'dil':('rdi',8)}
     r=r.lstrip('%')
     if r in aliases:return aliases[r]
     if re.fullmatch(r'r\d+d',r):return r[:-1],32

scripts/test_bsp_references.py

@@ -0,0 +1,92 @@
+#!/usr/bin/env python3
+"""Independent reference, connectivity and portal mutation oracles."""
+import json
+from pathlib import Path
+import unittest
+from analyze_collision_composition import evaluate
+from test_bsp_planes import graph
+
+
+class BspReferenceTests(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):cls.r=json.loads(Path('docs/research/bsp-references.json').read_text())
+
+    def test_reference_copy_allocation_and_canaries(self):
+        for n in self.r['native'].values():
+            for name,info in n['references'].items():
+                self.assertEqual(len(info['probes']),12)
+                self.assertEqual(info['direct_CM_LoadMap_calls'],[])
+                for p in info['probes']:
+                    self.assertEqual(p['output_words'],p['input_words']);self.assertEqual(p['count_word'],p['count']);self.assertEqual(p['base_word'],0x30000)
+                    self.assertEqual(p['allocation_bytes'],4*(p['count']+int(name.endswith('Brushes'))))
+                    self.assertEqual(p['canaries'],[0xa5,0xa5]);self.assertEqual(p['writes'],[{'address':0x30000+4*i,'bytes':4} for i in range(p['count'])])
+
+    def test_consumer_signed_addressing(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['consumer_probes']),24)
+            for p in n['consumer_probes']:
+                expected=0x90000+56*p['supplied_id'] if p['kind']=='brush' else 0xb0000+16*(p['supplied_id']+1)
+                self.assertEqual(p['pointer'],expected)
+
+    def test_connectivity_bypass_negative_and_upper_bound_priority(self):
+        labels=[1,1,2]
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['connectivity_probes']),225)
+            for p in n['connectivity_probes']:
+                a,b,c=p['area1'],p['area2'],p['area_count']
+                error=p['disabled_word']==0 and a>=0 and b>=0 and (a>=c or b>=c)
+                result=None if error else 1 if p['disabled_word'] else 0 if min(a,b)<0 else int(labels[a]==labels[b])
+                self.assertEqual(p['error'],error);self.assertEqual(p['result_word'],result)
+
+    def test_portal_pair_counters_errors_epoch_and_flood_request(self):
+        for n in self.r['native'].values():
+            self.assertEqual(len(n['portal_probes']),400);self.assertEqual(len(n['portal_omitted_pushes']),3)
+            for p in n['portal_probes']:
+                a,b=p['area1'],p['area2'];valid=0<=a<3 and 0<=b<3;m=[0]*9;writes=[];epoch=40;request=None;text=None
+                if valid:
+                    for i in [3*a+b,3*b+a]:m[i]=p['initial_counter']&0xffffffff
+                if min(a,b)<0:outcome='return'
+                elif not valid:outcome='error';text='CM_ChangeAreaPortalState: bad area number'
+                else:
+                    delta=1 if p['open_word'] else -1
+                    for i in [3*a+b,3*b+a]:m[i]=(m[i]+delta)&0xffffffff;writes.append(4*i)
+                    final=m[3*b+a];negative=final>=2**31
+                    if not p['open_word'] and negative:outcome='error';text='CM_AdjustAreaPortalState: negative reference count'
+                    else:outcome='flood_request';epoch=41;request=[0,1]
+                self.assertEqual(p['matrix_words'],m);self.assertEqual(p['matrix_writes'],writes);self.assertEqual(p['outcome'],outcome);self.assertEqual(p['error_text'],text);self.assertEqual(p['epoch_word'],epoch);self.assertEqual(p['flood_arguments'],request)
+
+    def test_self_pair_mutates_twice_and_nonzero_open_values_agree(self):
+        for n in self.r['native'].values():
+            rows=n['portal_probes']
+            selected=lambda opened:next(p for p in rows if p['area1']==p['area2']==0 and p['initial_counter']==1 and p['open_word']==opened)
+            self.assertEqual(selected(1)['matrix_words'][0],3)
+            self.assertEqual(selected(0)['matrix_words'][0],0xffffffff)
+            for v in [7,-1]:
+                self.assertEqual(selected(v)['matrix_words'],selected(1)['matrix_words']);self.assertEqual(selected(v)['outcome'],'flood_request')
+
+    def test_slots_connect_existing_leaf_loading_state(self):
+        prior=json.loads(Path('docs/research/bsp-loading.json').read_text())
+        for label,n in self.r['native'].items():
+            info=prior['native'][label]['loaders']['CMod_LoadLeafs'];self.assertEqual(n['area_slots']['count'],info['maxima_slots']['area'])
+            self.assertEqual(n['area_slots']['base']+8,n['area_slots']['matrix'])
+            self.assertEqual(n['references']['CMod_LoadLeafBrushes']['base_slot']+16,n['references']['CMod_LoadLeafSurfaces']['base_slot'])
+
+    def test_paired_outputs_and_trace_lengths(self):
+        a,b=self.r['native'].values()
+        def clean(rows):return [{k:v for k,v in p.items() if k not in ['trace','trace_sha256']} for p in rows]
+        for key in ['consumer_probes','connectivity_probes','portal_probes']:self.assertEqual(clean(a[key]),clean(b[key]))
+        for name in a['references']:self.assertEqual(clean(a['references'][name]['probes']),clean(b['references'][name]['probes']))
+        for n in self.r['native'].values():
+            for key in ['consumer_probes','connectivity_probes','portal_probes']:
+                for p in n[key]:
+                    if p['trace'] is not None:self.assertEqual(len(p['trace']),p['trace_length'])
+
+    def test_byte_alias_preservation_and_unknown_flags(self):
+        g=graph([('movb','$0x7, %dil'),('cmpl','$0x1, %eax'),('sete','%cl'),('movzbl','%cl, %edx'),('nop','')])
+        for v in [0,1,0xffffffff]:
+            t=evaluate(g,0,[4],{}, {'%rdi':0x123400,'%rcx':0x567800,'%rax':v},capture_registers=True)
+            self.assertEqual(t['registers']['rdi'],0x123407);self.assertEqual(t['registers']['rcx'],0x567800+int(v==1));self.assertEqual(t['registers']['rdx'],int(v==1))
+        with self.assertRaises(ValueError):evaluate(graph([('sete','%cl')]),0,[1],{}, {'%rcx':0})
+
+
+if __name__=='__main__':unittest.main()