nmarshall commited on
Commit
16c1385
·
1 Parent(s): b913272

qvm: execute small ranking sort

Browse files
docs/ASSET_PIPELINE_TODO.md CHANGED
@@ -964,3 +964,6 @@ Research progress: RE-767 [report](research/RUST_RECIPIENT_PAYLOAD.md); original
964
 
965
 
966
  Research progress: RE-769 [report](research/QVM_RANK_COMPARATOR.md); original assets unchanged.
 
 
 
 
964
 
965
 
966
  Research progress: RE-769 [report](research/QVM_RANK_COMPARATOR.md); original assets unchanged.
967
+
968
+
969
+ Research progress: RE-770 [report](research/QVM_RANK_SMALL_SORT.md); original assets unchanged.
docs/FINDINGS.md CHANGED
@@ -1310,3 +1310,6 @@ RE-725–734 batch: [health, contact and Rust impulse findings](research/QVM_HEA
1310
 
1311
 
1312
  **RE-769 — Complete ranking comparator gates ties and mode priorities:** 5441 preserves mode-specific score/time/class priorities and returns non-ordering results for some deferred/connecting pairs. [Evidence](research/QVM_RANK_COMPARATOR.md).
 
 
 
 
1310
 
1311
 
1312
  **RE-769 — Complete ranking comparator gates ties and mode priorities:** 5441 preserves mode-specific score/time/class priorities and returns non-ordering results for some deferred/connecting pairs. [Evidence](research/QVM_RANK_COMPARATOR.md).
1313
+
1314
+
1315
+ **RE-770 — Actual aligned small rank sorting with dynamic comparator:** 35095 aligned four-byte insertion sorting now executes actual5441 callbacks for fewer than7 entries. [Evidence](research/QVM_RANK_SMALL_SORT.md).
docs/PROVISIONAL_TODO.md CHANGED
@@ -1308,3 +1308,6 @@ RE-758–767 batch validation:1473 Python and80 Rust tests pass after ten indepe
1308
 
1309
 
1310
  Research progress: RE-769 [report](research/QVM_RANK_COMPARATOR.md); full native/platform/game scope remains open.
 
 
 
 
1308
 
1309
 
1310
  Research progress: RE-769 [report](research/QVM_RANK_COMPARATOR.md); full native/platform/game scope remains open.
1311
+
1312
+
1313
+ Research progress: RE-770 [report](research/QVM_RANK_SMALL_SORT.md); full native/platform/game scope remains open.
docs/RESEARCH_LEDGER.md CHANGED
@@ -1503,3 +1503,8 @@ Exactly ten task commits follow baseline `27a7f96f`. Every completed task passed
1503
  ## 2026-10-21 — RE-769 Complete ranking comparator gates ties and mode priorities
1504
 
1505
  5441 preserves mode-specific score/time/class priorities and returns non-ordering results for some deferred/connecting pairs. Focused `cargo xtask test-group rank-comparator` passes (3 Python tests); selected reports regenerate twice byte-identically, with storage, links and whitespace guards. New report: 136008 bytes. Full-suite validation is deferred to the tenth batch commit. No original assets, Rust runtime or shared VM evaluator changed.
 
 
 
 
 
 
1503
  ## 2026-10-21 — RE-769 Complete ranking comparator gates ties and mode priorities
1504
 
1505
  5441 preserves mode-specific score/time/class priorities and returns non-ordering results for some deferred/connecting pairs. Focused `cargo xtask test-group rank-comparator` passes (3 Python tests); selected reports regenerate twice byte-identically, with storage, links and whitespace guards. New report: 136008 bytes. Full-suite validation is deferred to the tenth batch commit. No original assets, Rust runtime or shared VM evaluator changed.
1506
+
1507
+
1508
+ ## 2026-10-21 — RE-770 Actual aligned small rank sorting with dynamic comparator
1509
+
1510
+ 35095 aligned four-byte insertion sorting now executes actual5441 callbacks for fewer than7 entries. Focused `cargo xtask test-group rank-small-sort` passes (4 Python tests); selected reports regenerate twice byte-identically, with storage, links and whitespace guards. New report: 142955 bytes. Full-suite validation is deferred to the tenth batch commit. Optional explicit result-offset mapping added to trace resumption; default instruction evaluation unchanged. Focused actual sort checks validate dynamic callback return placement. Correct repeated-call reentry is opt-in for this sort; legacy repeated-call analyses require an explicit audit. Broader full regression remains deferred until task ten.
docs/REVERSE_ENGINEERING_TODO.md CHANGED
@@ -1351,3 +1351,9 @@ Purpose: turn the supplied installation into an evidence-backed specification of
1351
 
1352
 
1353
  - [x] RE-769 Complete ranking comparator gates ties and mode priorities ([report](research/QVM_RANK_COMPARATOR.md)).
 
 
 
 
 
 
 
1351
 
1352
 
1353
  - [x] RE-769 Complete ranking comparator gates ties and mode priorities ([report](research/QVM_RANK_COMPARATOR.md)).
1354
+
1355
+
1356
+ - [x] RE-770 Actual aligned small rank sorting with dynamic comparator ([report](research/QVM_RANK_SMALL_SORT.md)).
1357
+
1358
+
1359
+ - [ ] Audit legacy `qvm_trace_support.resume` consumers for repeated visits to the same call-site return boundary; migrate with corrected reentry semantics and regenerate/review each affected oracle before broader parity claims. See [RE-770](research/QVM_RANK_SMALL_SORT.md).
docs/research/QVM_RANK_SMALL_SORT.md ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ # Actual aligned small rank sorting with dynamic comparator
2
+
3
+ RE-770. Reproduce with `python3 scripts/analyze_qvm_rank_small_sort.py`; validate with `cargo xtask test-group rank-small-sort`. See the [machine report](qvm-rank-small-sort.json).
4
+
5
+ Recovered the actual small-array branch across44 order/mode/count fixtures. Added optional explicit CALL result offsets to trace resumption because the dynamic callback pointer load follows the return destination. This preserves callback local136 and stores its result at local80. Also completed unsigned LEU/GEU evaluation and added boundary checks for all four unsigned comparison operators. Existing unsigned strict comparisons keep their behavior. Large quicksort, recursion, unaligned/byte swaps and native sorting parity remain open.
6
+
7
+ No original executable/full VM/native world collision or rendering runs. Supplied callback/native responses and fixture domains remain explicit; complete macOS/Linux Rust/Bevy/Solari gameplay remains open.
8
+
9
+
10
+ The actual sort exposed a loop-resumption issue: excluding the current return boundary for an entire segment also misses later visits to the same call site. New `repeated_calls=True` honors every reentry after skipping the initial boundary once. This sort opts in; earlier callers retain their legacy setting and repeated-call analyses require a separate audit before broader parity claims.
docs/research/README.md CHANGED
@@ -900,3 +900,6 @@ The extension collector adds five focused tests for native tail calls, command-r
900
 
901
 
902
  [RE-769: Complete ranking comparator gates ties and mode priorities](QVM_RANK_COMPARATOR.md).
 
 
 
 
900
 
901
 
902
  [RE-769: Complete ranking comparator gates ties and mode priorities](QVM_RANK_COMPARATOR.md).
903
+
904
+
905
+ [RE-770: Actual aligned small rank sorting with dynamic comparator](QVM_RANK_SMALL_SORT.md).
docs/research/qvm-rank-small-sort.json ADDED
The diff for this file is too large to render. See raw diff
 
scripts/analyze_deferred_auth.py CHANGED
@@ -17,7 +17,7 @@ def signed32(value):
17
 
18
 
19
  def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_memory=None, capture_stack=False,
20
- halt_on_leave=False, stop_targets=(), call_results=None, resume_at=None, local_base=0x40000000):
21
  """Evaluate only a selected straight/bounded slice against synthetic memory.
22
 
23
  Record calls rather than entering callees. Reject unknown reads/opcodes and
@@ -33,7 +33,7 @@ def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_m
33
  pc = start if resume_at is None else resume_at
34
  if not start <= pc <= stop:
35
  raise ValueError('resume outside bounded slice')
36
- while pc != stop and pc not in stop_targets:
37
  if not start <= pc < stop or len(trace) >= limit:
38
  raise ValueError('slice control flow escaped bound')
39
  ins = instructions[pc]
@@ -158,9 +158,9 @@ def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_m
158
  quotient = abs(left) // abs(right)
159
  quotient = -quotient if (left < 0) != (right < 0) else quotient
160
  stack.append(quotient if op == 'DIVI' else left - quotient*right)
161
- elif op in ('LTU', 'GTU'):
162
  right, left = stack.pop() & 0xffffffff, stack.pop() & 0xffffffff
163
- if (left < right if op == 'LTU' else left > right):
164
  pc = value
165
  elif op in ('EQ', 'NE', 'LTI', 'LEI', 'GTI', 'GEI'):
166
  right, left = signed32(stack.pop()), signed32(stack.pop())
 
17
 
18
 
19
  def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_memory=None, capture_stack=False,
20
+ halt_on_leave=False, stop_targets=(), call_results=None, resume_at=None, local_base=0x40000000, skip_initial_stop=False):
21
  """Evaluate only a selected straight/bounded slice against synthetic memory.
22
 
23
  Record calls rather than entering callees. Reject unknown reads/opcodes and
 
33
  pc = start if resume_at is None else resume_at
34
  if not start <= pc <= stop:
35
  raise ValueError('resume outside bounded slice')
36
+ while pc != stop and (pc not in stop_targets or skip_initial_stop and not trace):
37
  if not start <= pc < stop or len(trace) >= limit:
38
  raise ValueError('slice control flow escaped bound')
39
  ins = instructions[pc]
 
158
  quotient = abs(left) // abs(right)
159
  quotient = -quotient if (left < 0) != (right < 0) else quotient
160
  stack.append(quotient if op == 'DIVI' else left - quotient*right)
161
+ elif op in ('LTU', 'LEU', 'GTU', 'GEU'):
162
  right, left = stack.pop() & 0xffffffff, stack.pop() & 0xffffffff
163
+ if {'LTU':left<right,'LEU':left<=right,'GTU':left>right,'GEU':left>=right}[op]:
164
  pc = value
165
  elif op in ('EQ', 'NE', 'LTI', 'LEI', 'GTI', 'GEI'):
166
  right, left = signed32(stack.pop()), signed32(stack.pop())
scripts/analyze_qvm_rank_small_sort.py ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Actual small aligned rank sort with actual5441 dynamic comparator calls."""
3
+ import itertools
4
+ from pathlib import Path
5
+ from qvm_research_support import source
6
+ from qvm_rank_compare_support import seed,PTR
7
+ from qvm_rank_sort_support import sort
8
+ from analyze_qvm import write_json
9
+
10
+ def main():
11
+ q,raw,h,ins=source();rows=[];clients=[{'248':j%3,'280':j%2,'3032':j,'308':0 if j==0 else 7-j,'696':j%3} for j in range(6)];cases=[(mode,list(order)) for mode,order in itertools.product([0,3,9,11],itertools.permutations(range(3)))]+[(mode,list(reversed(range(n)))) for mode,n in itertools.product([0,3,9,11],[0,1,2,3,6])]
12
+ for mode,order in cases:
13
+ m=seed(mode,clients)
14
+ for j,v in enumerate(order):m[PTR+4*j]=v
15
+ _,_,p=sort(ins,m,{8:PTR,12:len(order),16:4,20:5441});p.update(mode=mode,clients=clients);rows.append(p)
16
+ write_json(Path('docs/research/qvm-rank-small-sort.json'),{'source':q['source'],'sha256':q['sha256'],'probes':rows,'limits':'Actual35095 aligned4-byte n<7 insertion branch with actual5441/33715. No supplied sorting result. Explicit dynamic CALL result offset35229->local80. Large quicksort, byte swaps, recursion and native parity remain open; eligible numeric clients only.'})
17
+ if __name__=='__main__':main()
scripts/qvm_rank_sort_support.py ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Actual35095 aligned four-byte insertion-sort branch for fewer than7 entries."""
2
+ from qvm_health_effects_support import H
3
+ from qvm_rank_compare_support import compare
4
+ from qvm_trace_support import resume
5
+ S=H+0x1b0000
6
+
7
+ def sort(ins,m,args):
8
+ assert args[12]<7 and args[16]==4 and args[8]%4==0 and args[20]==5441
9
+ for o,v in args.items():m[S+116+o]=v
10
+ before=[m[args[8]+4*j] for j in range(args[12])]
11
+ # Dynamic callback CALL loads its pointer after the destination LOCAL.
12
+ m,e=resume(ins,35096,36036,m,S,handlers={5441:compare},result_offsets={35229:80},repeated_calls=True)
13
+ return m,0,{'arguments':args,'initial':before,'final':[m[args[8]+4*j] for j in range(args[12])],'callback_pointer_after':m[S+136],'execution':e}
scripts/qvm_trace_support.py CHANGED
@@ -2,16 +2,19 @@
2
  from analyze_deferred_auth import trace_slice
3
  from analyze_bsp_references import trace_fields
4
 
5
- def resume(ins,start,stop,m,base,pc=None,handlers=None,boundaries=(),byte_inputs=None,limit=8192):
6
  handlers=handlers or {};pc=start if pc is None else pc;m=dict(m);trace=[];calls=[];children=[];writes=[];bytes_map=dict(byte_inputs or {});ret=None
7
  points=[i['index']+2 for i in ins[start:stop] if i['opcode']=='CALL']+list(boundaries)
8
  for _ in range(512):
9
- t=trace_slice(ins,start,stop,m,{},resume_at=pc,stop_targets=[p for p in points if p!=pc],local_base=base,byte_memory=bytes_map,halt_on_leave=True,capture_stack=True,limit=limit);m=t['memory'];pc=t.get('stop',stop);trace+=t['trace'];calls+=t['calls'];writes+=t['byte_writes'];bytes_map=t['byte_memory']
10
  for c in t['calls']:
11
  if c['target'] not in handlers:raise ValueError('Unrecovered trace child '+str(c['target']))
12
  m,value,e=handlers[c['target']](ins,m,c['args'])
13
  if ins[c['instruction']+1]['opcode']=='STORE4':
14
- dest=ins[c['instruction']-2];assert dest['opcode']=='LOCAL';m[base+dest['operand']]=value
 
 
 
15
  children.append({'call':c,'effect':e})
16
  if t['trace'] and ins[t['trace'][-1]]['opcode']=='LEAVE':ret=t['stack'][-1];break
17
  if pc==stop or pc in boundaries:break
 
2
  from analyze_deferred_auth import trace_slice
3
  from analyze_bsp_references import trace_fields
4
 
5
+ def resume(ins,start,stop,m,base,pc=None,handlers=None,boundaries=(),byte_inputs=None,limit=8192,result_offsets=None,repeated_calls=False):
6
  handlers=handlers or {};pc=start if pc is None else pc;m=dict(m);trace=[];calls=[];children=[];writes=[];bytes_map=dict(byte_inputs or {});ret=None
7
  points=[i['index']+2 for i in ins[start:stop] if i['opcode']=='CALL']+list(boundaries)
8
  for _ in range(512):
9
+ t=trace_slice(ins,start,stop,m,{},resume_at=pc,stop_targets=points if repeated_calls else [p for p in points if p!=pc],local_base=base,byte_memory=bytes_map,halt_on_leave=True,capture_stack=True,limit=limit,skip_initial_stop=repeated_calls);m=t['memory'];pc=t.get('stop',stop);trace+=t['trace'];calls+=t['calls'];writes+=t['byte_writes'];bytes_map=t['byte_memory']
10
  for c in t['calls']:
11
  if c['target'] not in handlers:raise ValueError('Unrecovered trace child '+str(c['target']))
12
  m,value,e=handlers[c['target']](ins,m,c['args'])
13
  if ins[c['instruction']+1]['opcode']=='STORE4':
14
+ offset=(result_offsets or {}).get(c['instruction'])
15
+ if offset is None:
16
+ dest=ins[c['instruction']-2];assert dest['opcode']=='LOCAL';offset=dest['operand']
17
+ m[base+offset]=value
18
  children.append({'call':c,'effect':e})
19
  if t['trace'] and ins[t['trace'][-1]]['opcode']=='LEAVE':ret=t['stack'][-1];break
20
  if pc==stop or pc in boundaries:break
scripts/test_qvm_rank_small_sort.py ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import functools,json,unittest
2
+ from pathlib import Path
3
+ from test_qvm_rank_comparator import compare
4
+ class SmallRankSortTests(unittest.TestCase):
5
+ @classmethod
6
+ def setUpClass(cls):cls.ps=json.loads(Path('docs/research/qvm-rank-small-sort.json').read_text())['probes']
7
+ def test_independent_stable_numeric_sort(self):
8
+ for p in self.ps:self.assertEqual(p['final'],sorted(p['initial'],key=functools.cmp_to_key(lambda a,b:compare(p['mode'],p['clients'][a],p['clients'][b]))))
9
+ def test_dynamic_result_patch_preserves_callback_pointer(self):
10
+ for p in self.ps:self.assertEqual(p['callback_pointer_after'],5441);self.assertEqual(p['execution']['stop'],36036);self.assertTrue(all(c['target']==5441 for c in p['execution']['calls']))
11
+ def test_unsigned_branch_boundaries(self):
12
+ from analyze_deferred_auth import trace_slice
13
+ for op in ['LTU','LEU','GTU','GEU']:
14
+ for a in [0,1,0x7fffffff,0x80000000,-1]:
15
+ for b in [0,1,0x7fffffff,0x80000000,-1]:
16
+ ins=[{'index':0,'opcode':'CONST','operand':a},{'index':1,'opcode':'CONST','operand':b},{'index':2,'opcode':op,'operand':5},{'index':3,'opcode':'CONST','operand':0},{'index':4,'opcode':'POP','operand':None}]
17
+ r=trace_slice(ins,0,5,{},{});x,y=a&0xffffffff,b&0xffffffff;branch={'LTU':x<y,'LEU':x<=y,'GTU':x>y,'GEU':x>=y}[op];self.assertEqual(3 not in r['trace'],branch,(op,a,b))
18
+ def test_domains_and_real_movement(self):
19
+ self.assertEqual(len(self.ps),44);self.assertTrue(any(p['final']!=p['initial'] for p in self.ps));self.assertEqual({len(p['initial']) for p in self.ps},{0,1,2,3,6})
20
+ if __name__=='__main__':unittest.main()
xtask/checks.tsv CHANGED
@@ -1326,3 +1326,6 @@ edge docs/research/rust-recipient-payload.json scripts/test_rust_recipient_paylo
1326
  report scripts/analyze_qvm_rank_comparator.py docs/research/qvm-rank-comparator.json
1327
  edge docs/research/qvm-rank-comparator.json scripts/test_qvm_rank_comparator.py
1328
  group rank-comparator scripts/analyze_qvm_rank_comparator.py
 
 
 
 
1326
  report scripts/analyze_qvm_rank_comparator.py docs/research/qvm-rank-comparator.json
1327
  edge docs/research/qvm-rank-comparator.json scripts/test_qvm_rank_comparator.py
1328
  group rank-comparator scripts/analyze_qvm_rank_comparator.py
1329
+ report scripts/analyze_qvm_rank_small_sort.py docs/research/qvm-rank-small-sort.json
1330
+ edge docs/research/qvm-rank-small-sort.json scripts/test_qvm_rank_small_sort.py
1331
+ group rank-small-sort scripts/analyze_qvm_rank_small_sort.py