qvm: execute small ranking sort
Browse files- docs/ASSET_PIPELINE_TODO.md +3 -0
- docs/FINDINGS.md +3 -0
- docs/PROVISIONAL_TODO.md +3 -0
- docs/RESEARCH_LEDGER.md +5 -0
- docs/REVERSE_ENGINEERING_TODO.md +6 -0
- docs/research/QVM_RANK_SMALL_SORT.md +10 -0
- docs/research/README.md +3 -0
- docs/research/qvm-rank-small-sort.json +0 -0
- scripts/analyze_deferred_auth.py +4 -4
- scripts/analyze_qvm_rank_small_sort.py +17 -0
- scripts/qvm_rank_sort_support.py +13 -0
- scripts/qvm_trace_support.py +6 -3
- scripts/test_qvm_rank_small_sort.py +20 -0
- xtask/checks.tsv +3 -0
docs/ASSET_PIPELINE_TODO.md
CHANGED
|
@@ -964,3 +964,6 @@ Research progress: RE-767 [report](research/RUST_RECIPIENT_PAYLOAD.md); original
|
|
| 964 |
|
| 965 |
|
| 966 |
Research progress: RE-769 [report](research/QVM_RANK_COMPARATOR.md); original assets unchanged.
|
|
|
|
|
|
|
|
|
|
|
|
| 964 |
|
| 965 |
|
| 966 |
Research progress: RE-769 [report](research/QVM_RANK_COMPARATOR.md); original assets unchanged.
|
| 967 |
+
|
| 968 |
+
|
| 969 |
+
Research progress: RE-770 [report](research/QVM_RANK_SMALL_SORT.md); original assets unchanged.
|
docs/FINDINGS.md
CHANGED
|
@@ -1310,3 +1310,6 @@ RE-725–734 batch: [health, contact and Rust impulse findings](research/QVM_HEA
|
|
| 1310 |
|
| 1311 |
|
| 1312 |
**RE-769 — Complete ranking comparator gates ties and mode priorities:** 5441 preserves mode-specific score/time/class priorities and returns non-ordering results for some deferred/connecting pairs. [Evidence](research/QVM_RANK_COMPARATOR.md).
|
|
|
|
|
|
|
|
|
|
|
|
| 1310 |
|
| 1311 |
|
| 1312 |
**RE-769 — Complete ranking comparator gates ties and mode priorities:** 5441 preserves mode-specific score/time/class priorities and returns non-ordering results for some deferred/connecting pairs. [Evidence](research/QVM_RANK_COMPARATOR.md).
|
| 1313 |
+
|
| 1314 |
+
|
| 1315 |
+
**RE-770 — Actual aligned small rank sorting with dynamic comparator:** 35095 aligned four-byte insertion sorting now executes actual5441 callbacks for fewer than7 entries. [Evidence](research/QVM_RANK_SMALL_SORT.md).
|
docs/PROVISIONAL_TODO.md
CHANGED
|
@@ -1308,3 +1308,6 @@ RE-758–767 batch validation:1473 Python and80 Rust tests pass after ten indepe
|
|
| 1308 |
|
| 1309 |
|
| 1310 |
Research progress: RE-769 [report](research/QVM_RANK_COMPARATOR.md); full native/platform/game scope remains open.
|
|
|
|
|
|
|
|
|
|
|
|
| 1308 |
|
| 1309 |
|
| 1310 |
Research progress: RE-769 [report](research/QVM_RANK_COMPARATOR.md); full native/platform/game scope remains open.
|
| 1311 |
+
|
| 1312 |
+
|
| 1313 |
+
Research progress: RE-770 [report](research/QVM_RANK_SMALL_SORT.md); full native/platform/game scope remains open.
|
docs/RESEARCH_LEDGER.md
CHANGED
|
@@ -1503,3 +1503,8 @@ Exactly ten task commits follow baseline `27a7f96f`. Every completed task passed
|
|
| 1503 |
## 2026-10-21 — RE-769 Complete ranking comparator gates ties and mode priorities
|
| 1504 |
|
| 1505 |
5441 preserves mode-specific score/time/class priorities and returns non-ordering results for some deferred/connecting pairs. Focused `cargo xtask test-group rank-comparator` passes (3 Python tests); selected reports regenerate twice byte-identically, with storage, links and whitespace guards. New report: 136008 bytes. Full-suite validation is deferred to the tenth batch commit. No original assets, Rust runtime or shared VM evaluator changed.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1503 |
## 2026-10-21 — RE-769 Complete ranking comparator gates ties and mode priorities
|
| 1504 |
|
| 1505 |
5441 preserves mode-specific score/time/class priorities and returns non-ordering results for some deferred/connecting pairs. Focused `cargo xtask test-group rank-comparator` passes (3 Python tests); selected reports regenerate twice byte-identically, with storage, links and whitespace guards. New report: 136008 bytes. Full-suite validation is deferred to the tenth batch commit. No original assets, Rust runtime or shared VM evaluator changed.
|
| 1506 |
+
|
| 1507 |
+
|
| 1508 |
+
## 2026-10-21 — RE-770 Actual aligned small rank sorting with dynamic comparator
|
| 1509 |
+
|
| 1510 |
+
35095 aligned four-byte insertion sorting now executes actual5441 callbacks for fewer than7 entries. Focused `cargo xtask test-group rank-small-sort` passes (4 Python tests); selected reports regenerate twice byte-identically, with storage, links and whitespace guards. New report: 142955 bytes. Full-suite validation is deferred to the tenth batch commit. Optional explicit result-offset mapping added to trace resumption; default instruction evaluation unchanged. Focused actual sort checks validate dynamic callback return placement. Correct repeated-call reentry is opt-in for this sort; legacy repeated-call analyses require an explicit audit. Broader full regression remains deferred until task ten.
|
docs/REVERSE_ENGINEERING_TODO.md
CHANGED
|
@@ -1351,3 +1351,9 @@ Purpose: turn the supplied installation into an evidence-backed specification of
|
|
| 1351 |
|
| 1352 |
|
| 1353 |
- [x] RE-769 Complete ranking comparator gates ties and mode priorities ([report](research/QVM_RANK_COMPARATOR.md)).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1351 |
|
| 1352 |
|
| 1353 |
- [x] RE-769 Complete ranking comparator gates ties and mode priorities ([report](research/QVM_RANK_COMPARATOR.md)).
|
| 1354 |
+
|
| 1355 |
+
|
| 1356 |
+
- [x] RE-770 Actual aligned small rank sorting with dynamic comparator ([report](research/QVM_RANK_SMALL_SORT.md)).
|
| 1357 |
+
|
| 1358 |
+
|
| 1359 |
+
- [ ] Audit legacy `qvm_trace_support.resume` consumers for repeated visits to the same call-site return boundary; migrate with corrected reentry semantics and regenerate/review each affected oracle before broader parity claims. See [RE-770](research/QVM_RANK_SMALL_SORT.md).
|
docs/research/QVM_RANK_SMALL_SORT.md
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Actual aligned small rank sorting with dynamic comparator
|
| 2 |
+
|
| 3 |
+
RE-770. Reproduce with `python3 scripts/analyze_qvm_rank_small_sort.py`; validate with `cargo xtask test-group rank-small-sort`. See the [machine report](qvm-rank-small-sort.json).
|
| 4 |
+
|
| 5 |
+
Recovered the actual small-array branch across44 order/mode/count fixtures. Added optional explicit CALL result offsets to trace resumption because the dynamic callback pointer load follows the return destination. This preserves callback local136 and stores its result at local80. Also completed unsigned LEU/GEU evaluation and added boundary checks for all four unsigned comparison operators. Existing unsigned strict comparisons keep their behavior. Large quicksort, recursion, unaligned/byte swaps and native sorting parity remain open.
|
| 6 |
+
|
| 7 |
+
No original executable/full VM/native world collision or rendering runs. Supplied callback/native responses and fixture domains remain explicit; complete macOS/Linux Rust/Bevy/Solari gameplay remains open.
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
The actual sort exposed a loop-resumption issue: excluding the current return boundary for an entire segment also misses later visits to the same call site. New `repeated_calls=True` honors every reentry after skipping the initial boundary once. This sort opts in; earlier callers retain their legacy setting and repeated-call analyses require a separate audit before broader parity claims.
|
docs/research/README.md
CHANGED
|
@@ -900,3 +900,6 @@ The extension collector adds five focused tests for native tail calls, command-r
|
|
| 900 |
|
| 901 |
|
| 902 |
[RE-769: Complete ranking comparator gates ties and mode priorities](QVM_RANK_COMPARATOR.md).
|
|
|
|
|
|
|
|
|
|
|
|
| 900 |
|
| 901 |
|
| 902 |
[RE-769: Complete ranking comparator gates ties and mode priorities](QVM_RANK_COMPARATOR.md).
|
| 903 |
+
|
| 904 |
+
|
| 905 |
+
[RE-770: Actual aligned small rank sorting with dynamic comparator](QVM_RANK_SMALL_SORT.md).
|
docs/research/qvm-rank-small-sort.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
scripts/analyze_deferred_auth.py
CHANGED
|
@@ -17,7 +17,7 @@ def signed32(value):
|
|
| 17 |
|
| 18 |
|
| 19 |
def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_memory=None, capture_stack=False,
|
| 20 |
-
halt_on_leave=False, stop_targets=(), call_results=None, resume_at=None, local_base=0x40000000):
|
| 21 |
"""Evaluate only a selected straight/bounded slice against synthetic memory.
|
| 22 |
|
| 23 |
Record calls rather than entering callees. Reject unknown reads/opcodes and
|
|
@@ -33,7 +33,7 @@ def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_m
|
|
| 33 |
pc = start if resume_at is None else resume_at
|
| 34 |
if not start <= pc <= stop:
|
| 35 |
raise ValueError('resume outside bounded slice')
|
| 36 |
-
while pc != stop and pc not in stop_targets:
|
| 37 |
if not start <= pc < stop or len(trace) >= limit:
|
| 38 |
raise ValueError('slice control flow escaped bound')
|
| 39 |
ins = instructions[pc]
|
|
@@ -158,9 +158,9 @@ def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_m
|
|
| 158 |
quotient = abs(left) // abs(right)
|
| 159 |
quotient = -quotient if (left < 0) != (right < 0) else quotient
|
| 160 |
stack.append(quotient if op == 'DIVI' else left - quotient*right)
|
| 161 |
-
elif op in ('LTU', 'GTU'):
|
| 162 |
right, left = stack.pop() & 0xffffffff, stack.pop() & 0xffffffff
|
| 163 |
-
if
|
| 164 |
pc = value
|
| 165 |
elif op in ('EQ', 'NE', 'LTI', 'LEI', 'GTI', 'GEI'):
|
| 166 |
right, left = signed32(stack.pop()), signed32(stack.pop())
|
|
|
|
| 17 |
|
| 18 |
|
| 19 |
def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_memory=None, capture_stack=False,
|
| 20 |
+
halt_on_leave=False, stop_targets=(), call_results=None, resume_at=None, local_base=0x40000000, skip_initial_stop=False):
|
| 21 |
"""Evaluate only a selected straight/bounded slice against synthetic memory.
|
| 22 |
|
| 23 |
Record calls rather than entering callees. Reject unknown reads/opcodes and
|
|
|
|
| 33 |
pc = start if resume_at is None else resume_at
|
| 34 |
if not start <= pc <= stop:
|
| 35 |
raise ValueError('resume outside bounded slice')
|
| 36 |
+
while pc != stop and (pc not in stop_targets or skip_initial_stop and not trace):
|
| 37 |
if not start <= pc < stop or len(trace) >= limit:
|
| 38 |
raise ValueError('slice control flow escaped bound')
|
| 39 |
ins = instructions[pc]
|
|
|
|
| 158 |
quotient = abs(left) // abs(right)
|
| 159 |
quotient = -quotient if (left < 0) != (right < 0) else quotient
|
| 160 |
stack.append(quotient if op == 'DIVI' else left - quotient*right)
|
| 161 |
+
elif op in ('LTU', 'LEU', 'GTU', 'GEU'):
|
| 162 |
right, left = stack.pop() & 0xffffffff, stack.pop() & 0xffffffff
|
| 163 |
+
if {'LTU':left<right,'LEU':left<=right,'GTU':left>right,'GEU':left>=right}[op]:
|
| 164 |
pc = value
|
| 165 |
elif op in ('EQ', 'NE', 'LTI', 'LEI', 'GTI', 'GEI'):
|
| 166 |
right, left = signed32(stack.pop()), signed32(stack.pop())
|
scripts/analyze_qvm_rank_small_sort.py
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Actual small aligned rank sort with actual5441 dynamic comparator calls."""
|
| 3 |
+
import itertools
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
from qvm_research_support import source
|
| 6 |
+
from qvm_rank_compare_support import seed,PTR
|
| 7 |
+
from qvm_rank_sort_support import sort
|
| 8 |
+
from analyze_qvm import write_json
|
| 9 |
+
|
| 10 |
+
def main():
|
| 11 |
+
q,raw,h,ins=source();rows=[];clients=[{'248':j%3,'280':j%2,'3032':j,'308':0 if j==0 else 7-j,'696':j%3} for j in range(6)];cases=[(mode,list(order)) for mode,order in itertools.product([0,3,9,11],itertools.permutations(range(3)))]+[(mode,list(reversed(range(n)))) for mode,n in itertools.product([0,3,9,11],[0,1,2,3,6])]
|
| 12 |
+
for mode,order in cases:
|
| 13 |
+
m=seed(mode,clients)
|
| 14 |
+
for j,v in enumerate(order):m[PTR+4*j]=v
|
| 15 |
+
_,_,p=sort(ins,m,{8:PTR,12:len(order),16:4,20:5441});p.update(mode=mode,clients=clients);rows.append(p)
|
| 16 |
+
write_json(Path('docs/research/qvm-rank-small-sort.json'),{'source':q['source'],'sha256':q['sha256'],'probes':rows,'limits':'Actual35095 aligned4-byte n<7 insertion branch with actual5441/33715. No supplied sorting result. Explicit dynamic CALL result offset35229->local80. Large quicksort, byte swaps, recursion and native parity remain open; eligible numeric clients only.'})
|
| 17 |
+
if __name__=='__main__':main()
|
scripts/qvm_rank_sort_support.py
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Actual35095 aligned four-byte insertion-sort branch for fewer than7 entries."""
|
| 2 |
+
from qvm_health_effects_support import H
|
| 3 |
+
from qvm_rank_compare_support import compare
|
| 4 |
+
from qvm_trace_support import resume
|
| 5 |
+
S=H+0x1b0000
|
| 6 |
+
|
| 7 |
+
def sort(ins,m,args):
|
| 8 |
+
assert args[12]<7 and args[16]==4 and args[8]%4==0 and args[20]==5441
|
| 9 |
+
for o,v in args.items():m[S+116+o]=v
|
| 10 |
+
before=[m[args[8]+4*j] for j in range(args[12])]
|
| 11 |
+
# Dynamic callback CALL loads its pointer after the destination LOCAL.
|
| 12 |
+
m,e=resume(ins,35096,36036,m,S,handlers={5441:compare},result_offsets={35229:80},repeated_calls=True)
|
| 13 |
+
return m,0,{'arguments':args,'initial':before,'final':[m[args[8]+4*j] for j in range(args[12])],'callback_pointer_after':m[S+136],'execution':e}
|
scripts/qvm_trace_support.py
CHANGED
|
@@ -2,16 +2,19 @@
|
|
| 2 |
from analyze_deferred_auth import trace_slice
|
| 3 |
from analyze_bsp_references import trace_fields
|
| 4 |
|
| 5 |
-
def resume(ins,start,stop,m,base,pc=None,handlers=None,boundaries=(),byte_inputs=None,limit=8192):
|
| 6 |
handlers=handlers or {};pc=start if pc is None else pc;m=dict(m);trace=[];calls=[];children=[];writes=[];bytes_map=dict(byte_inputs or {});ret=None
|
| 7 |
points=[i['index']+2 for i in ins[start:stop] if i['opcode']=='CALL']+list(boundaries)
|
| 8 |
for _ in range(512):
|
| 9 |
-
t=trace_slice(ins,start,stop,m,{},resume_at=pc,stop_targets=[p for p in points if p!=pc],local_base=base,byte_memory=bytes_map,halt_on_leave=True,capture_stack=True,limit=limit);m=t['memory'];pc=t.get('stop',stop);trace+=t['trace'];calls+=t['calls'];writes+=t['byte_writes'];bytes_map=t['byte_memory']
|
| 10 |
for c in t['calls']:
|
| 11 |
if c['target'] not in handlers:raise ValueError('Unrecovered trace child '+str(c['target']))
|
| 12 |
m,value,e=handlers[c['target']](ins,m,c['args'])
|
| 13 |
if ins[c['instruction']+1]['opcode']=='STORE4':
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
| 15 |
children.append({'call':c,'effect':e})
|
| 16 |
if t['trace'] and ins[t['trace'][-1]]['opcode']=='LEAVE':ret=t['stack'][-1];break
|
| 17 |
if pc==stop or pc in boundaries:break
|
|
|
|
| 2 |
from analyze_deferred_auth import trace_slice
|
| 3 |
from analyze_bsp_references import trace_fields
|
| 4 |
|
| 5 |
+
def resume(ins,start,stop,m,base,pc=None,handlers=None,boundaries=(),byte_inputs=None,limit=8192,result_offsets=None,repeated_calls=False):
|
| 6 |
handlers=handlers or {};pc=start if pc is None else pc;m=dict(m);trace=[];calls=[];children=[];writes=[];bytes_map=dict(byte_inputs or {});ret=None
|
| 7 |
points=[i['index']+2 for i in ins[start:stop] if i['opcode']=='CALL']+list(boundaries)
|
| 8 |
for _ in range(512):
|
| 9 |
+
t=trace_slice(ins,start,stop,m,{},resume_at=pc,stop_targets=points if repeated_calls else [p for p in points if p!=pc],local_base=base,byte_memory=bytes_map,halt_on_leave=True,capture_stack=True,limit=limit,skip_initial_stop=repeated_calls);m=t['memory'];pc=t.get('stop',stop);trace+=t['trace'];calls+=t['calls'];writes+=t['byte_writes'];bytes_map=t['byte_memory']
|
| 10 |
for c in t['calls']:
|
| 11 |
if c['target'] not in handlers:raise ValueError('Unrecovered trace child '+str(c['target']))
|
| 12 |
m,value,e=handlers[c['target']](ins,m,c['args'])
|
| 13 |
if ins[c['instruction']+1]['opcode']=='STORE4':
|
| 14 |
+
offset=(result_offsets or {}).get(c['instruction'])
|
| 15 |
+
if offset is None:
|
| 16 |
+
dest=ins[c['instruction']-2];assert dest['opcode']=='LOCAL';offset=dest['operand']
|
| 17 |
+
m[base+offset]=value
|
| 18 |
children.append({'call':c,'effect':e})
|
| 19 |
if t['trace'] and ins[t['trace'][-1]]['opcode']=='LEAVE':ret=t['stack'][-1];break
|
| 20 |
if pc==stop or pc in boundaries:break
|
scripts/test_qvm_rank_small_sort.py
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import functools,json,unittest
|
| 2 |
+
from pathlib import Path
|
| 3 |
+
from test_qvm_rank_comparator import compare
|
| 4 |
+
class SmallRankSortTests(unittest.TestCase):
|
| 5 |
+
@classmethod
|
| 6 |
+
def setUpClass(cls):cls.ps=json.loads(Path('docs/research/qvm-rank-small-sort.json').read_text())['probes']
|
| 7 |
+
def test_independent_stable_numeric_sort(self):
|
| 8 |
+
for p in self.ps:self.assertEqual(p['final'],sorted(p['initial'],key=functools.cmp_to_key(lambda a,b:compare(p['mode'],p['clients'][a],p['clients'][b]))))
|
| 9 |
+
def test_dynamic_result_patch_preserves_callback_pointer(self):
|
| 10 |
+
for p in self.ps:self.assertEqual(p['callback_pointer_after'],5441);self.assertEqual(p['execution']['stop'],36036);self.assertTrue(all(c['target']==5441 for c in p['execution']['calls']))
|
| 11 |
+
def test_unsigned_branch_boundaries(self):
|
| 12 |
+
from analyze_deferred_auth import trace_slice
|
| 13 |
+
for op in ['LTU','LEU','GTU','GEU']:
|
| 14 |
+
for a in [0,1,0x7fffffff,0x80000000,-1]:
|
| 15 |
+
for b in [0,1,0x7fffffff,0x80000000,-1]:
|
| 16 |
+
ins=[{'index':0,'opcode':'CONST','operand':a},{'index':1,'opcode':'CONST','operand':b},{'index':2,'opcode':op,'operand':5},{'index':3,'opcode':'CONST','operand':0},{'index':4,'opcode':'POP','operand':None}]
|
| 17 |
+
r=trace_slice(ins,0,5,{},{});x,y=a&0xffffffff,b&0xffffffff;branch={'LTU':x<y,'LEU':x<=y,'GTU':x>y,'GEU':x>=y}[op];self.assertEqual(3 not in r['trace'],branch,(op,a,b))
|
| 18 |
+
def test_domains_and_real_movement(self):
|
| 19 |
+
self.assertEqual(len(self.ps),44);self.assertTrue(any(p['final']!=p['initial'] for p in self.ps));self.assertEqual({len(p['initial']) for p in self.ps},{0,1,2,3,6})
|
| 20 |
+
if __name__=='__main__':unittest.main()
|
xtask/checks.tsv
CHANGED
|
@@ -1326,3 +1326,6 @@ edge docs/research/rust-recipient-payload.json scripts/test_rust_recipient_paylo
|
|
| 1326 |
report scripts/analyze_qvm_rank_comparator.py docs/research/qvm-rank-comparator.json
|
| 1327 |
edge docs/research/qvm-rank-comparator.json scripts/test_qvm_rank_comparator.py
|
| 1328 |
group rank-comparator scripts/analyze_qvm_rank_comparator.py
|
|
|
|
|
|
|
|
|
|
|
|
| 1326 |
report scripts/analyze_qvm_rank_comparator.py docs/research/qvm-rank-comparator.json
|
| 1327 |
edge docs/research/qvm-rank-comparator.json scripts/test_qvm_rank_comparator.py
|
| 1328 |
group rank-comparator scripts/analyze_qvm_rank_comparator.py
|
| 1329 |
+
report scripts/analyze_qvm_rank_small_sort.py docs/research/qvm-rank-small-sort.json
|
| 1330 |
+
edge docs/research/qvm-rank-small-sort.json scripts/test_qvm_rank_small_sort.py
|
| 1331 |
+
group rank-small-sort scripts/analyze_qvm_rank_small_sort.py
|