nmarshall commited on
Commit
e15838a
·
1 Parent(s): 01f2fa5

Analyze native QVM usercmd retrieval and game time prefix

Browse files
docs/ASSET_PIPELINE_TODO.md CHANGED
@@ -244,3 +244,5 @@ Research progress: [keyed usercmd codecs](research/MESSAGE_USERCMD.md) completes
244
  Research progress: [usercmd caller/hash/bundle evidence](research/USERCMD_CALLERS.md) completes bounded RE-397–399. RE-396/400 retain state producers, timestamp/VM acceptance, full bundle/netchan/runtime/platform and higher-level codecs; broad game/network/rendering milestones remain open.
245
 
246
  Research progress: [server usercmd acceptance](research/USERCMD_ACCEPTANCE.md) completes bounded RE-401–403. RE-400/404 retain producers/time clamping/QVM effects/full packet/runtime/platform work; game/network/rendering milestones remain open.
 
 
 
244
  Research progress: [usercmd caller/hash/bundle evidence](research/USERCMD_CALLERS.md) completes bounded RE-397–399. RE-396/400 retain state producers, timestamp/VM acceptance, full bundle/netchan/runtime/platform and higher-level codecs; broad game/network/rendering milestones remain open.
245
 
246
  Research progress: [server usercmd acceptance](research/USERCMD_ACCEPTANCE.md) completes bounded RE-401–403. RE-400/404 retain producers/time clamping/QVM effects/full packet/runtime/platform work; game/network/rendering milestones remain open.
247
+
248
+ Research progress: [native/QVM usercmd bridge](research/QVM_USERCMD.md) completes bounded RE-405–407. RE-404/408 retain producers/translation/VM effects/movement/full packet/runtime/platform contracts; broad game/network/rendering milestones remain open.
docs/FINDINGS.md CHANGED
@@ -462,3 +462,9 @@ Bounded qagame paths recover bomb-plant/inactivity correction requests, direct f
462
  **F-161 — Sequential signed command acceptance:** accepted time must be <= the bundle's final command time and > the stored previous command time; accepted records update all 24 bytes before dispatch. The cap is the final entry, not a computed maximum. Supplied negative/wrap behavior does not establish legal traffic or downstream time clamping. [ACCEPT-002](research/USERCMD_ACCEPTANCE.md).
463
 
464
  **F-162 — Native command copy and VM boundary:** standalone SV_ClientThink copies 24 bytes in all tested states and requests VM_Call(gvm,7,index) only in state4. Inlined accepted paths agree. Reciprocal arithmetic supports candidate client stride0x2df40 on aligned supplied addresses; allocation and actual callback effects remain unverified. [ACCEPT-003](research/USERCMD_ACCEPTANCE.md).
 
 
 
 
 
 
 
462
  **F-161 — Sequential signed command acceptance:** accepted time must be <= the bundle's final command time and > the stored previous command time; accepted records update all 24 bytes before dispatch. The cap is the final entry, not a computed maximum. Supplied negative/wrap behavior does not establish legal traffic or downstream time clamping. [ACCEPT-002](research/USERCMD_ACCEPTANCE.md).
463
 
464
  **F-162 — Native command copy and VM boundary:** standalone SV_ClientThink copies 24 bytes in all tested states and requests VM_Call(gvm,7,index) only in state4. Inlined accepted paths agree. Reciprocal arithmetic supports candidate client stride0x2df40 on aligned supplied addresses; allocation and actual callback effects remain unverified. [ACCEPT-003](research/USERCMD_ACCEPTANCE.md).
465
+
466
+ **F-163 — Native usercmd retrieval:** file-backed trap36 converts destination before signed index guards; valid paths copy 24 bytes at explicit client stride0x2df40 and return zero. Standalone SV_GetUsercmd agrees. Error calls stop; allocation/pointer/runtime contracts remain open. [QCMD-001](research/QVM_USERCMD.md).
467
+
468
+ **F-164 — Game selector7 bridge:** qagame case65 calls155718, which requests trap36(index, entity[+520]+472), stores game time at client+4044 and gates helper154089 on entity bit8/global34084868. Native copy results are mapped into synthetic QVM word memory; flag meanings/full runtime bridge remain open. [QCMD-002](research/QVM_USERCMD.md).
469
+
470
+ **F-165 — Game-side command time prefix:** helper154089 preserves raw time, sequentially clamps command time to signed game-time bounds -1000/+200, computes/caps elapsed independently and optionally adjusts timestamp in eight-unit steps afterward. Early returns retain preceding mutations; signed-boundary fixtures do not establish legal gameplay. [QCMD-003](research/QVM_USERCMD.md).
docs/PROVISIONAL_TODO.md CHANGED
@@ -556,3 +556,5 @@ Research progress: [keyed usercmd codecs](research/MESSAGE_USERCMD.md) completes
556
  Research progress: [usercmd caller/hash/bundle evidence](research/USERCMD_CALLERS.md) completes bounded RE-397–399. RE-396/400 retain state producers, timestamp/VM acceptance, full bundle/netchan/runtime/platform and higher-level codecs; broad game/network/rendering milestones remain open.
557
 
558
  Research progress: [server usercmd acceptance](research/USERCMD_ACCEPTANCE.md) completes bounded RE-401–403. RE-400/404 retain producers/time clamping/QVM effects/full packet/runtime/platform work; game/network/rendering milestones remain open.
 
 
 
556
  Research progress: [usercmd caller/hash/bundle evidence](research/USERCMD_CALLERS.md) completes bounded RE-397–399. RE-396/400 retain state producers, timestamp/VM acceptance, full bundle/netchan/runtime/platform and higher-level codecs; broad game/network/rendering milestones remain open.
557
 
558
  Research progress: [server usercmd acceptance](research/USERCMD_ACCEPTANCE.md) completes bounded RE-401–403. RE-400/404 retain producers/time clamping/QVM effects/full packet/runtime/platform work; game/network/rendering milestones remain open.
559
+
560
+ Research progress: [native/QVM usercmd bridge](research/QVM_USERCMD.md) completes bounded RE-405–407. RE-404/408 retain producers/translation/VM effects/movement/full packet/runtime/platform contracts; broad game/network/rendering milestones remain open.
docs/RESEARCH_LEDGER.md CHANGED
@@ -271,3 +271,9 @@ Validation: all preceding composition reports regenerated byte-identically; the
271
  Completed bounded RE-401–403 with `analyze_usercmd_acceptance.py`: four verified graphs across both Linux ELF64 engines, 48 post-decode state/purity gate probes, 184 timestamp-loop probes and 32 standalone SV_ClientThink probes. ACCEPT-001–003/F-160–162 distinguish external game-state/drop/enter-world requests from completed lifecycle effects, final-bundle signed timestamp caps from a computed maximum/server clock, sequential strict-newer filtering from decoder legality, and unconditional standalone 24-byte copies from state4-only VM dispatch. Every ordered command count1–32 and sampled duplicate/descending/unordered/stale/negative/sign-boundary cases match independent accepted-record/final-state oracles. Inlined loops produce 1,370 stopped VM_Call(gvm,7,index) requests; copied records are observed before the boundary. Native reciprocal arithmetic supports candidate stride0x2df40 only on aligned supplied addresses, pending allocation evidence. State/flags/decoded records are supplied; gate helpers stop and VM calls supply return0/retained caller registers with no VM side effects. No original executable/native callee/game VM, packet decode or runtime traffic executes. Eight tests verify gates/sentinel mutation, signed timestamps/all ordered counts, inlined VM arguments/copy order, standalone state behavior, reciprocal/stride arithmetic, canaries/command memory retention, paired outcomes and verified identities. All 415 tests pass; storage, links and whitespace checks pass. RE-400 remains open; RE-404 tracks purity/state producers/client allocation stride, command generation and server/game-side time clamping, selector7 QVM callback/command retrieval/VM mutation effects, complete decoded-bundle acceptance, real packet/runtime/all-platform conformance, higher-level codecs and Rust/Bevy/Solari goals.
272
 
273
  Validation: all preceding composition reports regenerated byte-identically; the new acceptance report reproduced twice byte-identically. Final suite: 415 passing tests. Storage policy/filter setup, documentation links and whitespace checks pass.
 
 
 
 
 
 
 
271
  Completed bounded RE-401–403 with `analyze_usercmd_acceptance.py`: four verified graphs across both Linux ELF64 engines, 48 post-decode state/purity gate probes, 184 timestamp-loop probes and 32 standalone SV_ClientThink probes. ACCEPT-001–003/F-160–162 distinguish external game-state/drop/enter-world requests from completed lifecycle effects, final-bundle signed timestamp caps from a computed maximum/server clock, sequential strict-newer filtering from decoder legality, and unconditional standalone 24-byte copies from state4-only VM dispatch. Every ordered command count1–32 and sampled duplicate/descending/unordered/stale/negative/sign-boundary cases match independent accepted-record/final-state oracles. Inlined loops produce 1,370 stopped VM_Call(gvm,7,index) requests; copied records are observed before the boundary. Native reciprocal arithmetic supports candidate stride0x2df40 only on aligned supplied addresses, pending allocation evidence. State/flags/decoded records are supplied; gate helpers stop and VM calls supply return0/retained caller registers with no VM side effects. No original executable/native callee/game VM, packet decode or runtime traffic executes. Eight tests verify gates/sentinel mutation, signed timestamps/all ordered counts, inlined VM arguments/copy order, standalone state behavior, reciprocal/stride arithmetic, canaries/command memory retention, paired outcomes and verified identities. All 415 tests pass; storage, links and whitespace checks pass. RE-400 remains open; RE-404 tracks purity/state producers/client allocation stride, command generation and server/game-side time clamping, selector7 QVM callback/command retrieval/VM mutation effects, complete decoded-bundle acceptance, real packet/runtime/all-platform conformance, higher-level codecs and Rust/Bevy/Solari goals.
272
 
273
  Validation: all preceding composition reports regenerated byte-identically; the new acceptance report reproduced twice byte-identically. Final suite: 415 passing tests. Storage policy/filter setup, documentation links and whitespace checks pass.
274
+
275
+ ## 2026-10-11 — Native retrieval, selector7 QVM bridge and game-side command time
276
+
277
+ Completed bounded RE-405–407 with `analyze_qvm_usercmd.py`: two verified native retrieval graphs, two file-backed trap36 cases, shipped qagame selector/table/case/helper evidence, 112 native guard/copy probes, 108 native-to-QVM wrapper compositions and 147 game-side time-prefix probes. QCMD-001–003/F-163–165 confirm explicit retrieval stride0x2df40, destination conversion before signed index errors, full 24-byte valid copies and zero EAX at the dispatch return jump; connect selector7/case65/helper155718/trap36 to client+472; recover metadata time and wrapper flag/global gates; and recover helper154089 state/raw-time/observed-time/clamp/elapsed/mode/alignment sequence. Native copy bytes map into synthetic QVM word memory under supplied pointer/address-space translation. Calls and Com_Error boundaries stop; no original executable/native callee/full QVM executes. The time prefix ends before remaining simulation, and signed-boundary arithmetic is modeled rather than runtime/legality evidence. Eight independent tests verify native signed guards/conversion ordering/copies/canaries/retention, switch slot/file identities and stride, selector/argument mapping, wrapper words/metadata/gates, signed time clamps/raw-time/elapsed/cap/rounding, paired outcomes, provenance and restricted optional slice-stop behavior. All 423 tests pass; storage/links/whitespace checks pass. RE-404 remains open; RE-408 tracks flag/global/state producers, allocation/translation, command generation and elapsed consumers, remaining movement/simulation and VM mutation effects, other trap36 callers, full packet-to-game/runtime/all-platform behavior and Rust/Bevy/Solari goals.
278
+
279
+ Validation: all preceding composition reports regenerated byte-identically; final QVM usercmd report reproduced twice byte-identically, including explicit negative-time rounding fixtures. Deferred-auth and engine-control reports also remain byte-identical after the optional slice-stop extension. Final suite: 423 passing tests. Storage policy/filter setup, documentation links and whitespace checks pass.
docs/REVERSE_ENGINEERING_TODO.md CHANGED
@@ -487,3 +487,8 @@ Purpose: turn the supplied installation into an evidence-backed specification of
487
  - [x] RE-402 Recover bounded signed sequential command-time filtering against final-bundle time and stored previous time; verify full accepted records for all ordered counts and duplicate/stale/unordered/negative/boundary fixtures. [ACCEPT-002](research/USERCMD_ACCEPTANCE.md).
488
  - [x] RE-403 Recover standalone/inlined 24-byte client command copies, state-4-only VM selector7 requests and candidate stride/index arithmetic with independent stopped-boundary checks. [ACCEPT-003](research/USERCMD_ACCEPTANCE.md).
489
  - [ ] RE-404 Confirm purity/state-field producers and client allocation stride, command generation/server/game time clamping, selector7 QVM callback and command retrieval, VM mutation effects; compose decoded bundles through acceptance and validate real traffic/all-platform behavior before Rust compatibility claims. Continue entity/player/snapshot and rendering work.
 
 
 
 
 
 
487
  - [x] RE-402 Recover bounded signed sequential command-time filtering against final-bundle time and stored previous time; verify full accepted records for all ordered counts and duplicate/stale/unordered/negative/boundary fixtures. [ACCEPT-002](research/USERCMD_ACCEPTANCE.md).
488
  - [x] RE-403 Recover standalone/inlined 24-byte client command copies, state-4-only VM selector7 requests and candidate stride/index arithmetic with independent stopped-boundary checks. [ACCEPT-003](research/USERCMD_ACCEPTANCE.md).
489
  - [ ] RE-404 Confirm purity/state-field producers and client allocation stride, command generation/server/game time clamping, selector7 QVM callback and command retrieval, VM mutation effects; compose decoded bundles through acceptance and validate real traffic/all-platform behavior before Rust compatibility claims. Continue entity/player/snapshot and rendering work.
490
+
491
+ - [x] RE-405 Recover bounded native trap36/standalone usercmd retrieval, signed guards, pointer-conversion ordering, explicit client stride and 24-byte copy/valid dispatch return. [QCMD-001](research/QVM_USERCMD.md).
492
+ - [x] RE-406 Connect shipped qagame selector7 to helper155718/trap36, entity/client command placement, metadata time and think gates using supplied native/QVM address mapping. [QCMD-002](research/QVM_USERCMD.md).
493
+ - [x] RE-407 Recover bounded helper154089 state guard, raw/observed timestamps, sequential signed clamps, elapsed early-return/cap and eight-unit timestamp adjustment with independent arithmetic probes. [QCMD-003](research/QVM_USERCMD.md).
494
+ - [ ] RE-408 Confirm wrapper/prefix flag/global/state meanings and producers; trace input generation, elapsed/timestamp consumers and remaining movement/simulation; audit trap36 callers/VM mutation/address translation, compose full packet-to-game processing and validate real traffic/all-platform behavior. Continue entity/player/snapshot and Rust/Bevy/Solari implementation.
docs/research/QVM_USERCMD.md ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Native usercmd retrieval and game-QVM time prefix
2
+
3
+ RE-405–407 / F-163–165. [qvm-usercmd.json](qvm-usercmd.json) retains verified Linux ELF64 retrieval graphs/trap cases and shipped qagame instruction evidence. Reproduce with `python3 scripts/analyze_qvm_usercmd.py`; verify with `python3 scripts/test_qvm_usercmd.py`.
4
+
5
+ This extends [server acceptance and selector 7](USERCMD_ACCEPTANCE.md). Original executables, native callees and the complete QVM never execute. VM_ArgPtr results and cross-address-space mapping are supplied. Native retrieval instructions copy bytes, which are explicitly mapped into the synthetic QVM word memory; this is a bounded composition, not a running engine/VM bridge. Calls in QVM entry/wrapper slices are recorded. The time prefix stops before remaining simulation.
6
+
7
+ ## QCMD-001 — Trap 36, index guards and native record copy (RE-405)
8
+
9
+ The file-backed SV_GameSystemCalls switch slot 36 resolves to dedicated `0x40b7c8` / client-engine `0x473738`. The report verifies the slot's little-endian pointer against the source file and retains its file offset and case instructions. This case first converts argument slot 2 through VM_ArgPtr, then treats slot 1's low 32 bits as a signed client index.
10
+
11
+ An index below zero or at least the signed sv_maxclients integer reaches `Com_Error(1, format, index)`. The analysis stops at that call and does not assume normal continuation; the native graph's syntactic fallthrough is not an error-recovery contract. Pointer conversion occurs before this guard even on the invalid-index paths. Translation validity, pointer masking/bounds and actual error unwinding remain open.
12
+
13
+ Valid paths read the client-array base from svs+0x10, multiply the sign-extended index by **0x2df40**, and copy three 64-bit words from client +0x2081c/+0x20824/+0x2082c to the converted destination. The trap case sets return EAX to zero before its common epilogue jump. The standalone `SV_GetUsercmd` performs the same signed guards, explicit stride multiplication and 24-byte copy; it is not called by the inlined trap case.
14
+
15
+ This independently confirms the earlier arithmetic **candidate stride as an explicit access stride** in these two native paths. Actual array allocation, valid slot lifetime and maximum-client policy still require recovery.
16
+
17
+ 112 probes total (28 standalone and 28 trap cases per engine) cover indices -1/0/1/7/63/64/0x7fffffff against maxima 0/1/8/64. Independent tests verify guards, conversion requests, invalid destination retention, exact 24-byte copies, canaries and valid trap return zero. No Com_Error or VM_ArgPtr callee executes.
18
+
19
+ ## QCMD-002 — Selector 7 wrapper and command placement (RE-406)
20
+
21
+ The shipped qagame entry dispatch reads jump-table word at VM address 3564 and selects instruction 65 for selector 7. The case loads incoming local 48, forwards it in argument slot 8 and calls helper **155718**, then returns zero. This matches the native selector/client-index boundary recovered previously; the local/table mapping is checked against the source QVM and existing ABI report.
22
+
23
+ Helper 155718 computes `entity = 34091368 + 880*index`, loads its client pointer from entity +520, and records trap target **-37** (trap 36) with `(index, client_pointer+472)`. Thus the native 24-byte stored usercmd is fetched into the game client record at +472. Entity indexing and pointer validity are supplied, not checked by this wrapper slice.
24
+
25
+ After retrieval, the helper writes global time word 34992532 into client +4044. It calls helper **154089** with the entity pointer only when `(entity[+424] & 8) == 0` and global word **34084868 == 0**. Bit-8 and global-gate producer semantics remain open; names such as bot/synchronous-client mode require independent confirmation. Retrieval and callback-time storage happen even when those gates suppress the think call.
26
+
27
+ 108 wrapper compositions (54 per engine) cover indices 0/1/63, flag words 0/8/9, gate words 0/1/-1 and time words 1000/0xffffffff. Native copy results become six QVM words under explicitly supplied address mapping. Independent checks verify trap arguments, every copied word, metadata time and exact think-call gating. Native client storage and QVM game-client storage have different bases/layouts; they must remain separate in a Rust port.
28
+
29
+ ## QCMD-003 — Game-side time clamp, elapsed gate and alignment (RE-407)
30
+
31
+ Helper 154089 begins with a 436-byte QVM frame. The analyzed prefix (154090 through boundary 154210) loads entity +520 and requires client[+468] ==2; other sampled states jump to the return tail at 155716 without this prefix's time mutations.
32
+
33
+ For state 2, observed operations are sequential:
34
+
35
+ 1. Write game time 34992532 to client +6172.
36
+ 2. Preserve the raw command time from client +472 at client +6176.
37
+ 3. Clamp command time downward when it exceeds signed32(game_time+200), then upward when it is below signed32(game_time-1000).
38
+ 4. Compute signed32(command_time-client[+0]) as elapsed time.
39
+ 5. If elapsed <1 and client[+3108] !=2, return after the preceding time stores/clamp.
40
+ 6. Cap elapsed at 200. This modifies local 328, independently of the command timestamp.
41
+ 7. If client[+712] is nonzero, set command time to signed32(trunc_toward_zero(signed32(command_time+7)/8)*8).
42
+
43
+ The eight-unit adjustment occurs **after** elapsed calculation and its cap; this prefix does not recompute elapsed or reapply the clamp after rounding. For ordinary positive times this rounds upward to a multiple of eight. Negative/boundary inputs require the recovered signed/truncating expression rather than a generic ceiling formula. Client +0 is provisionally the player-state command time; the +3108 mode and +712 alignment flag still need producer/consumer confirmation.
44
+
45
+ 147 probes cover three rejecting states, 128 ordinary combinations of command time/prior time/mode/alignment, four negative-time rounding fixtures, and twelve signed-boundary combinations near 0x7fffffff/0x80000000. Independent modular arithmetic predicts command time, raw/observed time stores, elapsed and early return. Overflow in the game-time bounds can produce unusual sequential-clamp behavior in synthetic boundary fixtures. These are modeled instruction results, not evidence of legitimate clock wrapping or runtime parity.
46
+
47
+ The evaluator adds an optional explicit stop-target boundary, allowing state guards and the continuing prefix to stop before the remaining helper. Existing callers keep their original behavior/output. A focused test proves that an unsupported boundary instruction is not executed and still fails when the optional stop is absent.
48
+
49
+ ## Validation and remaining work
50
+
51
+ Eight tests verify native signed guards/pointer ordering/copy retention, file-backed trap slot and explicit stride, selector/table/argument mapping, wrapper copy/metadata/gates, game-side signed clamps/elapsed/mode/alignment, paired results, source identities and optional stop behavior.
52
+
53
+ RE-404 remains open for complete callback/producer/allocation/runtime work. RE-408 must confirm the global/record flag meanings, recover command generation and all timestamp/elapsed consumers, continue helper154089 into movement/simulation, characterize selector7 trap/VM mutation effects and address translation, audit other trap36 callers, compose packets through native acceptance/retrieval/game-side processing and validate real traffic/all-platform behavior. Full entity/player/snapshot codecs and Rust/Bevy/Solari implementation remain open.
docs/research/README.md CHANGED
@@ -157,3 +157,5 @@ The extension collector adds five focused tests for native tail calls, command-r
157
  [Usercmd caller keys and bundles](USERCMD_CALLERS.md) adds RE-397–399. Reproduce with `python3 scripts/analyze_usercmd_callers.py`; check with `python3 scripts/test_usercmd_callers.py`. Hash instructions are evaluated; caller hash returns and server codec effects are supplied. Full packet/runtime conformance remains open.
158
 
159
  [Server usercmd acceptance and VM boundary](USERCMD_ACCEPTANCE.md) adds RE-401–403. Reproduce with `python3 scripts/analyze_usercmd_acceptance.py`; check with `python3 scripts/test_usercmd_acceptance.py`. Supplied state/decoded records and stopped external/VM boundaries do not validate live packets or game simulation.
 
 
 
157
  [Usercmd caller keys and bundles](USERCMD_CALLERS.md) adds RE-397–399. Reproduce with `python3 scripts/analyze_usercmd_callers.py`; check with `python3 scripts/test_usercmd_callers.py`. Hash instructions are evaluated; caller hash returns and server codec effects are supplied. Full packet/runtime conformance remains open.
158
 
159
  [Server usercmd acceptance and VM boundary](USERCMD_ACCEPTANCE.md) adds RE-401–403. Reproduce with `python3 scripts/analyze_usercmd_acceptance.py`; check with `python3 scripts/test_usercmd_acceptance.py`. Supplied state/decoded records and stopped external/VM boundaries do not validate live packets or game simulation.
160
+
161
+ [Native retrieval and game-QVM usercmd prefix](QVM_USERCMD.md) adds RE-405–407. Reproduce with `python3 scripts/analyze_qvm_usercmd.py`; check with `python3 scripts/test_qvm_usercmd.py`. Cross-address mapping is supplied and remaining VM/simulation/runtime effects are open.
docs/research/USERCMD_ACCEPTANCE.md CHANGED
@@ -50,3 +50,5 @@ Index arithmetic subtracts the file-backed `svs+0x10` client-array pointer, arit
50
  Eight tests verify connection/purity gates and sentinel mutation, signed sequential timestamps and final-bundle cap, every ordered count, record copy before VM requests, standalone state behavior, reciprocal/stride arithmetic, canaries/source retention, paired engine outcomes and source/disassembly identities. The report records instruction graphs, trace lengths/hashes, complete supplied/accepted records and requested VM arguments.
51
 
52
  RE-400 remains open for complete producer and packet/runtime work. RE-404 must confirm purity/state-field meanings and client stride through producers/allocation, trace command generation and server/game-side time clamping, connect selector 7 through the QVM callback and command-retrieval syscall, compose actual decoded bundles with acceptance, characterize VM mutation effects and validate real traffic/all-platform behavior. Higher-level entity/player/snapshot codecs, Rust compatibility and Bevy/Solari rendering remain open.
 
 
 
50
  Eight tests verify connection/purity gates and sentinel mutation, signed sequential timestamps and final-bundle cap, every ordered count, record copy before VM requests, standalone state behavior, reciprocal/stride arithmetic, canaries/source retention, paired engine outcomes and source/disassembly identities. The report records instruction graphs, trace lengths/hashes, complete supplied/accepted records and requested VM arguments.
51
 
52
  RE-400 remains open for complete producer and packet/runtime work. RE-404 must confirm purity/state-field meanings and client stride through producers/allocation, trace command generation and server/game-side time clamping, connect selector 7 through the QVM callback and command-retrieval syscall, compose actual decoded bundles with acceptance, characterize VM mutation effects and validate real traffic/all-platform behavior. Higher-level entity/player/snapshot codecs, Rust compatibility and Bevy/Solari rendering remain open.
53
+
54
+ Follow-up: [native/QVM retrieval and time prefix](QVM_USERCMD.md) confirms explicit stride0x2df40 in retrieval paths, connects selector7/trap36 and recovers bounded game-side time clamping. Allocation, producer domains, address translation, remaining callback effects and runtime validation remain open.
docs/research/qvm-usercmd.json ADDED
The diff for this file is too large to render. See raw diff
 
scripts/analyze_deferred_auth.py CHANGED
@@ -15,12 +15,14 @@ def signed32(value):
15
 
16
 
17
  def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_memory=None, capture_stack=False,
18
- halt_on_leave=False):
19
  """Evaluate only a selected straight/bounded slice against synthetic memory.
20
 
21
  Record calls rather than entering callees. Reject unknown reads/opcodes and
22
  control flow outside the selected slice. This is not a QVM runtime.
23
  """
 
 
24
  stack, args, calls, trace, byte_writes = [], {}, [], [], []
25
  memory = dict(memory)
26
  bytes_map = None if byte_memory is None else dict(byte_memory)
@@ -28,7 +30,7 @@ def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_m
28
  for offset, value in locals_map.items():
29
  memory[local_base+offset] = value
30
  pc = start
31
- while pc != stop:
32
  if not start <= pc < stop or len(trace) >= limit:
33
  raise ValueError('slice control flow escaped bound')
34
  ins = instructions[pc]
@@ -106,6 +108,8 @@ def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_m
106
  result['byte_memory'] = bytes_map
107
  if capture_stack:
108
  result['stack'] = stack
 
 
109
  return result
110
 
111
 
 
15
 
16
 
17
  def trace_slice(instructions, start, stop, memory, locals_map, limit=256, byte_memory=None, capture_stack=False,
18
+ halt_on_leave=False, stop_targets=()):
19
  """Evaluate only a selected straight/bounded slice against synthetic memory.
20
 
21
  Record calls rather than entering callees. Reject unknown reads/opcodes and
22
  control flow outside the selected slice. This is not a QVM runtime.
23
  """
24
+ if any(not start <= target <= stop for target in stop_targets):
25
+ raise ValueError('stop target outside bounded slice')
26
  stack, args, calls, trace, byte_writes = [], {}, [], [], []
27
  memory = dict(memory)
28
  bytes_map = None if byte_memory is None else dict(byte_memory)
 
30
  for offset, value in locals_map.items():
31
  memory[local_base+offset] = value
32
  pc = start
33
+ while pc != stop and pc not in stop_targets:
34
  if not start <= pc < stop or len(trace) >= limit:
35
  raise ValueError('slice control flow escaped bound')
36
  ins = instructions[pc]
 
108
  result['byte_memory'] = bytes_map
109
  if capture_stack:
110
  result['stack'] = stack
111
+ if stop_targets:
112
+ result['stop'] = pc
113
  return result
114
 
115
 
scripts/analyze_qvm_usercmd.py ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Bounded native usercmd retrieval, selector7 bridge and QVM time prefix."""
3
+ import gzip
4
+ import itertools
5
+ import json
6
+ import struct
7
+ from pathlib import Path
8
+ from analyze_bsp_references import global_address,operand,trace_fields
9
+ from analyze_collision_composition import get_word,put_word
10
+ from analyze_deferred_auth import partition_function,signed32,trace_slice
11
+ from analyze_engine_control import control_flow
12
+ from analyze_message_widths import run
13
+ from analyze_native_dispatch import recover
14
+ from analyze_qvm import decode,write_json
15
+ from analyze_vm_abi import entry_dispatch
16
+ from analyze_vm_extensions import native_functions,verified
17
+
18
+ CLIENT_BASE=0x500000
19
+ DEST=0xc0000
20
+ ARGS=0xd0000
21
+ CVAR=0xe0000
22
+ ENTITY_BASE=34091368
23
+ QCLIENT=0x10000
24
+
25
+
26
+ def native_probe(g,index,maxclients,dispatch=False,start=None):
27
+ rows=g['instructions'];start=start if dispatch else next(x['address'] for x in rows if x['opcode']=='testl' and operand(x)=='%edi, %edi');end=next(x['address'] for x in rows if x['address']>start and (x['opcode']=='jmp' if dispatch else x['opcode']=='movq' and operand(x)=='0x8(%rsp), %rbx'));calls={x['address']:i for i,x in enumerate(rows) if x['opcode']=='callq'};m={};record=(1234).to_bytes(4,'little')+bytes(range(20));client=CLIENT_BASE+index*0x2df40
28
+ for x in rows:
29
+ if '<sv_maxclients>' in x['operands']:put_word(m,global_address(x),CVAR,8)
30
+ if '<svs+0x10>' in x['operands']:put_word(m,global_address(x),CLIENT_BASE,8)
31
+ put_word(m,CVAR+48,maxclients)
32
+ for j,v in enumerate(record):m[client+0x2081c+j]=v
33
+ for j in range(-1,25):m[DEST+j]=0xa5
34
+ put_word(m,ARGS+8,index,8);put_word(m,ARGS+16,0x1234,8);regs={'%rdi':ARGS if dispatch else index&0xffffffff,'%rsi':DEST,'%rbp':ARGS,'%rsp':0xf0000};pc=start;trace=[];pointer_request=None;error=None
35
+ while True:
36
+ t=run(g,pc,list(calls)+[end],m,regs);m=t['memory'];regs=t['registers'];trace+=t['trace']
37
+ if t['stop']==end:break
38
+ i=calls[t['stop']];target=rows[i]['operands']
39
+ if '<VM_ArgPtr>' in target:pointer_request=regs['rdi'];regs['rax']=DEST
40
+ elif '<Com_Error>' in target:error=[regs[k] for k in ['rdi','rsi','rdx']];break
41
+ else:raise ValueError('retrieval boundary '+target)
42
+ pc=rows[i+1]['address']
43
+ return {'index_word':index&0xffffffff,'maxclients_word':maxclients&0xffffffff,'source_hex':record.hex(),'destination_hex':bytes(m[DEST+j] for j in range(24)).hex(),'canaries':[m[DEST-1],m[DEST+24]],'pointer_request':pointer_request,'supplied_pointer_result':DEST if dispatch else None,'error_arguments':error,'completed':error is None,'dispatch_return_word':regs['rax']&0xffffffff if dispatch and error is None else None,**trace_fields({'trace':trace},False)}
44
+
45
+
46
+ def wrapper_probe(ins,index,flags,sync,now,native_graph):
47
+ entity=ENTITY_BASE+880*index;m={entity+520:QCLIENT,entity+424:flags,34084868:sync,34992532:signed32(now)}
48
+ pre=trace_slice(ins,155719,155741,m,{28:index});trap=pre['calls'];native=native_probe(native_graph,index,64);record=bytes.fromhex(native['destination_hex']);dest=trap[0]['args'][12]
49
+ # Native byte-copy result mapped into the QVM word memory; pointer translation
50
+ # and engine/VM address-space mapping are supplied, not a running bridge.
51
+ m=pre['memory']
52
+ for j in range(0,24,4):m[dest+j]=signed32(int.from_bytes(record[j:j+4],'little'))
53
+ post=trace_slice(ins,155741,155770,m,{16:entity});return {'index':index,'entity_flags_word':flags,'sync_word':sync,'now_word':now&0xffffffff,'entity_address':entity,'client_pointer':QCLIENT,'retrieval_calls':trap,'native_retrieval':native,'qvm_command_address':dest,'command_words':[post['memory'][dest+j]&0xffffffff for j in range(0,24,4)],'last_callback_time_word':post['memory'][QCLIENT+4044]&0xffffffff,'think_calls':post['calls'],**trace_fields({'trace':pre['trace']+post['trace']},False)}
54
+
55
+
56
+ def time_probe(ins,state,now,cmd,previous,mode,rounding):
57
+ m={0x20000+520:QCLIENT,QCLIENT+468:state,34992532:signed32(now),QCLIENT+472:signed32(cmd),QCLIENT:signed32(previous),QCLIENT+3108:mode,QCLIENT+712:rounding};t=trace_slice(ins,154090,155716,m,{444:0x20000},stop_targets=[154210],capture_stack=True,limit=512);mem=t['memory'];reached=t['stop']==154210
58
+ return {'state':state,'now_word':now&0xffffffff,'command_word':cmd&0xffffffff,'previous_word':previous&0xffffffff,'mode_word':mode,'rounding_word':rounding,'continued':reached,'result_command_word':mem[QCLIENT+472]&0xffffffff,'observed_time_word':mem.get(QCLIENT+6172),'raw_command_time_word':mem.get(QCLIENT+6176),'elapsed_word':mem.get(0x40000000+328),'calls':t['calls'],**trace_fields(t,False)}
59
+
60
+
61
+ def main():
62
+ qr=json.loads(Path('docs/research/qvm/qagame.json').read_text());data=verified(Path(qr['source']['member']['output']).read_bytes(),qr['sha256']);header,ins=decode(data);dispatch=entry_dispatch(ins,data,header,7);abi=json.loads(Path('docs/research/qvm/qagame.abi-candidates.json').read_text());observed=next(x for x in abi['entry_dispatch'] if x['command']==7)
63
+ if abi['sha256']!=qr['sha256'] or any(observed[k]!=dispatch[k] for k in dispatch):raise ValueError('selector provenance')
64
+ out={'native':{},'qvm':{'source':qr['source'],'sha256':qr['sha256'],'selector_dispatch':dispatch,'entry_case':ins[65:73],'wrapper':partition_function(ins,155718,data,header),'think_prefix':ins[154089:154210]},'limitations':'Synthetic stopped native/QVM instruction slices only; no original executable/native callee/full QVM executes. VM_ArgPtr results and native-to-QVM address mapping supplied; native retrieval byte effects composed into QVM word memory only. Com_Error stops with no normal continuation. Entry/wrapper/think calls are recorded, not executed. QVM prefix stops before remaining simulation and callback effects; no legal producer/time/packet/runtime/all-platform conformance. Arithmetic is explicit signed32 modular/truncating model, not runtime validation.'}
65
+ for label in ['linux--Quake3-UrT-Ded.x86_64','linux--Quake3-UrT.x86_64']:
66
+ r=json.loads(Path(f'docs/research/native/{label}.json').read_text());raw=verified((Path('.cache/UrbanTerror43')/r['source']['path']).read_bytes(),r['source']['sha256']);a=r['artifacts']['disassembly.txt'];text=verified(gzip.open(a['path'],'rb').read(),a['uncompressed_sha256']).decode();fs=native_functions(text);g=control_flow(fs['SV_GetUsercmd']);sg=control_flow(fs['SV_GameSystemCalls']);d=recover(raw,text,'SV_GameSystemCalls');case=next(c for c in d['cases'] if 36 in c['numbers_sharing_target']);start=case['target_address'];end=next(x['address'] for x in sg['instructions'] if x['address']>start and x['opcode']=='jmp');table_offset=d['table_file_offset']+36*8;slot=raw[table_offset:table_offset+8]
67
+ if struct.unpack('<Q',slot)[0]!=start:raise ValueError('native trap table')
68
+ out['native'][label]={'source':r['source'],'disassembly_sha256':a['uncompressed_sha256'],'retrieval_graph':g,'trap36_table':{'table_address':d['table_address'],'slot_file_offset':table_offset,'slot_hex':slot.hex(),'target_address':start},'trap36_instructions':[x for x in sg['instructions'] if start<=x['address']<=end],'retrieval_probes':[native_probe(g,i,n) for i,n in itertools.product([-1,0,1,7,63,64,0x7fffffff],[0,1,8,64])],'dispatch_probes':[native_probe(sg,i,n,True,start) for i,n in itertools.product([-1,0,1,7,63,64,0x7fffffff],[0,1,8,64])],'wrapper_probes':[wrapper_probe(ins,*v,g) for v in itertools.product([0,1,63],[0,8,9],[0,1,-1],[1000,0xffffffff])]}
69
+ fixtures=[(state,1000,1001,1000,0,0) for state in [0,1,3]]+[(2,1000,cmd,previous,mode,rounding) for cmd,previous,mode,rounding in itertools.product([-1,0,999,1000,1001,1200,1201,3000],[0,999,1000,1200],[0,2],[0,1])]+[(2,now,cmd,previous,0,rounding) for now,cmd,previous,rounding in itertools.product([0x7fffffff,0x80000000],[0,0x7fffffff,0x80000000],[0],[0,1])]
70
+ fixtures += [(2,-100,cmd,-200,0,1) for cmd in [-100,-99,-108,-107]]
71
+ out['qvm']['time_probes']=[time_probe(ins,*v) for v in fixtures];write_json(Path('docs/research/qvm-usercmd.json'),out);print('Recovered selector7 native retrieval bridge and bounded QVM time prefix')
72
+
73
+
74
+ if __name__=='__main__':main()
scripts/test_qvm_usercmd.py ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Independent native guard/copy, selector bridge and signed time oracles."""
3
+ import hashlib
4
+ import json
5
+ from pathlib import Path
6
+ import struct
7
+ import unittest
8
+ from analyze_deferred_auth import trace_slice
9
+
10
+
11
+ def s32(v):
12
+ v &=0xffffffff;return v if v<2**31 else v-2**32
13
+
14
+
15
+ def time_oracle(p):
16
+ cmd=s32(p['command_word']);now=s32(p['now_word']);prev=s32(p['previous_word'])
17
+ if p['state']!=2:return False,cmd,None,None,None
18
+ raw=cmd
19
+ if cmd>s32(now+200):cmd=s32(now+200)
20
+ if cmd<s32(now-1000):cmd=s32(now-1000)
21
+ elapsed=s32(cmd-prev)
22
+ if elapsed<1 and p['mode_word']!=2:return False,cmd,now,raw,elapsed
23
+ elapsed=min(elapsed,200)
24
+ if p['rounding_word']:
25
+ v=s32(cmd+7);quotient=abs(v)//8;quotient=-quotient if v<0 else quotient;cmd=s32(quotient*8)
26
+ return True,cmd,now,raw,elapsed
27
+
28
+
29
+ class QvmUsercmdTests(unittest.TestCase):
30
+ @classmethod
31
+ def setUpClass(cls):cls.r=json.loads(Path('docs/research/qvm-usercmd.json').read_text())
32
+
33
+ def test_native_signed_index_guards_and_translation_before_error(self):
34
+ for n in self.r['native'].values():
35
+ for kind in ['retrieval_probes','dispatch_probes']:
36
+ self.assertEqual(len(n[kind]),28)
37
+ for p in n[kind]:
38
+ valid=0<=s32(p['index_word'])<s32(p['maxclients_word']);self.assertEqual(p['completed'],valid);self.assertEqual(p['error_arguments'] is None,valid)
39
+ if not valid:self.assertEqual(p['error_arguments'][0],1);self.assertEqual(p['error_arguments'][2]&0xffffffff,p['index_word'])
40
+ if kind=='dispatch_probes':self.assertEqual(p['pointer_request'],0x1234);self.assertEqual(p['supplied_pointer_result'],0xc0000);self.assertEqual(p['dispatch_return_word'],0 if valid else None)
41
+
42
+ def test_native_full_copy_and_invalid_destination_retention(self):
43
+ for n in self.r['native'].values():
44
+ for kind in ['retrieval_probes','dispatch_probes']:
45
+ for p in n[kind]:self.assertEqual(p['destination_hex'],p['source_hex'] if p['completed'] else bytes([0xa5]*24).hex());self.assertEqual(p['canaries'],[0xa5]*2)
46
+
47
+ def test_table_file_provenance_explicit_stride_and_qvm_selector(self):
48
+ for n in self.r['native'].values():
49
+ p=n['trap36_table'];raw=(Path('.cache/UrbanTerror43')/n['source']['path']).read_bytes();self.assertEqual(raw[p['slot_file_offset']:p['slot_file_offset']+8],bytes.fromhex(p['slot_hex']));self.assertEqual(struct.unpack('<Q',bytes.fromhex(p['slot_hex']))[0],p['target_address'])
50
+ for rows in [n['retrieval_graph']['instructions'],n['trap36_instructions']]:self.assertTrue(any(x['opcode']=='imulq' and '$0x2df40,' in x['operands'] for x in rows))
51
+ q=self.r['qvm'];self.assertEqual(q['selector_dispatch']['case_instruction'],65);self.assertEqual(q['selector_dispatch']['table_reads'],[{'value':65,'vm_address':3564}]);self.assertEqual([i['opcode'] for i in q['entry_case']],['LOCAL','LOAD4','ARG','CONST','CALL','POP','CONST','LEAVE']);self.assertEqual(q['entry_case'][3]['operand'],155718)
52
+
53
+ def test_qvm_wrapper_retrieval_mapping_metadata_and_think_gates(self):
54
+ for n in self.r['native'].values():
55
+ self.assertEqual(len(n['wrapper_probes']),54)
56
+ for p in n['wrapper_probes']:
57
+ self.assertEqual(p['entity_address'],34091368+880*p['index']);self.assertEqual(p['qvm_command_address'],p['client_pointer']+472);call=p['retrieval_calls'][0];self.assertEqual(call['target'],-37);self.assertEqual(call['args'],{'8':p['index'],'12':p['client_pointer']+472});raw=bytes.fromhex(p['native_retrieval']['source_hex']);self.assertEqual(p['command_words'],list(struct.unpack('<6I',raw)));self.assertEqual(p['last_callback_time_word'],p['now_word']);expected=[] if p['entity_flags_word']&8 or p['sync_word']!=0 else [{'instruction':155768,'target':154089,'args':{'8':p['entity_address']}}];self.assertEqual(p['think_calls'],expected)
58
+
59
+ def test_qvm_time_clamp_raw_time_elapsed_exception_and_rounding(self):
60
+ rows=self.r['qvm']['time_probes'];self.assertEqual(len(rows),147)
61
+ for p in rows:
62
+ continued,cmd,now,raw,elapsed=time_oracle(p);self.assertEqual(p['continued'],continued);self.assertEqual(p['result_command_word'],cmd&0xffffffff);self.assertEqual(p['observed_time_word'],now);self.assertEqual(p['raw_command_time_word'],raw);self.assertEqual(p['elapsed_word'],elapsed);self.assertEqual(p['calls'],[])
63
+
64
+ def test_paired_native_and_wrapper_outcomes(self):
65
+ a,b=self.r['native'].values()
66
+ def clean(rows):
67
+ out=[]
68
+ for p in rows:
69
+ p={k:v for k,v in p.items() if k not in ['trace_sha256','trace_length']}
70
+ if p.get('error_arguments'):p['error_arguments']=[p['error_arguments'][0],p['error_arguments'][2]]
71
+ if 'native_retrieval' in p:p['native_retrieval']=clean([p['native_retrieval']])[0]
72
+ out.append(p)
73
+ return out
74
+ for kind in ['retrieval_probes','dispatch_probes','wrapper_probes']:self.assertEqual(clean(a[kind]),clean(b[kind]))
75
+
76
+ def test_source_identities(self):
77
+ q=self.r['qvm'];source=Path(json.loads(Path('docs/research/qvm/qagame.json').read_text())['source']['member']['output']);self.assertEqual(hashlib.sha256(source.read_bytes()).hexdigest(),q['sha256'])
78
+ for label,n in self.r['native'].items():
79
+ raw=(Path('.cache/UrbanTerror43')/n['source']['path']).read_bytes();self.assertEqual(hashlib.sha256(raw).hexdigest(),n['source']['sha256']);self.assertEqual(n['disassembly_sha256'],json.loads(Path('docs/research/native',label+'.json').read_text())['artifacts']['disassembly.txt']['uncompressed_sha256'])
80
+
81
+ def test_optional_slice_stop_does_not_execute_boundary(self):
82
+ ins=[{'index':0,'opcode':'CONST','operand':7},{'index':1,'opcode':'ARG','operand':8},{'index':2,'opcode':'BREAK','operand':None}];r=trace_slice(ins,0,3,{}, {},stop_targets=[2],capture_stack=True);self.assertEqual(r['stop'],2);self.assertEqual(r['trace'],[0,1]);self.assertEqual(r['calls'],[])
83
+ with self.assertRaisesRegex(ValueError,'unsupported slice opcode'):trace_slice(ins,0,3,{}, {})
84
+ with self.assertRaisesRegex(ValueError,'stop target outside bounded slice'):trace_slice(ins,0,3,{}, {},stop_targets=[4])
85
+
86
+
87
+ if __name__=='__main__':unittest.main()