nmarshall commited on
Commit
66138c3
·
1 Parent(s): 010acde

Recover Linux frame timing event dispatch and pause gates

Browse files
docs/FINDINGS.md CHANGED
@@ -202,3 +202,11 @@ Exact native fork/source revision, runtime resource lookup, legacy launch enviro
202
  **F-037 — Observed: error frame-abort versus process-exit paths.** Linux mode-1 Com_Error ordinarily shuts down/disconnects and longjmps to the frame boundary. Build-script, close-spaced repeated errors and recursion can select process-exit paths through Sys_Error/_exit(1). Complete cleanup/platform parity remain open. [CTL-002](research/ENGINE_CONTROL.md).
203
 
204
  **F-038 — Observed: authentication announcement helper.** The begin callback's field-3200 gate sets a one-shot value before invoking helper 165352, whose formats and trap-17 calls announce authentication/account status through print/chat commands. Recipient/mode/state-update semantics and permission rules are not yet specified. [CTL-003](research/ENGINE_CONTROL.md).
 
 
 
 
 
 
 
 
 
202
  **F-037 — Observed: error frame-abort versus process-exit paths.** Linux mode-1 Com_Error ordinarily shuts down/disconnects and longjmps to the frame boundary. Build-script, close-spaced repeated errors and recursion can select process-exit paths through Sys_Error/_exit(1). Complete cleanup/platform parity remain open. [CTL-002](research/ENGINE_CONTROL.md).
203
 
204
  **F-038 — Observed: authentication announcement helper.** The begin callback's field-3200 gate sets a one-shot value before invoking helper 165352, whose formats and trap-17 calls announce authentication/account status through print/chat commands. Recipient/mode/state-update semantics and permission rules are not yet specified. [CTL-003](research/ENGINE_CONTROL.md).
205
+
206
+ ## Frame timing and event-source update — 2026-10-11
207
+
208
+ **F-039 — Observed static branches, synthetic instruction traces:** both Linux ELF64 helpers apply nonzero fixed time before nonzero-timescale float multiplication, use a separate camera/zero-timescale branch, and cap intervals at 200 or 5000 ms according to dedicated/server cvars. Dedicated running-server candidates strictly greater than 500 ms trigger a diagnostic. Zero timescale alone does not force zero elapsed; finite bounded fixtures distinguish the branches. [TIME-001](research/FRAME_TIMING.md).
209
+
210
+ **F-040 — Observed event-source and dispatch bytes:** Sys_GetEvent emits a zero-type record with a host timestamp when no event is queued; journal modes can replace or record those events. Both seven-entry native switch tables map key/character/mouse/joystick/console/packet paths. Com_Frame derives raw elapsed from the event-loop return before applying its inlined modifier. Complete scheduling and runtime wrap behavior remain open. [TIME-002](research/FRAME_TIMING.md).
211
+
212
+ **F-041 — Observed native pause predicates:** cl_paused gates a native record count using state > 1 and unsigned field 0x24ee4 >= 1; count <= 1 selects sv_paused=1, larger counts select zero. SV_Frame inlines the gate before simulation steps. The counted field’s semantic name, game timeout interactions and runtime/platform parity remain unknown. [TIME-003](research/FRAME_TIMING.md).
docs/PROVISIONAL_TODO.md CHANGED
@@ -33,6 +33,8 @@ For implementation items, the minimum done condition is a documented contract, a
33
 
34
  ### First research batch
35
 
 
 
36
  Start with the open questions raised by the current binary evidence. These tasks depend on the linked reports, not on a working rewrite.
37
 
38
  - [ ] PROV-001: Trace Com_EventLoop, Com_Frame and SV_Frame together, including elapsed-time adjustment, timescale, pause, catch-up and restart ordering; extend RE-231.
 
33
 
34
  ### First research batch
35
 
36
+ Progress: [RE-232–234 / TIME-001–003](research/FRAME_TIMING.md) now cover Linux ELF64 elapsed modification, event sources and pause predicates. PROV-001 remains open for complete scheduling and platform coverage.
37
+
38
  Start with the open questions raised by the current binary evidence. These tasks depend on the linked reports, not on a working rewrite.
39
 
40
  - [ ] PROV-001: Trace Com_EventLoop, Com_Frame and SV_Frame together, including elapsed-time adjustment, timescale, pause, catch-up and restart ordering; extend RE-231.
docs/RESEARCH_LEDGER.md CHANGED
@@ -93,3 +93,7 @@ Completed static milestones RE-222–224: copy-count/final-NUL behavior, invalid
93
  ## 2026-10-11 — Engine timing, errors and announcements
94
 
95
  Completed static milestones RE-227–229. `analyze_engine_control.py` retains ten native instruction graphs, timing/error windows, the full QVM announcement helper and synthetic gate traces. CTL-001–003/F-036–038 establish simulation millisecond units, native time guard, ordinary mode-1 frame abort and escalation/process exit paths, and one-shot authentication status announcements. Six tests verify syntactic CFG edge handling, native service targets, native tick/VM-command connection and synthetic gate behavior. Computed native jumps and call continuations remain explicitly provisional; no full game/VM execution or runtime conformance is claimed.
 
 
 
 
 
93
  ## 2026-10-11 — Engine timing, errors and announcements
94
 
95
  Completed static milestones RE-227–229. `analyze_engine_control.py` retains ten native instruction graphs, timing/error windows, the full QVM announcement helper and synthetic gate traces. CTL-001–003/F-036–038 establish simulation millisecond units, native time guard, ordinary mode-1 frame abort and escalation/process exit paths, and one-shot authentication status announcements. Six tests verify syntactic CFG edge handling, native service targets, native tick/VM-command connection and synthetic gate behavior. Computed native jumps and call continuations remain explicitly provisional; no full game/VM execution or runtime conformance is claimed.
96
+
97
+ ## 2026-10-11 — Frame elapsed, event sources and pause gates
98
+
99
+ Completed bounded static milestones RE-232–234 with `analyze_frame_timing.py`: ten Linux ELF64 graphs, file-backed float constants and event tables, pause-assignment strings, inline frame windows and 64 decoded-native-instruction traces with synthetic cvar inputs. TIME-001–003 and F-039–041 record the clock distinctions, zero-timescale/camera behavior, elapsed caps, journal/event timestamp source and raw pause-count predicates. Seven tests check boundary outputs in both binaries, native table/constant identity, real trace addresses and rejection of unsupported domains. RE-231 stays open; new RE-235/236 retain layout and runtime questions. Original game code was not launched; the evaluator does not enter native callees.
docs/REVERSE_ENGINEERING_TODO.md CHANGED
@@ -271,3 +271,9 @@ Purpose: turn the supplied installation into an evidence-backed specification of
271
  - [x] RE-229 Identify begin callback field-3200 one-shot gate and helper 165352 as an authentication status announcement formatter/service caller. **Evidence:** CTL-003, helper body/string operands and five synthetic gate cases; recipient/state-update semantics remain open.
272
  - [ ] RE-230 Recover authentication announcement mode/recipient/role formatting branches and every field-3200 reset or alias write; compare UI output with deterministic runtime fixtures.
273
  - [ ] RE-231 Trace native event-loop elapsed-time production, timescale/pause/catch-up/map-restart paths and connect/begin/frame callback ordering on every platform.
 
 
 
 
 
 
 
271
  - [x] RE-229 Identify begin callback field-3200 one-shot gate and helper 165352 as an authentication status announcement formatter/service caller. **Evidence:** CTL-003, helper body/string operands and five synthetic gate cases; recipient/state-update semantics remain open.
272
  - [ ] RE-230 Recover authentication announcement mode/recipient/role formatting branches and every field-3200 reset or alias write; compare UI output with deterministic runtime fixtures.
273
  - [ ] RE-231 Trace native event-loop elapsed-time production, timescale/pause/catch-up/map-restart paths and connect/begin/frame callback ordering on every platform.
274
+
275
+ - [x] RE-232 Recover Linux ELF64 frame modification constants, fixedtime/timescale/camera branches, upper bounds and hitch threshold with decoded-instruction traces ([TIME-001](research/FRAME_TIMING.md)).
276
+ - [x] RE-233 Recover Linux ELF64 event timestamp sources, journal record paths and file-backed event dispatch tables; connect first-pump elapsed production to frame windows ([TIME-002](research/FRAME_TIMING.md)).
277
+ - [x] RE-234 Recover Linux ELF64 native pause gate predicates, record stride, direct cvar strings and inline server-frame placement ([TIME-003](research/FRAME_TIMING.md)).
278
+ - [ ] RE-235 Recover the semantic layout of native client field 0x24ee4; validate pause counting, stale pause-state handling and game timeout interactions in runtime fixtures.
279
+ - [ ] RE-236 Validate journal timestamps/payload ownership, event queue overflow and full frame scheduling with original-runtime traces; cover unsupported floating-point domains and cvar constraints.
docs/research/FRAME_TIMING.md ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Event timestamps, frame elapsed time and pause gates
2
+
3
+ Static pass, 2026-10-11. Reproduce with `python3 scripts/analyze_frame_timing.py`; validate with `python3 scripts/test_frame_timing.py`. [Generated evidence](frame-timing.json) records ten instruction graphs, file-backed constants, two seven-entry event tables, direct pause cvar assignments, frame/pause instruction windows and 64 bounded synthetic native instruction traces. Inputs are verified against the original and disassembly SHA-256 identities before analysis. No original process or complete VM is launched.
4
+
5
+ ## TIME-001: elapsed-time modification
6
+
7
+ Both Linux ELF64 images contain `Com_ModifyMsec` (dedicated `0x42d4de`, client `0x43df8a`) and equivalent inlined calculations in `Com_Frame`. The memory float compared with timescale is **0.0**, verified from the actual file-backed bytes `00000000`; a nearby symbol name does not identify that constant.
8
+
9
+ For finite, in-range values, the observed helper selects milliseconds in this order:
10
+
11
+ 1. A nonzero `com_fixedtime` integer replaces incoming elapsed time, bypassing multiplication.
12
+ 2. Otherwise a nonzero timescale multiplies elapsed time using single-precision conversion/multiplication and truncation toward zero.
13
+ 3. Otherwise a nonzero camera-mode integer keeps the zero candidate; with camera mode zero, incoming elapsed time is used.
14
+ 4. A nonpositive candidate is replaced with one if timescale is nonzero. With zero timescale that correction is skipped. The zero-timescale/camera path also bypasses the nonpositive test with its zero candidate.
15
+ 5. Apply the upper bound according to the table below.
16
+
17
+ | Dedicated cvar | Local server running | Upper bound | Diagnostic |
18
+ | --- | --- | --- | --- |
19
+ | Nonzero | Either | 5000 ms | With a running server, a candidate strictly greater than 500 ms is passed to `Com_Printf` before clamping. |
20
+ | Zero | Nonzero | 200 ms | No dedicated-hitch diagnostic in this path. |
21
+ | Zero | Zero | 5000 ms | No dedicated-hitch diagnostic in this path. |
22
+
23
+ The table depends on cvars, rather than simply the executable filename. Representative traces in **both images** produce: elapsed 201 → 200 for a local nondedicated server; elapsed 5001 → 5000 when no local server runs; dedicated/running 500 → 500 with no print, 501 → 501 with one print. Timescale 0 with elapsed 16 and camera mode 0 produces 16; setting camera mode 1 produces 0. Fixed time -5 with zero timescale produces -5, whereas with timescale 1 it becomes 1. Fixed time 17 with timescale 2 remains 17. These branches make “timescale zero always pauses” an incorrect replacement contract for this helper.
24
+
25
+ `Com_Frame` stores the raw measured interval in `com_frameMsec` before the inlined modification. It sends the resulting interval to `SV_Frame`; on its nondedicated branch, it pumps events and executes buffered commands again before `CL_Frame`. The earlier [simulation-tick calculation](ENGINE_CONTROL.md) separately multiplies the FPS-derived tick by timescale. Do not merge the raw host interval, modified frame interval and authoritative simulation clock into one Rust clock.
26
+
27
+ The evaluator executes only decoded instructions from `Com_ModifyMsec`, with symbolic cvar pointers and synthetic values. It rounds scalar operations to float32, follows the actual conditional targets and records diagnostic calls without entering callees. It rejects unknown opcodes/operands, nonfinite timescale and out-of-range float-to-int conversion. It does not model SSE exceptions, arbitrary memory, ABI execution, the full inlined frame body or reachable cvar constraints. Negative fixed time is a synthetic boundary example, not a claim about normal gameplay settings.
28
+
29
+ ## TIME-002: event timestamp source and dispatch
30
+
31
+ | Function | Dedicated address | Client address |
32
+ | --- | --- | --- |
33
+ | `Sys_GetEvent` | `0x4a7308` | `0x555d88` |
34
+ | `Com_GetRealEvent` | `0x42ee3e` | `0x43fba4` |
35
+ | `Com_EventLoop` | `0x42f1de` | `0x43ff42` |
36
+
37
+ `Sys_GetEvent` first consumes an already queued event if available. Otherwise it polls key events, console input, input-frame processing and network packets. If no event becomes available, it zeroes a 32-byte event record and places `Sys_Milliseconds()` at offset zero. `Com_GetRealEvent` normally delegates to that provider; journal mode 2 instead reads a 32-byte event record and optional payload, while mode 1 writes the record and payload. Short record/payload I/O routes to `Com_Error`. Journal paths therefore prevent an assumption that every frame timestamp comes directly from a fresh host-clock sample.
38
+
39
+ `Com_EventLoop` consumes its pushed-event ring before calling `Com_GetRealEvent`. For type zero it drains client and server loopback packet queues, then returns the event timestamp. Nonzero types dispatch through the file-backed tables at dedicated `0x4b3ea8` and client `0x56ba10`:
40
+
41
+ | Type | Observed service/path |
42
+ | --- | --- |
43
+ | 0 | Table entry points to payload cleanup, but the earlier type-zero branch bypasses this switch and handles loopback/return. |
44
+ | 1 | `CL_KeyEvent`, with event time forwarded. |
45
+ | 2 | `CL_CharEvent`. |
46
+ | 3 | `CL_MouseEvent`, with event time forwarded. |
47
+ | 4 | `CL_JoystickEvent`, with event time forwarded. |
48
+ | 5 | `Cbuf_AddText` calls for console input and a second string. |
49
+ | 6 | Packet handling with `com_dropsim`, payload length checks and server/client routing. |
50
+
51
+ The collector reads all 56 table bytes from their mapped file offsets and verifies every target is an instruction in the function. Linear entry windows can span other cases and are not standalone case executions. Out-of-range event types route to `Com_Error`; payload cleanup calls `Z_Free` on the ordinary handled-event paths. Detailed packet/drop simulation and queue overflow/ownership behavior remain follow-up work.
52
+
53
+ At the first frame event pump, `Com_Frame` compares the returned timestamp with retained `lastTime` using a signed comparison. A smaller timestamp resets `lastTime` to that value before subtraction. It pumps again until the elapsed value meets the selected minimum interval. The ordinary minimum is one millisecond; the nondedicated, positive-maxfps, non-timedemo path uses integer `1000 / com_maxfps`. Raw elapsed is recorded before the modification described above. This is bounded static ordering evidence, not a complete wraparound or scheduling specification.
54
+
55
+ ## TIME-003: pause state and inline gate
56
+
57
+ `SV_CheckPaused` appears at dedicated `0x41035a` and client `0x4782da`. With `cl_paused` zero, it returns zero directly. Otherwise it scans native client records with stride **0x2df40**, admitting a record into its count when the state word is unsigned greater than one and the word at **0x24ee4** is unsigned at least one. The second predicate follows from `cmp 1` and `sbb -1`; its semantic field name remains unresolved here. Do not call this a human-client or bot filter without recovering that layout.
58
+
59
+ With count at most one, the helper sets `sv_paused` to string `"1"` if needed and returns one. With count greater than one, it sets `sv_paused` to `"0"` if needed and returns zero. Both direct `Cvar_Set` argument pairs are decoded from the original bytes and included in the report. If `cl_paused` is zero, this helper does not clear a previously set `sv_paused` value.
60
+
61
+ `SV_Frame` contains an equivalent inlined gate before its tick calculation and game callback loop: the pause path returns without running those simulation steps. This is server-engine pause evidence, not the game-mode timeout implementation. Full paused networking, accumulated-time behavior, client presentation, game timeout rules and all-platform parity remain unverified.
62
+
63
+ ## Completion and follow-up
64
+
65
+ RE-232–234 cover these bounded static deliverables. RE-231 and PROV-001 remain open because complete catch-up, restart, callback ordering, runtime constraints and Windows/macOS/ELF32 behavior have not been recovered. Next recover the counted native field, validate actual configurations and journal fixtures, and compare the complete host-frame/server-frame paths across platforms. The Rust scheduling contract must preserve observed clock distinctions and explicitly address unsupported numeric domains after further evidence.
docs/research/README.md CHANGED
@@ -28,10 +28,11 @@ python3 scripts/test_native_metadata.py
28
  python3 scripts/test_native_followup.py
29
  python3 scripts/test_deferred_auth.py
30
  python3 scripts/test_engine_control.py
 
31
  python3 scripts/storage_policy.py check --require-setup
32
  ```
33
 
34
- The pipeline runs `fetch_analysis_references.py`, `analyze_qvm.py`, `analyze_native.py`, `analyze_vm_abi.py`, `analyze_native_dispatch.py`, `analyze_vm_extensions.py`, `analyze_native_metadata.py`, `analyze_native_coverage.py`, `analyze_auth_drop.py`, `analyze_deferred_auth.py`, and `analyze_engine_control.py` in that order. Python 3.9+ and **LLVM objdump** with x86 ELF/COFF/Mach-O support are required. This pass used Apple LLVM 17.0.0 on macOS arm64; it disassembled x86 images without running them. Upstream references are downloaded only if absent, checked against the committed SHA-256 manifest, and cached under `.cache/research/upstream/`. A clone also needs `git lfs pull` to obtain the derived artifacts/extracted members.
35
 
36
  Generated JSON/JSONL and compressed evidence are regenerated in place; keep prose notes in Markdown. Rerunning the four analysis stages reproduced every generated report and gzip artifact byte-for-byte on the same toolchain. gzip timestamps and stored filenames are fixed. Different objdump versions can produce different disassembly text, so tool version and uncompressed evidence hashes are recorded. Reports are UTF-8 normal Git text, with individual files below the storage policy's 2 MiB limit.
37
 
@@ -55,3 +56,5 @@ RE-041 is complete for the supplied three modules: decoder, full address-keyed i
55
  Next work should complete the extra game/UI entry command specifications and packet grammar, finish the 519 unresolved call sites and computed branches, recover syscall argument structures across platforms, decode SDL STABS and Windows resources, then follow QVM gameplay functions into movement/weapon/entity specifications and deterministic runtime fixtures.
56
 
57
  The extension collector adds five focused tests for native tail calls, command-register clobbers and intervening control transfers, fail-closed source identities/function boundaries, and shipped command/service connections. RE-211–213 record completed static milestones; RE-205/206 remain open for full behavior and platform coverage.
 
 
 
28
  python3 scripts/test_native_followup.py
29
  python3 scripts/test_deferred_auth.py
30
  python3 scripts/test_engine_control.py
31
+ python3 scripts/test_frame_timing.py
32
  python3 scripts/storage_policy.py check --require-setup
33
  ```
34
 
35
+ The pipeline runs `fetch_analysis_references.py`, `analyze_qvm.py`, `analyze_native.py`, `analyze_vm_abi.py`, `analyze_native_dispatch.py`, `analyze_vm_extensions.py`, `analyze_native_metadata.py`, `analyze_native_coverage.py`, `analyze_auth_drop.py`, `analyze_deferred_auth.py`, `analyze_engine_control.py`, and `analyze_frame_timing.py` in that order. Python 3.9+ and **LLVM objdump** with x86 ELF/COFF/Mach-O support are required. This pass used Apple LLVM 17.0.0 on macOS arm64; it disassembled x86 images without running them. Upstream references are downloaded only if absent, checked against the committed SHA-256 manifest, and cached under `.cache/research/upstream/`. A clone also needs `git lfs pull` to obtain the derived artifacts/extracted members.
36
 
37
  Generated JSON/JSONL and compressed evidence are regenerated in place; keep prose notes in Markdown. Rerunning the four analysis stages reproduced every generated report and gzip artifact byte-for-byte on the same toolchain. gzip timestamps and stored filenames are fixed. Different objdump versions can produce different disassembly text, so tool version and uncompressed evidence hashes are recorded. Reports are UTF-8 normal Git text, with individual files below the storage policy's 2 MiB limit.
38
 
 
56
  Next work should complete the extra game/UI entry command specifications and packet grammar, finish the 519 unresolved call sites and computed branches, recover syscall argument structures across platforms, decode SDL STABS and Windows resources, then follow QVM gameplay functions into movement/weapon/entity specifications and deterministic runtime fixtures.
57
 
58
  The extension collector adds five focused tests for native tail calls, command-register clobbers and intervening control transfers, fail-closed source identities/function boundaries, and shipped command/service connections. RE-211–213 record completed static milestones; RE-205/206 remain open for full behavior and platform coverage.
59
+
60
+ [Frame elapsed, event sources and pause gates](FRAME_TIMING.md) adds RE-232–234, ten Linux ELF64 graphs and 64 bounded native instruction traces. Reproduce with `python3 scripts/analyze_frame_timing.py`; check with `python3 scripts/test_frame_timing.py`. Original-runtime and all-platform scheduling remain open.
docs/research/frame-timing.json ADDED
The diff for this file is too large to render. See raw diff
 
scripts/analyze_binaries.py CHANGED
@@ -11,7 +11,7 @@ def main():
11
  for script in ('fetch_analysis_references.py', 'analyze_qvm.py', 'analyze_native.py',
12
  'analyze_vm_abi.py', 'analyze_native_dispatch.py', 'analyze_vm_extensions.py',
13
  'analyze_native_metadata.py', 'analyze_native_coverage.py', 'analyze_auth_drop.py',
14
- 'analyze_deferred_auth.py', 'analyze_engine_control.py'):
15
  subprocess.run([sys.executable, str(Path('scripts')/script)], check=True)
16
 
17
 
 
11
  for script in ('fetch_analysis_references.py', 'analyze_qvm.py', 'analyze_native.py',
12
  'analyze_vm_abi.py', 'analyze_native_dispatch.py', 'analyze_vm_extensions.py',
13
  'analyze_native_metadata.py', 'analyze_native_coverage.py', 'analyze_auth_drop.py',
14
+ 'analyze_deferred_auth.py', 'analyze_engine_control.py', 'analyze_frame_timing.py'):
15
  subprocess.run([sys.executable, str(Path('scripts')/script)], check=True)
16
 
17
 
scripts/analyze_frame_timing.py ADDED
@@ -0,0 +1,245 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Static Linux frame evidence and a bounded Com_ModifyMsec instruction evaluator."""
3
+ import gzip
4
+ import json
5
+ import math
6
+ from pathlib import Path
7
+ import re
8
+ import struct
9
+
10
+ from analyze_engine_control import control_flow, windows
11
+ from analyze_native import elf
12
+ from analyze_qvm import write_json
13
+ from analyze_vm_extensions import native_functions, verified
14
+
15
+
16
+ def read_address(data, sections, address, size):
17
+ if size < 0:
18
+ raise ValueError('negative read size')
19
+ matches = [s for s in sections if not s['zero_fill'] and
20
+ s['address'] <= address and address + size <= s['address'] + s['bytes']]
21
+ if len(matches) != 1:
22
+ raise ValueError('address is not uniquely file-backed')
23
+ offset = matches[0]['file_offset'] + address - matches[0]['address']
24
+ if offset < 0 or offset + size > len(data):
25
+ raise ValueError('section read outside file')
26
+ return data[offset:offset+size], offset
27
+
28
+
29
+ def float_constants(data, sections, graph):
30
+ result = []
31
+ for ins in graph['instructions']:
32
+ if ins['opcode'] != 'comiss' or '(%rip)' not in ins['operands']:
33
+ continue
34
+ match = re.search(r'#\s*0x([0-9a-f]+)', ins['operands'])
35
+ if not match:
36
+ raise ValueError('missing RIP-relative constant address')
37
+ address = int(match[1], 16)
38
+ raw, offset = read_address(data, sections, address, 4)
39
+ result.append({'instruction': ins['address'], 'address': address,
40
+ 'file_offset': offset, 'hex': raw.hex(), 'float32': struct.unpack('<f', raw)[0]})
41
+ return result
42
+
43
+
44
+ def event_table(data, sections, graph):
45
+ jumps = [i for i in graph['instructions'] if i['opcode'] == 'jmpq' and
46
+ re.fullmatch(r'\*0x[0-9a-f]+\(,%rax,8\)', i['operands'])]
47
+ if len(jumps) != 1:
48
+ raise ValueError('unexpected event switch shape')
49
+ ins = jumps[0]
50
+ base = int(re.search(r'0x([0-9a-f]+)', ins['operands'])[1], 16)
51
+ # This collector is deliberately specific to the observed 0..6 switch.
52
+ if not any(i['opcode'] == 'cmpl' and i['operands'].startswith('$0x6, %edx')
53
+ for i in graph['instructions']):
54
+ raise ValueError('missing event switch upper-bound evidence')
55
+ raw, offset = read_address(data, sections, base, 7*8)
56
+ targets = struct.unpack('<7Q', raw)
57
+ by_address = {i['address']: k for k, i in enumerate(graph['instructions'])}
58
+ if any(t not in by_address for t in targets):
59
+ raise ValueError('event table target outside function')
60
+ return {'instruction': ins['address'], 'address': base, 'file_offset': offset,
61
+ 'raw_hex': raw.hex(), 'entries': [{'event_type': n, 'target': t,
62
+ 'entry_window': graph['instructions'][by_address[t]:by_address[t]+16]}
63
+ for n, t in enumerate(targets)],
64
+ 'limitations': 'Type zero is handled before the switch. Entry windows are linear evidence, not case reachability.'}
65
+
66
+
67
+ def pause_assignments(data, sections, graph):
68
+ result = []
69
+ for k, ins in enumerate(graph['instructions']):
70
+ if ins['opcode'] != 'callq' or '<Cvar_Set>' not in ins['operands']:
71
+ continue
72
+ arguments = {}
73
+ for previous in graph['instructions'][max(0, k-3):k]:
74
+ match = re.match(r'\$0x([0-9a-f]+), %(edi|esi)(?:\s|$)', previous['operands'])
75
+ if previous['opcode'] != 'movl' or not match:
76
+ continue
77
+ address = int(match[1], 16)
78
+ payload = bytearray()
79
+ for at in range(address, address+256):
80
+ raw, _ = read_address(data, sections, at, 1)
81
+ if raw == b'\0':
82
+ break
83
+ payload.extend(raw)
84
+ else:
85
+ raise ValueError('unterminated cvar argument')
86
+ arguments[match[2]] = {'address': address, 'text': payload.decode('ascii')}
87
+ if set(arguments) != {'edi', 'esi'}:
88
+ raise ValueError('missing direct pause cvar arguments')
89
+ result.append({'call': ins['address'], 'arguments': arguments})
90
+ return result
91
+
92
+
93
+ def f32(value):
94
+ try:
95
+ return struct.unpack('<f', struct.pack('<f', value))[0]
96
+ except OverflowError as exc:
97
+ raise ValueError('float operation outside evaluator domain') from exc
98
+
99
+
100
+ def evaluate_modify(graph, constants, values):
101
+ """Interpret only the observed scalar subset; no callees or game code execute.
102
+
103
+ Input domain: signed-int32 inputs and finite, representable float operations.
104
+ Pointer registers are symbolic cvar handles. Calls only record a diagnostic.
105
+ Unsupported operands, opcodes and arithmetic domains fail closed.
106
+ """
107
+ required = {'elapsed', 'com_fixedtime', 'com_timescale', 'com_cameraMode',
108
+ 'com_dedicated', 'com_sv_running'}
109
+ if set(values) != required:
110
+ raise ValueError('unexpected timing input fields')
111
+ for key in required - {'com_timescale'}:
112
+ if not isinstance(values[key], int) or not -(2**31) <= values[key] < 2**31:
113
+ raise ValueError('input outside signed-int32 domain')
114
+ scale = f32(values['com_timescale'])
115
+ if not math.isfinite(scale):
116
+ raise ValueError('nonfinite timescale unsupported')
117
+ values = dict(values, com_timescale=scale)
118
+ instructions = graph['instructions']
119
+ indexes = {i['address']: n for n, i in enumerate(instructions)}
120
+ const = {c['instruction']: c['float32'] for c in constants}
121
+ regs = {'%edi': values['elapsed']}
122
+ trace, calls = [], []
123
+ equal = less = greater = False
124
+ pc = 0
125
+
126
+ def operand(text):
127
+ if text.startswith('$'):
128
+ value = int(text[1:], 0)
129
+ return value - 2**32 if 2**31 <= value < 2**32 else value
130
+ if text in regs:
131
+ return regs[text]
132
+ match = re.fullmatch(r'0x(2c|30)\((%\w+)\)', text)
133
+ if match:
134
+ kind, key = regs[match[2]]
135
+ if kind != 'cvar' or ((match[1] == '2c') != (key == 'com_timescale')):
136
+ raise ValueError('unexpected cvar field access')
137
+ return values[key]
138
+ raise ValueError('unsupported operand: ' + text)
139
+
140
+ for _ in range(1000):
141
+ ins = instructions[pc]
142
+ trace.append(ins['address'])
143
+ op = ins['opcode']
144
+ args = ins['operands'].split('#')[0].strip()
145
+ fields = [x.strip() for x in args.split(',')]
146
+ next_pc = pc + 1
147
+ if op in ('pushq', 'popq', 'nop', 'nopw', 'nopl'):
148
+ pass
149
+ elif op == 'movq' and '(%rip)' in args:
150
+ match = re.search(r'<(com_\w+)>', ins['operands'])
151
+ if not match or match[1] not in required:
152
+ raise ValueError('unexpected cvar pointer')
153
+ regs[fields[1]] = ('cvar', match[1])
154
+ elif op in ('movl', 'movss'):
155
+ regs[fields[1]] = operand(fields[0])
156
+ elif op == 'xorps' and fields[0] == fields[1]:
157
+ regs[fields[1]] = 0.0
158
+ elif op == 'cvtsi2ss':
159
+ regs[fields[1]] = f32(operand(fields[0]))
160
+ elif op == 'mulss':
161
+ regs[fields[1]] = f32(operand(fields[0]) * operand(fields[1]))
162
+ elif op == 'cvttss2si':
163
+ number = operand(fields[0])
164
+ if not math.isfinite(number) or not -(2**31) <= math.trunc(number) < 2**31:
165
+ raise ValueError('float conversion outside evaluator domain')
166
+ regs[fields[1]] = math.trunc(number)
167
+ elif op in ('cmpl', 'comiss', 'testl'):
168
+ a = const[ins['address']] if ins['address'] in const else operand(fields[0])
169
+ b = operand(fields[1])
170
+ if op == 'testl':
171
+ if fields[0] != fields[1]:
172
+ raise ValueError('unsupported bit test')
173
+ a, b = 0, b
174
+ equal, less, greater = b == a, b < a, b > a
175
+ elif op in ('cmovlel', 'cmovnel'):
176
+ condition = (less or equal) if op == 'cmovlel' else not equal
177
+ if condition:
178
+ regs[fields[1]] = operand(fields[0])
179
+ elif op in ('jne', 'je', 'jle', 'jg', 'jmp'):
180
+ take = {'jne': not equal, 'je': equal, 'jle': less or equal,
181
+ 'jg': greater, 'jmp': True}[op]
182
+ if take:
183
+ next_pc = indexes[int(fields[0].split()[0], 16)]
184
+ elif op == 'xorl' and fields[0] == fields[1]:
185
+ regs[fields[1]] = 0
186
+ equal, less, greater = True, False, False
187
+ elif op == 'callq' and '<Com_Printf>' in ins['operands']:
188
+ calls.append({'address': ins['address'], 'service': 'Com_Printf',
189
+ 'msec_argument': regs['%esi']})
190
+ # This observed path overwrites all later-used caller-saved values.
191
+ for register in ('%eax', '%ecx', '%edx', '%esi', '%edi', '%r8', '%r9', '%r10', '%r11'):
192
+ regs.pop(register, None)
193
+ elif op == 'retq':
194
+ return {'result': regs['%eax'], 'trace': trace, 'calls': calls}
195
+ else:
196
+ raise ValueError('unsupported instruction: ' + ins['line'])
197
+ pc = next_pc
198
+ raise ValueError('instruction budget exceeded')
199
+
200
+
201
+ def timing_cases():
202
+ defaults = {'elapsed': 16, 'com_fixedtime': 0, 'com_timescale': 1.0,
203
+ 'com_cameraMode': 0, 'com_dedicated': 0, 'com_sv_running': 1}
204
+ cases = [dict(defaults)]
205
+ for field, samples in [('elapsed', [0, -1, 200, 201, 500, 501, 5000, 5001]),
206
+ ('com_fixedtime', [-5, 1, 250, 6000]),
207
+ ('com_timescale', [0.0, -1.0, 0.01, 0.5, 1.5, 2.0]),
208
+ ('com_cameraMode', [1])]:
209
+ cases.extend(dict(defaults, **{field: value}) for value in samples)
210
+ cases.extend(dict(defaults, elapsed=elapsed, com_dedicated=ded, com_sv_running=running)
211
+ for elapsed in [500, 501, 5001] for ded, running in [(1, 1), (1, 0), (0, 0)])
212
+ cases.extend([dict(defaults, com_fixedtime=-5, com_timescale=0.0),
213
+ dict(defaults, com_cameraMode=1, com_timescale=0.0),
214
+ dict(defaults, com_fixedtime=17, com_timescale=2.0)])
215
+ return cases
216
+
217
+
218
+ def main():
219
+ output = {'native': {}, 'limitations': 'Static Linux ELF64 only. Original processes are not launched. '
220
+ 'Synthetic native instruction evaluator is bounded to Com_ModifyMsec finite arithmetic; '
221
+ 'it does not execute callees, model the event loop, or prove all-platform runtime parity.'}
222
+ names = ['Com_ModifyMsec', 'Com_GetRealEvent', 'Com_EventLoop', 'Sys_GetEvent', 'SV_CheckPaused']
223
+ for label in ('linux--Quake3-UrT-Ded.x86_64', 'linux--Quake3-UrT.x86_64'):
224
+ report = json.loads(Path(f'docs/research/native/{label}.json').read_text())
225
+ data = verified((Path('.cache/UrbanTerror43') / report['source']['path']).read_bytes(), report['source']['sha256'])
226
+ artifact = report['artifacts']['disassembly.txt']
227
+ functions = native_functions(verified(gzip.open(artifact['path'], 'rb').read(), artifact['uncompressed_sha256']).decode())
228
+ metadata, _ = elf(data)
229
+ graphs = {name: control_flow(functions[name]) for name in names}
230
+ constants = float_constants(data, metadata['sections'], graphs['Com_ModifyMsec'])
231
+ frame = control_flow(functions['Com_Frame'])
232
+ output['native'][label] = {'source': report['source'], 'disassembly_sha256': artifact['uncompressed_sha256'],
233
+ 'graphs': graphs, 'modify_float_constants': constants,
234
+ 'event_switch': event_table(data, metadata['sections'], graphs['Com_EventLoop']),
235
+ 'pause_assignments': pause_assignments(data, metadata['sections'], graphs['SV_CheckPaused']),
236
+ 'frame_float_constants': float_constants(data, metadata['sections'], frame),
237
+ 'frame_windows': windows(functions['Com_Frame'], [r'<lastTime\.', r'<com_fixedtime>', r'<com_timescale>', r'<com_cameraMode>', r'<com_maxfps>', r'<SV_Frame>', r'<CL_Frame>', r'<Com_EventLoop>', r'\$0x1f4', r'\$0x1388'], 3, 12),
238
+ 'server_pause_windows': windows(functions['SV_Frame'], [r'<cl_paused>', r'<sv_paused>', r'\$0x24ee4', r'\$0x2df40'], 4, 12),
239
+ 'synthetic_modify_cases': [{'input': case, **evaluate_modify(graphs['Com_ModifyMsec'], constants, case)} for case in timing_cases()]}
240
+ write_json(Path('docs/research/frame-timing.json'), output)
241
+ print(f'Recovered event/timing/pause graphs and {2*len(timing_cases())} bounded native instruction traces')
242
+
243
+
244
+ if __name__ == '__main__':
245
+ main()
scripts/test_frame_timing.py ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Check real native timing boundaries and reject unsupported synthetic domains."""
3
+ import copy
4
+ import json
5
+ from pathlib import Path
6
+ import unittest
7
+
8
+ from analyze_frame_timing import evaluate_modify, read_address, timing_cases
9
+
10
+
11
+ class FrameTimingTests(unittest.TestCase):
12
+ @classmethod
13
+ def setUpClass(cls):
14
+ cls.report = json.loads(Path('docs/research/frame-timing.json').read_text())
15
+
16
+ def run_case(self, **changes):
17
+ results = []
18
+ for native in self.report['native'].values():
19
+ values = dict(timing_cases()[0], **changes)
20
+ results.append(evaluate_modify(native['graphs']['Com_ModifyMsec'],
21
+ native['modify_float_constants'], values))
22
+ self.assertEqual([r['result'] for r in results], [results[0]['result']] * 2)
23
+ return results
24
+
25
+ def test_zero_constant_and_frame_inline_evidence(self):
26
+ for n in self.report['native'].values():
27
+ self.assertEqual([c['hex'] for c in n['modify_float_constants']], ['00000000'])
28
+ self.assertEqual([c['float32'] for c in n['frame_float_constants']], [0.0])
29
+ frame = '\n'.join(n['frame_windows'])
30
+ for marker in ['<com_fixedtime>', '<com_cameraMode>', '<com_timescale>', '<SV_Frame>', '<CL_Frame>']:
31
+ self.assertIn(marker, frame)
32
+
33
+ def test_clamps_and_dedicated_hitch_boundary(self):
34
+ self.assertEqual(self.run_case(elapsed=201)[0]['result'], 200)
35
+ self.assertEqual(self.run_case(elapsed=5001, com_sv_running=0)[0]['result'], 5000)
36
+ for elapsed, count, result in [(500, 0, 500), (501, 1, 501), (5001, 1, 5000)]:
37
+ for r in self.run_case(elapsed=elapsed, com_dedicated=1):
38
+ self.assertEqual(len(r['calls']), count)
39
+ self.assertEqual(r['result'], result)
40
+ if count:
41
+ self.assertEqual(r['calls'][0]['msec_argument'], elapsed)
42
+ self.assertEqual(len(self.run_case(elapsed=501, com_dedicated=1, com_sv_running=0)[0]['calls']), 0)
43
+
44
+ def test_zero_timescale_camera_and_fixedtime_paths(self):
45
+ self.assertEqual(self.run_case(com_timescale=0.0)[0]['result'], 16)
46
+ self.assertEqual(self.run_case(com_timescale=0.0, com_cameraMode=1)[0]['result'], 0)
47
+ self.assertEqual(self.run_case(com_fixedtime=-5, com_timescale=0.0)[0]['result'], -5)
48
+ self.assertEqual(self.run_case(com_fixedtime=-5)[0]['result'], 1)
49
+ self.assertEqual(self.run_case(com_fixedtime=17, com_timescale=2.0)[0]['result'], 17)
50
+ self.assertEqual(self.run_case(elapsed=17, com_timescale=0.5)[0]['result'], 8)
51
+ self.assertEqual(self.run_case(com_timescale=0.01)[0]['result'], 1)
52
+
53
+ def test_recorded_traces_are_actual_graph_addresses(self):
54
+ for n in self.report['native'].values():
55
+ g = n['graphs']['Com_ModifyMsec']
56
+ addresses = {i['address'] for i in g['instructions']}
57
+ self.assertEqual(len(n['synthetic_modify_cases']), 32)
58
+ for case in n['synthetic_modify_cases']:
59
+ result = evaluate_modify(g, n['modify_float_constants'], case['input'])
60
+ self.assertEqual(result, {k: case[k] for k in ('result', 'calls', 'trace')})
61
+ self.assertTrue(set(result['trace']) <= addresses)
62
+
63
+ def test_event_source_and_loop_services(self):
64
+ for n in self.report['native'].values():
65
+ def calls(name):
66
+ return {c['label'] for c in n['graphs'][name]['calls']}
67
+ self.assertTrue({'Sys_Milliseconds', 'Sys_GetPacket', 'Sys_ConsoleInput', 'IN_Frame'} <= calls('Sys_GetEvent'))
68
+ self.assertTrue({'Sys_GetEvent', 'FS_Read', 'FS_Write'} <= calls('Com_GetRealEvent'))
69
+ self.assertTrue({'Com_GetRealEvent', 'NET_GetLoopPacket', 'SV_PacketEvent', 'CL_PacketEvent', 'Z_Free'} <= calls('Com_EventLoop'))
70
+ self.assertIn('Cvar_Set', calls('SV_CheckPaused'))
71
+ table = n['event_switch']
72
+ self.assertEqual(len(bytes.fromhex(table['raw_hex'])), 56)
73
+ self.assertEqual([e['event_type'] for e in table['entries']], list(range(7)))
74
+ for e in table['entries']:
75
+ self.assertEqual(e['entry_window'][0]['address'], e['target'])
76
+ for e, service in zip(table['entries'][1:6], ['CL_KeyEvent', 'CL_CharEvent', 'CL_MouseEvent', 'CL_JoystickEvent', 'Cbuf_AddText']):
77
+ first_call = next(i for i in e['entry_window'] if i['opcode'] == 'callq')
78
+ self.assertIn(f'<{service}>', first_call['operands'])
79
+ self.assertEqual([(a['arguments']['edi']['text'], a['arguments']['esi']['text'])
80
+ for a in n['pause_assignments']], [('sv_paused', '0'), ('sv_paused', '1')])
81
+
82
+ def test_unsupported_domains_and_instructions_fail_closed(self):
83
+ for changes in [dict(com_timescale=float('nan')), dict(com_timescale=float('inf')),
84
+ dict(elapsed=2**31), dict(elapsed=2**30, com_timescale=4.0), dict(unknown=1)]:
85
+ with self.assertRaises(ValueError):
86
+ self.run_case(**changes)
87
+ n = next(iter(self.report['native'].values()))
88
+ g = copy.deepcopy(n['graphs']['Com_ModifyMsec'])
89
+ g['instructions'][0]['opcode'] = 'unsupported'
90
+ with self.assertRaises(ValueError):
91
+ evaluate_modify(g, n['modify_float_constants'], timing_cases()[0])
92
+
93
+ def test_file_backed_address_bounds(self):
94
+ sections = [{'address': 100, 'bytes': 4, 'file_offset': 1, 'zero_fill': False}]
95
+ self.assertEqual(read_address(b'abcde', sections, 101, 3), (b'cde', 2))
96
+ for address in [99, 102]:
97
+ with self.assertRaises(ValueError):
98
+ read_address(b'abcde', sections, address, 4)
99
+ with self.assertRaises(ValueError):
100
+ read_address(b'abcde', sections * 2, 100, 4)
101
+
102
+
103
+ if __name__ == '__main__':
104
+ unittest.main()