nmarshall commited on
Commit
61c0834
·
1 Parent(s): 7f999fd

assets: exhaustive md3 inventory

Browse files
docs/ASSET_PIPELINE_TODO.md CHANGED
@@ -745,3 +745,6 @@ Research progress: RE-690 [report](research/QVM_GEOMETRIC_REWOUND_TRACE.md); rem
745
 
746
 
747
  Research progress: RE-692 [report](research/MD3_CONTRACT.md); remaining native/content/runtime/platform scope stays open.
 
 
 
 
745
 
746
 
747
  Research progress: RE-692 [report](research/MD3_CONTRACT.md); remaining native/content/runtime/platform scope stays open.
748
+
749
+
750
+ Research progress: RE-693 [report](research/MD3_INVENTORY.md); remaining native/content/runtime/platform scope stays open.
docs/FINDINGS.md CHANGED
@@ -1091,3 +1091,6 @@ Bounded qagame paths recover bomb-plant/inactivity correction requests, direct f
1091
 
1092
 
1093
  **RE-692 — Bounds-checked original MD3 reader and QVM layout contract:** Original MD3 version15 headers/surfaces match recovered model offsets; all eight selected LOD3 files parse, including case-mismatched orion/lower_3.MD3. [Evidence](research/MD3_CONTRACT.md).
 
 
 
 
1091
 
1092
 
1093
  **RE-692 — Bounds-checked original MD3 reader and QVM layout contract:** Original MD3 version15 headers/surfaces match recovered model offsets; all eight selected LOD3 files parse, including case-mismatched orion/lower_3.MD3. [Evidence](research/MD3_CONTRACT.md).
1094
+
1095
+
1096
+ **RE-693 — Exhaustive extracted MD3 variant inventory:** All619 case-insensitively discovered MD3 occurrences validate, totaling152772340 bytes; case-sensitive suffix searches would miss originals. [Evidence](research/MD3_INVENTORY.md).
docs/PROVISIONAL_TODO.md CHANGED
@@ -1077,3 +1077,6 @@ RE-681–690 batch complete: [source model geometry through rewound trace](resea
1077
 
1078
 
1079
  Research progress: RE-692 [report](research/MD3_CONTRACT.md); remaining native/content/runtime/platform scope stays open.
 
 
 
 
1077
 
1078
 
1079
  Research progress: RE-692 [report](research/MD3_CONTRACT.md); remaining native/content/runtime/platform scope stays open.
1080
+
1081
+
1082
+ Research progress: RE-693 [report](research/MD3_INVENTORY.md); remaining native/content/runtime/platform scope stays open.
docs/RESEARCH_LEDGER.md CHANGED
@@ -1107,3 +1107,8 @@ Actual star/nested model hierarchies generate categories1/3/5/6, then source sor
1107
  ## 2026-10-21 — RE-692 Bounds-checked original MD3 reader and QVM layout contract (separate task commit)
1108
 
1109
  Original MD3 version15 headers/surfaces match recovered model offsets; all eight selected LOD3 files parse, including case-mismatched orion/lower_3.MD3. Five parser/accessor tests independently check packed coordinates, tags, indices, UVs/shaders, malformed counts/ranges/signatures and original selected files. Full suite: 1223 tests pass. Report regenerates byte-identically (22517 bytes). Storage/filter, links and whitespace checks pass. No assets/Rust runtime/shared VM evaluator changed; native/reference/legal/full platform scope remains open.
 
 
 
 
 
 
1107
  ## 2026-10-21 — RE-692 Bounds-checked original MD3 reader and QVM layout contract (separate task commit)
1108
 
1109
  Original MD3 version15 headers/surfaces match recovered model offsets; all eight selected LOD3 files parse, including case-mismatched orion/lower_3.MD3. Five parser/accessor tests independently check packed coordinates, tags, indices, UVs/shaders, malformed counts/ranges/signatures and original selected files. Full suite: 1223 tests pass. Report regenerates byte-identically (22517 bytes). Storage/filter, links and whitespace checks pass. No assets/Rust runtime/shared VM evaluator changed; native/reference/legal/full platform scope remains open.
1110
+
1111
+
1112
+ ## 2026-10-21 — RE-693 Exhaustive extracted MD3 variant inventory (separate task commit)
1113
+
1114
+ All619 case-insensitively discovered MD3 occurrences validate, totaling152772340 bytes; case-sensitive suffix searches would miss originals. Four inventory checks verify exhaustive paths, all validation outcomes, byte totals, sampled source hashes and variant grouping without selecting precedence. Full suite: 1227 tests pass. Report regenerates byte-identically (945502 bytes). Storage/filter, links and whitespace checks pass. No assets/Rust runtime/shared VM evaluator changed; native/reference/legal/full platform scope remains open.
docs/REVERSE_ENGINEERING_TODO.md CHANGED
@@ -1120,3 +1120,6 @@ Purpose: turn the supplied installation into an evidence-backed specification of
1120
 
1121
 
1122
  - [x] RE-692 Bounds-checked original MD3 reader and QVM layout contract ([report](research/MD3_CONTRACT.md)).
 
 
 
 
1120
 
1121
 
1122
  - [x] RE-692 Bounds-checked original MD3 reader and QVM layout contract ([report](research/MD3_CONTRACT.md)).
1123
+
1124
+
1125
+ - [x] RE-693 Exhaustive extracted MD3 variant inventory ([report](research/MD3_INVENTORY.md)).
docs/research/MD3_INVENTORY.md ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ # Exhaustive extracted MD3 variant inventory
2
+
3
+ RE-693. Reproduce with `python3 scripts/analyze_md3_inventory.py`; check with `python3 scripts/test_md3_inventory.py`. The [machine report](md3-inventory.json) retains source identity and bounded fixtures.
4
+
5
+ Every extracted file with case-insensitive.md3 suffix is independently retained by archive/member path and SHA256. All619 original models pass header/surface/index checks plus finite frames/tags/UV reads. The report records frame/tag/surface/shader counts, UV ranges, tag-name changes and case-folded path groups without changing names or resolving precedence. The selected set totals152772340 bytes and619 distinct case-folded virtual paths. A lowercase-only scan found only467 files, demonstrating a practical cross-platform discovery gap. No asset is rewritten, and structural validation does not establish rendering/native runtime parity.
6
+
7
+ No original executable/full QVM/native collision/physics/rendering executes. Native responses and legal input domains are supplied or unverified where stated; full runtime/macOS/Linux Rust/Bevy/Solari gameplay remains open.
docs/research/README.md CHANGED
@@ -678,3 +678,6 @@ The extension collector adds five focused tests for native tail calls, command-r
678
 
679
 
680
  [RE-692: Bounds-checked original MD3 reader and QVM layout contract](MD3_CONTRACT.md). Reproduce with `python3 scripts/analyze_md3_contract.py`; check with `python3 scripts/test_md3_contract.py`.
 
 
 
 
678
 
679
 
680
  [RE-692: Bounds-checked original MD3 reader and QVM layout contract](MD3_CONTRACT.md). Reproduce with `python3 scripts/analyze_md3_contract.py`; check with `python3 scripts/test_md3_contract.py`.
681
+
682
+
683
+ [RE-693: Exhaustive extracted MD3 variant inventory](MD3_INVENTORY.md). Reproduce with `python3 scripts/analyze_md3_inventory.py`; check with `python3 scripts/test_md3_inventory.py`.
docs/research/md3-inventory.json ADDED
The diff for this file is too large to render. See raw diff
 
scripts/analyze_md3_inventory.py ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Inventory every extracted MD3 occurrence, preserving archive/case variants."""
3
+ import hashlib
4
+ from collections import defaultdict
5
+ from pathlib import Path
6
+ from md3 import MD3
7
+ from analyze_qvm import write_json
8
+
9
+ def paths():return sorted(p for p in Path('assets/extracted').rglob('*') if p.is_file() and p.suffix.lower()=='.md3')
10
+ def virtual(p):
11
+ parts=p.parts;j=parts.index('members');return '/'.join(parts[j+2:])
12
+ def main():
13
+ rows=[];groups=defaultdict(list)
14
+ for p in paths():
15
+ raw=p.read_bytes();r={'path':str(p),'virtual_path':virtual(p),'sha256':hashlib.sha256(raw).hexdigest(),'bytes':len(raw)}
16
+ try:
17
+ m=MD3(raw);surfaces=[];changes=[];first=[m.tag(0,j)['name'] for j in range(m.tags)] if m.frames else []
18
+ for frame in range(m.frames):
19
+ m.frame(frame)
20
+ for j in range(m.tags):
21
+ tag=m.tag(frame,j)
22
+ if tag['name']!=first[j]:changes.append([frame,j,tag['name']])
23
+ for s in m.surfaces:
24
+ uv=[m.uv(s,j) for j in range(s.vertices)];surfaces.append({'name':s.name,'frames':s.frames,'vertices':s.vertices,'triangles':s.triangles,'uv_bounds':[[min(v[j] for v in uv),max(v[j] for v in uv)] for j in range(2)] if uv else None,'shaders':[m.shader(s,j) for j in range(s.shaders)]})
25
+ r.update({'frames':m.frames,'tag_names':first,'tag_name_changes':changes,'surfaces':surfaces,'status':'validated'})
26
+ except (ValueError,IndexError) as e:r.update({'status':'rejected','error':str(e)})
27
+ rows.append(r);groups[r['virtual_path'].casefold()].append({'path':str(p),'sha256':r['sha256'],'stored_virtual_path':r['virtual_path']})
28
+ duplicates={k:v for k,v in groups.items() if len(v)>1};write_json(Path('docs/research/md3-inventory.json'),{'models':rows,'totals':{'occurrences':len(rows),'validated':sum(r['status']=='validated' for r in rows),'bytes':sum(r['bytes'] for r in rows),'casefold_paths':len(groups)},'path_variants':duplicates,'limits':'Every extracted occurrence retained; casefold groups are comparisons, not VFS precedence or asset renames. Frames/tags/UVs validated finite, triangle bounds and packed regions validated. Normal interpretation/rendering/runtime resolution remain separate.'})
29
+ if __name__=='__main__':main()
scripts/test_md3_inventory.py ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import hashlib,json,unittest
2
+ from pathlib import Path
3
+ from analyze_md3_inventory import paths,virtual
4
+ class MD3InventoryTests(unittest.TestCase):
5
+ @classmethod
6
+ def setUpClass(cls):cls.r=json.loads(Path('docs/research/md3-inventory.json').read_text())
7
+ def test_exhaustive_case_insensitive_occurrences(self):
8
+ p=paths();self.assertEqual(len(p),619);self.assertEqual([str(x) for x in p],[r['path'] for r in self.r['models']]);self.assertTrue(any(x.suffix=='.MD3' for x in p))
9
+ def test_every_original_passes_structural_frame_tag_uv_validation(self):self.assertEqual(self.r['totals']['validated'],619);self.assertTrue(all(x['status']=='validated' for x in self.r['models']))
10
+ def test_original_hashes_and_byte_totals(self):
11
+ self.assertEqual(self.r['totals']['bytes'],sum(Path(r['path']).stat().st_size for r in self.r['models']))
12
+ for r in self.r['models'][::100]:self.assertEqual(hashlib.sha256(Path(r['path']).read_bytes()).hexdigest(),r['sha256']);self.assertEqual(r['virtual_path'],virtual(Path(r['path'])))
13
+ def test_no_variant_is_silently_selected(self):
14
+ names=[r['virtual_path'].casefold() for r in self.r['models']];self.assertEqual(self.r['totals']['casefold_paths'],len(set(names)));self.assertEqual(len(self.r['path_variants']),sum(names.count(n)>1 for n in set(names)))
15
+ if __name__=='__main__':unittest.main()