nmarshall commited on
Commit
fc56f58
·
1 Parent(s): 72b1843

document: reindex properties

Browse files
Cargo.lock CHANGED
@@ -31,6 +31,13 @@ dependencies = [
31
  "nanobrush-geometry",
32
  ]
33
 
 
 
 
 
 
 
 
34
  [[package]]
35
  name = "nanobrush-geometry"
36
  version = "0.1.0"
 
31
  "nanobrush-geometry",
32
  ]
33
 
34
+ [[package]]
35
+ name = "nanobrush-document"
36
+ version = "0.1.0"
37
+ dependencies = [
38
+ "mb3-dat",
39
+ ]
40
+
41
  [[package]]
42
  name = "nanobrush-geometry"
43
  version = "0.1.0"
Cargo.toml CHANGED
@@ -1,5 +1,5 @@
1
  [workspace]
2
- members = ["crates/mb3-dat", "crates/mb3-txt", "crates/nanobrush-cli", "crates/mb3-obj", "crates/nanobrush-geometry"]
3
  resolver = "3"
4
 
5
  [workspace.package]
 
1
  [workspace]
2
+ members = ["crates/mb3-dat", "crates/mb3-txt", "crates/nanobrush-cli", "crates/mb3-obj", "crates/nanobrush-geometry", "crates/nanobrush-document"]
3
  resolver = "3"
4
 
5
  [workspace.package]
README.md CHANGED
@@ -76,3 +76,5 @@ The asset-free Nix check `nix build .#checks.aarch64-darwin.rust --no-link` runs
76
  `nix run .#nanobrush -- inspect FILE` prints a JSON summary; `validate FILE` checks structure. Convert with `convert INPUT OUTPUT --to txt`, `--to dat-201501280`, or `--to dat-201512200`. Inside the dev shell, use `cargo run -p nanobrush-cli -- ...`. Existing outputs require `--force`; source aliases and output symlinks are refused.
77
 
78
  [TXT compatibility](docs/RUST_TXT.md) documents strict UTF-8 input, BOM output, finite numbers and computed-data consistency. [CLI usage and publication](docs/CLI.md) covers limits, exit codes and atomic saves. Full Cargo tests require hydrated converter fixtures; Nix checks and the native package build exclude assets.
 
 
 
76
  `nix run .#nanobrush -- inspect FILE` prints a JSON summary; `validate FILE` checks structure. Convert with `convert INPUT OUTPUT --to txt`, `--to dat-201501280`, or `--to dat-201512200`. Inside the dev shell, use `cargo run -p nanobrush-cli -- ...`. Existing outputs require `--force`; source aliases and output symlinks are refused.
77
 
78
  [TXT compatibility](docs/RUST_TXT.md) documents strict UTF-8 input, BOM output, finite numbers and computed-data consistency. [CLI usage and publication](docs/CLI.md) covers limits, exit codes and atomic saves. Full Cargo tests require hydrated converter fixtures; Nix checks and the native package build exclude assets.
79
+
80
+ OBJ export is available through `convert INPUT OUTPUT --to obj --allow-lossy`; add `--compute` to explicitly rebuild native brush/patch geometry first. `preflight INPUT --to FORMAT` prints grouped compatibility/loss JSON. Legacy TXT decoding is explicit with `--text-encoding latin1`. See the [ten-task ledger](docs/PROGRESS_2026-10-26.md), [geometry policy](docs/GEOMETRY.md) and [property-edit API](docs/PROPERTIES.md).
crates/nanobrush-document/Cargo.toml ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ [package]
2
+ name = "nanobrush-document"
3
+ version = "0.1.0"
4
+ edition.workspace = true
5
+ rust-version.workspace = true
6
+ publish = false
7
+ [dependencies]
8
+ mb3-dat = { path = "../mb3-dat" }
9
+ [lints]
10
+ workspace = true
crates/nanobrush-document/src/lib.rs ADDED
@@ -0,0 +1,314 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ //! Explicit transactional table reordering/removal over the existing world model.
2
+ use mb3_dat::{Limits, Property, Uv, World};
3
+ use std::fmt;
4
+ #[derive(Clone, Debug, PartialEq, Eq)]
5
+ pub enum Error {
6
+ Index,
7
+ DuplicateOrder,
8
+ InvalidSlots,
9
+ Limit,
10
+ Allocation,
11
+ Overflow,
12
+ }
13
+ impl fmt::Display for Error {
14
+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
15
+ write!(f, "property edit: {self:?}")
16
+ }
17
+ }
18
+ impl std::error::Error for Error {}
19
+ type Result<T> = std::result::Result<T, Error>;
20
+ #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
21
+ pub struct Report {
22
+ pub dropped_string_names: usize,
23
+ pub dropped_uv_names: usize,
24
+ pub dropped_string_values: usize,
25
+ pub dropped_uv_values: usize,
26
+ }
27
+ struct Budget {
28
+ items: usize,
29
+ bytes: usize,
30
+ limits: Limits,
31
+ }
32
+ impl Budget {
33
+ fn items(&mut self, n: usize) -> Result<()> {
34
+ self.items = self.items.checked_add(n).ok_or(Error::Overflow)?;
35
+ if self.items > self.limits.max_items {
36
+ Err(Error::Limit)
37
+ } else {
38
+ Ok(())
39
+ }
40
+ }
41
+ fn vec<T>(&mut self, n: usize) -> Result<Vec<T>> {
42
+ self.bytes(
43
+ n.checked_mul(std::mem::size_of::<T>())
44
+ .ok_or(Error::Overflow)?,
45
+ )?;
46
+ let mut v = Vec::new();
47
+ v.try_reserve_exact(n).map_err(|_| Error::Allocation)?;
48
+ Ok(v)
49
+ }
50
+ fn bytes(&mut self, n: usize) -> Result<()> {
51
+ self.bytes = self.bytes.checked_add(n).ok_or(Error::Overflow)?;
52
+ if self.bytes > self.limits.max_owned_bytes {
53
+ Err(Error::Limit)
54
+ } else {
55
+ Ok(())
56
+ }
57
+ }
58
+ fn string(&mut self, s: &str) -> Result<String> {
59
+ if s.len() > self.limits.max_string_bytes {
60
+ return Err(Error::Limit);
61
+ }
62
+ self.bytes(s.len())?;
63
+ let mut out = String::new();
64
+ out.try_reserve_exact(s.len())
65
+ .map_err(|_| Error::Allocation)?;
66
+ out.push_str(s);
67
+ Ok(out)
68
+ }
69
+ fn table(&mut self, names: &[String], order: &[u32]) -> Result<(Vec<String>, Vec<u32>)> {
70
+ self.items(names.len())?;
71
+ self.items(order.len())?;
72
+ if order.len() > i32::MAX as usize {
73
+ return Err(Error::Overflow);
74
+ }
75
+ let mut map = self.vec(names.len())?;
76
+ map.resize(names.len(), u32::MAX);
77
+ let mut out = self.vec(order.len())?;
78
+ for (new, old) in order.iter().enumerate() {
79
+ let slot = map.get_mut(*old as usize).ok_or(Error::Index)?;
80
+ if *slot != u32::MAX {
81
+ return Err(Error::DuplicateOrder);
82
+ }
83
+ *slot = new as u32;
84
+ out.push(self.string(&names[*old as usize])?);
85
+ }
86
+ Ok((out, map))
87
+ }
88
+ }
89
+ fn validate<T>(slots: &[Property<T>], map: &[u32]) -> Result<usize> {
90
+ let mut previous = None;
91
+ let mut kept = 0;
92
+ for slot in slots {
93
+ if previous.is_some_and(|n| n >= slot.index) {
94
+ return Err(Error::InvalidSlots);
95
+ }
96
+ previous = Some(slot.index);
97
+ let id = map.get(slot.index as usize).ok_or(Error::InvalidSlots)?;
98
+ if *id != u32::MAX {
99
+ kept += 1;
100
+ }
101
+ }
102
+ Ok(kept)
103
+ }
104
+ struct Update {
105
+ strings: Vec<Property<String>>,
106
+ uv: Vec<Property<Uv>>,
107
+ }
108
+ /// New tables select old indices in the desired order. Omitted indices explicitly
109
+ /// remove names and associated values. All validation/allocation precedes mutation.
110
+ pub fn remap(
111
+ world: &mut World,
112
+ string_order: &[u32],
113
+ uv_order: &[u32],
114
+ limits: Limits,
115
+ ) -> Result<Report> {
116
+ let mut b = Budget {
117
+ items: 0,
118
+ bytes: 0,
119
+ limits,
120
+ };
121
+ let (strings, sm) = b.table(&world.string_properties, string_order)?;
122
+ let (uv, um) = b.table(&world.uv_properties, uv_order)?;
123
+ let mut report = Report {
124
+ dropped_string_names: world.string_properties.len() - strings.len(),
125
+ dropped_uv_names: world.uv_properties.len() - uv.len(),
126
+ ..Report::default()
127
+ };
128
+ let mut count = 0usize;
129
+ for brush in &world.brushes {
130
+ b.items(1)?;
131
+ count = count
132
+ .checked_add(brush.planes.len())
133
+ .ok_or(Error::Overflow)?;
134
+ }
135
+ b.items(count)?;
136
+ let mut updates = b.vec(count)?;
137
+ for p in world.brushes.iter().flat_map(|b| &b.planes) {
138
+ b.items(p.strings.len())?;
139
+ b.items(p.uv.len())?;
140
+ let ns = validate(&p.strings, &sm)?;
141
+ let nu = validate(&p.uv, &um)?;
142
+ report.dropped_string_values = report
143
+ .dropped_string_values
144
+ .checked_add(p.strings.len() - ns)
145
+ .ok_or(Error::Overflow)?;
146
+ report.dropped_uv_values = report
147
+ .dropped_uv_values
148
+ .checked_add(p.uv.len() - nu)
149
+ .ok_or(Error::Overflow)?;
150
+ let mut ss = b.vec(ns)?;
151
+ for s in &p.strings {
152
+ let index = sm[s.index as usize];
153
+ if index != u32::MAX {
154
+ ss.push(Property {
155
+ index,
156
+ value: b.string(&s.value)?,
157
+ });
158
+ }
159
+ }
160
+ ss.sort_unstable_by_key(|p| p.index);
161
+ let mut us = b.vec(nu)?;
162
+ for s in &p.uv {
163
+ let index = um[s.index as usize];
164
+ if index != u32::MAX {
165
+ us.push(Property {
166
+ index,
167
+ value: s.value.clone(),
168
+ });
169
+ }
170
+ }
171
+ us.sort_unstable_by_key(|p| p.index);
172
+ updates.push(Update {
173
+ strings: ss,
174
+ uv: us,
175
+ });
176
+ }
177
+ world.string_properties = strings;
178
+ world.uv_properties = uv;
179
+ for (p, u) in world
180
+ .brushes
181
+ .iter_mut()
182
+ .flat_map(|b| &mut b.planes)
183
+ .zip(updates)
184
+ {
185
+ p.strings = u.strings;
186
+ p.uv = u.uv;
187
+ }
188
+ Ok(report)
189
+ }
190
+ #[cfg(test)]
191
+ mod tests {
192
+ use super::*;
193
+ use mb3_dat::{Brush, Document, Plane, Version};
194
+ fn world() -> World {
195
+ World {
196
+ string_properties: vec!["unused".into(), "a".into(), "b".into()],
197
+ uv_properties: vec!["uv".into()],
198
+ brushes: vec![Brush {
199
+ planes: vec![Plane {
200
+ p0: [1., 2., 3.],
201
+ strings: vec![
202
+ Property {
203
+ index: 1,
204
+ value: String::new(),
205
+ },
206
+ Property {
207
+ index: 2,
208
+ value: "value".into(),
209
+ },
210
+ ],
211
+ uv: vec![Property {
212
+ index: 0,
213
+ value: Uv {
214
+ u: [1., 0., 0.],
215
+ v: [0., 1., 0.],
216
+ u_scale: 1.,
217
+ u_shift: 0.,
218
+ v_scale: 1.,
219
+ v_shift: 0.,
220
+ },
221
+ }],
222
+ ..Default::default()
223
+ }],
224
+ }],
225
+ ..Default::default()
226
+ }
227
+ }
228
+ #[test]
229
+ fn reorder_remove_empty_values_and_geometry_preserved() {
230
+ let mut w = world();
231
+ let p = w.brushes[0].planes[0].p0;
232
+ let r = remap(&mut w, &[2, 1], &[], Limits::default()).unwrap();
233
+ assert_eq!(w.string_properties, ["b", "a"]);
234
+ assert_eq!(
235
+ w.brushes[0].planes[0].strings,
236
+ [
237
+ Property {
238
+ index: 0,
239
+ value: "value".into()
240
+ },
241
+ Property {
242
+ index: 1,
243
+ value: String::new()
244
+ }
245
+ ]
246
+ );
247
+ assert_eq!(w.brushes[0].planes[0].p0, p);
248
+ assert_eq!(
249
+ r,
250
+ Report {
251
+ dropped_string_names: 1,
252
+ dropped_uv_names: 1,
253
+ dropped_uv_values: 1,
254
+ dropped_string_values: 0
255
+ }
256
+ );
257
+ let d = Document {
258
+ world: w,
259
+ version: Version::V201512200,
260
+ };
261
+ assert_eq!(
262
+ mb3_dat::decode(
263
+ &mb3_dat::encode(&d, &Limits::default()).unwrap(),
264
+ &Limits::default()
265
+ )
266
+ .unwrap(),
267
+ d
268
+ );
269
+ }
270
+ #[test]
271
+ fn explicit_drop_counts_and_roundtrip_permutation() {
272
+ let mut w = world();
273
+ let original = w.clone();
274
+ remap(&mut w, &[2, 0, 1], &[0], Limits::default()).unwrap();
275
+ remap(&mut w, &[1, 2, 0], &[0], Limits::default()).unwrap();
276
+ assert_eq!(w, original);
277
+ let r = remap(&mut w, &[], &[], Limits::default()).unwrap();
278
+ assert_eq!(r.dropped_string_values, 2);
279
+ assert!(w.brushes[0].planes[0].strings.is_empty());
280
+ }
281
+ #[test]
282
+ fn errors_are_transactional_including_late_slots() {
283
+ for order in [vec![1, 1], vec![9]] {
284
+ let mut w = world();
285
+ let before = w.clone();
286
+ assert!(remap(&mut w, &order, &[0], Limits::default()).is_err());
287
+ assert_eq!(w, before);
288
+ }
289
+ let mut w = world();
290
+ w.brushes[0].planes[0].strings[1].index = 99;
291
+ let before = w.clone();
292
+ assert!(remap(&mut w, &[0, 1, 2], &[0], Limits::default()).is_err());
293
+ assert_eq!(w, before);
294
+ for l in [
295
+ Limits {
296
+ max_items: 1,
297
+ ..Limits::default()
298
+ },
299
+ Limits {
300
+ max_owned_bytes: 1,
301
+ ..Limits::default()
302
+ },
303
+ Limits {
304
+ max_string_bytes: 1,
305
+ ..Limits::default()
306
+ },
307
+ ] {
308
+ let mut w = world();
309
+ let before = w.clone();
310
+ assert!(remap(&mut w, &[0, 1, 2], &[0], l).is_err());
311
+ assert_eq!(w, before);
312
+ }
313
+ }
314
+ }
crates/nanobrush-geometry/src/brush.rs CHANGED
@@ -131,8 +131,8 @@ pub fn contours(brush: &mb3_dat::Brush, policy: Policy, limits: ComputeLimits) -
131
  });
132
  let start = (0..face.len())
133
  .min_by(|a, c| {
134
- for d in 0..3 {
135
- let order = vertices[face[*a]][d].total_cmp(&vertices[face[*c]][d]);
136
  if !order.is_eq() {
137
  return order;
138
  }
 
131
  });
132
  let start = (0..face.len())
133
  .min_by(|a, c| {
134
+ for (left, right) in vertices[face[*a]].iter().zip(vertices[face[*c]].iter()) {
135
+ let order = left.total_cmp(right);
136
  if !order.is_eq() {
137
  return order;
138
  }
docs/CLI.md CHANGED
@@ -1,6 +1,6 @@
1
  # Native inspection, validation and conversion — F31
2
 
3
- The `nanobrush` executable is built from `crates/nanobrush-cli`, backed by the strict [TXT](RUST_TXT.md) and [DAT](RUST_DAT.md) libraries. It supports TXT and DAT versions 201501280/201512200. MAP/VMF/OBJ, geometry evaluation, user-interface replay and the editor application remain future work.
4
 
5
  Run from the repository:
6
 
 
1
  # Native inspection, validation and conversion — F31
2
 
3
+ The `nanobrush` executable is built from `crates/nanobrush-cli`, backed by the strict [TXT](RUST_TXT.md) and [DAT](RUST_DAT.md) libraries. It supports TXT and DAT versions 201501280/201512200, computed OBJ export, explicit recoding and opt-in native geometry computation. MAP/VMF codecs, user-interface replay and the editor application remain future work.
4
 
5
  Run from the repository:
6
 
docs/FINDINGS.md CHANGED
@@ -161,3 +161,15 @@ The [Rust DAT codec](RUST_DAT.md) reads and writes both recovered layouts with c
161
  ## F30/F31 — TXT serialization and native CLI
162
 
163
  The [TXT codec](RUST_TXT.md) implements the latest grammar and old property-free subset with bounded UTF-8 parsing, canonical BOM output, finite round-trip decimals and consistent computed arrays. The [CLI](CLI.md) inspects/validates and converts TXT plus both DAT versions, refusing silent loss and publishing validated outputs with explicit no-clobber/force behavior. [E04](experiments/txt-cli-2026-10-21.md) covers 58 exact DAT→TXT→DAT round trips through the real process, seven TXT outputs, 16 direct TXT→DAT pairs and nine invalid/lossy source rejections.
 
 
 
 
 
 
 
 
 
 
 
 
 
161
  ## F30/F31 — TXT serialization and native CLI
162
 
163
  The [TXT codec](RUST_TXT.md) implements the latest grammar and old property-free subset with bounded UTF-8 parsing, canonical BOM output, finite round-trip decimals and consistent computed arrays. The [CLI](CLI.md) inspects/validates and converts TXT plus both DAT versions, refusing silent loss and publishing validated outputs with explicit no-clobber/force behavior. [E04](experiments/txt-cli-2026-10-21.md) covers 58 exact DAT→TXT→DAT round trips through the real process, seven TXT outputs, 16 direct TXT→DAT pairs and nine invalid/lossy source rejections.
164
+
165
+ ## F32 — Native export and explicit recoding
166
+
167
+ [OBJ export](OBJ.md) matches all four computed converter OBJ captures. [CLI preflight](CLI.md) reports grouped target incompatibilities/losses; explicit OBJ acknowledgement reuses atomic source-preserving publication. Latin-1 TXT input is opt-in, bounded and distinct from UTF-8 output or CP1252. See the [ten-task ledger](PROGRESS_2026-10-26.md).
168
+
169
+ ## F33 — Analytic native geometry
170
+
171
+ [Geometry contracts](GEOMETRY.md) define normalized equations, bounds, strict convex brush reconstruction, tensor-quadratic Patch9 sampling, computed worlds and stable nearest-hit rays. Analytic fixtures establish this native policy only. Interior side, epsilons, seam normals and invalid-object behavior still need native-editor comparison.
172
+
173
+ ## F34 — Transactional property edits
174
+
175
+ [Property editing](PROPERTIES.md) explicitly reorders/removes name tables and sparse values, preserving empty versus absent and reporting dropped data. All planning/validation/allocation precedes mutation. This is not a measured replacement for host interrupt ownership/UV-lock semantics.
docs/OBJ.md CHANGED
@@ -4,4 +4,4 @@
4
 
5
  Computed geometry is required; no clipping/tessellation is performed here. Polygons need at least three vertices; malformed patch array counts and non-finite geometry fail. This is structural export, not a manifold/convexity check. Editable planes/anchors, property tables, UVs and stored normals are absent from OBJ. The CLI must explicitly acknowledge those losses before publication.
6
 
7
- Limits cover encoded bytes, aggregate geometry/face elements and requested deduplication-map storage (two key/index slots per unique vertex as a conservative accounting unit, excluding allocator overhead). Reservations are fallible. Formatting uses Rust round-trip decimals, so arbitrary float text need not match historical .NET printing even though the four captures match. No original executable or asset is modified.
 
4
 
5
  Computed geometry is required; no clipping/tessellation is performed here. Polygons need at least three vertices; malformed patch array counts and non-finite geometry fail. This is structural export, not a manifold/convexity check. Editable planes/anchors, property tables, UVs and stored normals are absent from OBJ. The CLI must explicitly acknowledge those losses before publication.
6
 
7
+ Limits cover encoded bytes, aggregate geometry/face elements and requested deduplication-map storage (two key/index slots per unique vertex as a logical accounting unit, excluding hash-table capacity/control overhead and allocator overhead). Reservations are fallible. Formatting uses Rust round-trip decimals, so arbitrary float text need not match historical .NET printing even though the four captures match. No original executable or asset is modified.
docs/PROGRESS_2026-10-26.md CHANGED
@@ -13,4 +13,10 @@ Scope: extend native headless formats and geometry. Each stage is independently
13
  | 7 | P5.17 | Patch9 tessellation | Complete: tiled tensor-quadratic positions, analytic derivatives, OBJ-consistent normals, shared-position seam policy and bounded allocations/work. Planar/curved/multi-tile/degenerate/limit tests pass. |
14
  | 8 | P5.18 | Computed world pipeline/CLI | Complete: fallible editable-data copying, aggregate work/storage/position budgets, complete-or-error derived data and explicit `--compute`. Mixed-world/property/DAT/late-failure/global-limit and CLI cube→OBJ tests pass. |
15
  | 9 | P5.19 | Nearest-hit ray queries | Complete: bounded double-sided computed-surface queries, normalized rays, stable object/face/quad ties and winding normals. Cube/inside/miss/range/tie/patch/backface/budget tests pass. |
16
- | 10 | P5.20 | Property table edits/reindexing | Pending |
 
 
 
 
 
 
 
13
  | 7 | P5.17 | Patch9 tessellation | Complete: tiled tensor-quadratic positions, analytic derivatives, OBJ-consistent normals, shared-position seam policy and bounded allocations/work. Planar/curved/multi-tile/degenerate/limit tests pass. |
14
  | 8 | P5.18 | Computed world pipeline/CLI | Complete: fallible editable-data copying, aggregate work/storage/position budgets, complete-or-error derived data and explicit `--compute`. Mixed-world/property/DAT/late-failure/global-limit and CLI cube→OBJ tests pass. |
15
  | 9 | P5.19 | Nearest-hit ray queries | Complete: bounded double-sided computed-surface queries, normalized rays, stable object/face/quad ties and winding normals. Cube/inside/miss/range/tie/patch/backface/budget tests pass. |
16
+ | 10 | P5.20 | Property table edits/reindexing | Complete: explicit reorder/drop maps, sparse string/UV reindexing, removal counts and allocation-before-mutation. Inverse permutation, empty values, DAT, geometry preservation and early/late/budget rollback tests pass. |
17
+
18
+ ## Final validation
19
+
20
+ All ten stages have targeted passing tests and separate terse commits. Final macOS ARM64 validation: 82 Rust tests (67 unit, 13 integration, two doctests) pass in debug and release; formatting and Clippy with warnings denied pass. Asset-free Nix Rust/setup checks and the native CLI package build pass; setup retains 52 Python regressions. All four OBJ captures match, existing DAT/TXT fixture tests remain green, and all 1,082 hydrated Xet/LFS assets match their unchanged staged identities. Packaged CLI smoke checks exercised computed OBJ preflight and explicit Latin-1 input.
21
+
22
+ Native geometry remains an adopted, analytically tested policy. Native-editor comparison, Linux/Intel execution, broader numerical/fuzz/performance qualification, visibility/selection interactions and renderer integration remain open. The remaining broad parent TODOs were kept open; ten scoped child tasks are checked.
docs/PROPERTIES.md ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ # Native property-table editing
2
+
3
+ `nanobrush_document::remap` takes explicit old-index lists for the new string and UV tables. Reordering updates every retained sparse slot; omission removes a name and all its values, with separate dropped-name/value counts. Empty values remain present. Geometry, computed arrays, anchors and the computed flag are unchanged. Names may be Unicode or index-distinct duplicates, as in DAT; the target codec remains responsible for ASCII/TXT-name compatibility.
4
+
5
+ Orders must contain unique in-range indices; old slots must be unique, sorted and in range. The entire new name/slot plan is validated and fallibly allocated before any mutation. Errors leave the world unchanged, including late slot errors. Aggregate planning item/byte and string limits use `mb3_dat::Limits`, conservatively counting old maps and new names as well as visited records/slots. No implicit usage sorting, unused-name pruning or UV transform occurs. Renderer/document revision invalidation belongs to a future command layer.
6
+
7
+ Tests cover permutation/inverse identity, explicit removal counts, UV reindex/removal, empty values, unchanged coordinates, DAT round trip, duplicate/out-of-range orders, late invalid slots and resource-failure rollback. This is a native editing API, not a measured replacement for the host's property interrupt behavior.
docs/README.md CHANGED
@@ -40,3 +40,5 @@ The original planning backlog contained 197 tasks across 17 phases; source-resea
40
  ## Maintaining this plan
41
 
42
  Give each new finding an ID, source version/hash, confidence, and reproducible observation. Link it to a specification, a TODO item, and a validation case. Store original and generated payloads under Xet-filtered `assets/` or `references/` paths; keep original investigation prose, manifests, scripts, specifications, and Rust source in code Git. Update this index when splitting a large specification into its own document.
 
 
 
40
  ## Maintaining this plan
41
 
42
  Give each new finding an ID, source version/hash, confidence, and reproducible observation. Link it to a specification, a TODO item, and a validation case. Store original and generated payloads under Xet-filtered `assets/` or `references/` paths; keep original investigation prose, manifests, scripts, specifications, and Rust source in code Git. Update this index when splitting a large specification into its own document.
43
+
44
+ The [2026-10-26 ten-task ledger](PROGRESS_2026-10-26.md) extends the native stack with preflight reports, explicit Latin-1 input, [computed OBJ export](OBJ.md), [geometry/brush/Patch9/ray kernels](GEOMETRY.md), computed-world CLI integration and [transactional property editing](PROPERTIES.md). These native geometry policies remain separate from unverified editor parity.
docs/RUST_TXT.md CHANGED
@@ -12,7 +12,7 @@ assert_eq!(mb3_txt::decode(&bytes, &Limits::default())?, world);
12
  # Ok::<(), mb3_txt::Error>(())
13
  ```
14
 
15
- `decode` / `encode` work on bytes; `read` / `write` support standard Rust I/O, short operations and `Interrupted`. Input must be UTF-8, with an optional leading BOM. Output is canonical **UTF-8 with a BOM**, LF line endings and shortest round-tripping finite f32 decimals. Invariant number formatting preserves finite f32 values, signed zero and subnormals. Comments, whitespace, original decimal spelling and brush/patch interleaving are not retained: the model stores brushes and patches separately, and the writer emits brushes first. Names, their original table order, unused names, property indices, empty versus absent values, and all explicit geometry arrays are retained.
16
 
17
  The parser accepts line/block comments, exact case-sensitive keywords, signed decimal integers, decimal-point/exponent floats, ordinary strings and the six supplied escape forms (`\r`, `\n`, `\t`, `\'`, `\"`, `\\`). Single quotes inside strings must be escaped, as in the supplied grammar. Strings may contain Unicode directly; general `\u`, hex and unknown escapes are rejected. Non-nested block comments terminate at the first closing delimiter. Line comments may end at EOF. Unknown fields, trailing objects/text, malformed punctuation and duplicate property names/slots are errors. Sparse slots can arrive out of index order and are sorted after parsing; writer input requires unique increasing indices.
18
 
@@ -52,6 +52,6 @@ Native Unicode output follows the recovered grammar/encoding contract; a broad n
52
  - **Nine** invalid or silently lossy TXT source fixtures are rejected, including duplicate names/slots, bad indices, missing computed arrays, syntax errors and ignored polygons.
53
  - The property-optimization fixture retains `[unused, rare, common]`, while the original CLI emitted `[common, rare]`; Unicode source fixtures retain `é`, while DAT conversion refuses replacement and the historical BOM-less path drifted.
54
 
55
- Twelve unit tests cover lexer errors, comments, exponent syntax, escapes/UTF-8, finite extremes, signed zero, proper-prefix truncations, all resource policies, array consistency, streaming I/O, 3,000 bounded deterministic mutations and rejection of recursive-looking syntax. Five integration tests require hydrated E02 fixtures. Coverage-guided fuzzing, large real TXT files, explicit legacy Latin-1 recoding, native-editor interoperability and cross-platform execution remain open. See [E04](experiments/txt-cli-2026-10-21.md) for reproduction.
56
 
57
  Explicit legacy input is now available through `Encoding::Latin1`, `decode_with_encoding` and `read_with_encoding`; the CLI uses `--text-encoding latin1`. ISO-8859-1 byte values map directly to Unicode, including C1 controls; this is not CP1252. UTF-8 BOM conflicts fail. Limits apply to raw and expanded UTF-8 bytes, and transcoding storage is subtracted from decoder ownership allowance. Parse error offsets in this mode refer to expanded UTF-8 bytes. Output stays BOM-prefixed UTF-8; no heuristic fallback occurs.
 
12
  # Ok::<(), mb3_txt::Error>(())
13
  ```
14
 
15
+ `decode` / `encode` work on bytes; `read` / `write` support standard Rust I/O, short operations and `Interrupted`. Default input must be UTF-8, with an optional leading BOM. Explicit Latin-1 decoding is described below. Output is canonical **UTF-8 with a BOM**, LF line endings and shortest round-tripping finite f32 decimals. Invariant number formatting preserves finite f32 values, signed zero and subnormals. Comments, whitespace, original decimal spelling and brush/patch interleaving are not retained: the model stores brushes and patches separately, and the writer emits brushes first. Names, their original table order, unused names, property indices, empty versus absent values, and all explicit geometry arrays are retained.
16
 
17
  The parser accepts line/block comments, exact case-sensitive keywords, signed decimal integers, decimal-point/exponent floats, ordinary strings and the six supplied escape forms (`\r`, `\n`, `\t`, `\'`, `\"`, `\\`). Single quotes inside strings must be escaped, as in the supplied grammar. Strings may contain Unicode directly; general `\u`, hex and unknown escapes are rejected. Non-nested block comments terminate at the first closing delimiter. Line comments may end at EOF. Unknown fields, trailing objects/text, malformed punctuation and duplicate property names/slots are errors. Sparse slots can arrive out of index order and are sorted after parsing; writer input requires unique increasing indices.
18
 
 
52
  - **Nine** invalid or silently lossy TXT source fixtures are rejected, including duplicate names/slots, bad indices, missing computed arrays, syntax errors and ignored polygons.
53
  - The property-optimization fixture retains `[unused, rare, common]`, while the original CLI emitted `[common, rare]`; Unicode source fixtures retain `é`, while DAT conversion refuses replacement and the historical BOM-less path drifted.
54
 
55
+ Twelve unit tests cover lexer errors, comments, exponent syntax, escapes/UTF-8, finite extremes, signed zero, proper-prefix truncations, all resource policies, array consistency, streaming I/O, 3,000 bounded deterministic mutations and rejection of recursive-looking syntax. Five integration tests require hydrated E02 fixtures. Coverage-guided fuzzing, large real TXT files, native-editor interoperability and cross-platform execution remain open. See [E04](experiments/txt-cli-2026-10-21.md) for reproduction.
56
 
57
  Explicit legacy input is now available through `Encoding::Latin1`, `decode_with_encoding` and `read_with_encoding`; the CLI uses `--text-encoding latin1`. ISO-8859-1 byte values map directly to Unicode, including C1 controls; this is not CP1252. UTF-8 BOM conflicts fail. Limits apply to raw and expanded UTF-8 bytes, and transcoding storage is subtracted from decoder ownership allowance. Parse error offsets in this mode refer to expanded UTF-8 bytes. Output stays BOM-prefixed UTF-8; no heuristic fallback occurs.
docs/TODO.md CHANGED
@@ -1,6 +1,6 @@
1
  # Exhaustive work list
2
 
3
- Baseline: 2026-10-21. The setup and extraction implementation is complete; checked items are verified in [SETUP.md](SETUP.md). The single-repository layout supersedes the initial separate-dataset proposal. Findings F01–F31 are in [FINDINGS.md](FINDINGS.md). The [investigation method](REVERSE_ENGINEERING.md) defines evidence requirements; [technical direction](TECHNICAL_DIRECTION.md) describes proposed boundaries.
4
 
5
  Each task ID is stable. Split tasks into child IDs as evidence grows. Record dependencies, output documents/code, validation evidence, and remaining limitations when checking a task off. New discoveries extend this list. P0–P2 establish preservation; P3–P9 establish behavioral contracts; P10–P16 implement and prove the rewrite. P10's GPU spike should happen early alongside source research, before renderer architecture is locked.
6
 
@@ -136,6 +136,8 @@ Each task ID is stable. Split tasks into child IDs as evidence grows. Record dep
136
 
137
  - [x] P5.19 Implement bounded deterministic native nearest-hit ray queries over computed brush/patch surfaces ([GEOMETRY.md](GEOMETRY.md)).
138
 
 
 
139
  ## P6 — File formats and converters
140
 
141
  **Depends on:** P3 oracle where available, P5 document model. **Exit:** standalone format specifications, declared compatibility subsets, fixture corpus, and native codecs.
 
1
  # Exhaustive work list
2
 
3
+ Baseline: 2026-10-21. The setup and extraction implementation is complete; checked items are verified in [SETUP.md](SETUP.md). The single-repository layout supersedes the initial separate-dataset proposal. Findings F01–F34 are in [FINDINGS.md](FINDINGS.md). The [investigation method](REVERSE_ENGINEERING.md) defines evidence requirements; [technical direction](TECHNICAL_DIRECTION.md) describes proposed boundaries.
4
 
5
  Each task ID is stable. Split tasks into child IDs as evidence grows. Record dependencies, output documents/code, validation evidence, and remaining limitations when checking a task off. New discoveries extend this list. P0–P2 establish preservation; P3–P9 establish behavioral contracts; P10–P16 implement and prove the rewrite. P10's GPU spike should happen early alongside source research, before renderer architecture is locked.
6
 
 
136
 
137
  - [x] P5.19 Implement bounded deterministic native nearest-hit ray queries over computed brush/patch surfaces ([GEOMETRY.md](GEOMETRY.md)).
138
 
139
+ - [x] P5.20 Implement transactional native property table reorder/removal and sparse reindexing with explicit loss counts ([PROPERTIES.md](PROPERTIES.md)).
140
+
141
  ## P6 — File formats and converters
142
 
143
  **Depends on:** P3 oracle where available, P5 document model. **Exit:** standalone format specifications, declared compatibility subsets, fixture corpus, and native codecs.
docs/parity.md CHANGED
@@ -45,3 +45,7 @@ Keep the broad parent TODO tasks open until their promised behavioral evidence e
45
  ## TXT and CLI implementation (E04)
46
 
47
  The native TXT codec and CLI cover all 58 E02 DAT outputs through exact DAT→TXT→DAT reconstruction, seven text outputs through canonical semantic round trips, 16 direct TXT-input/DAT-output pairs and nine rejected malformed/lossy text inputs. Original formatting, guessed encodings, ignored arrays and property optimization are explicit differences in F30. CLI no-clobber/force publication, alias/symlink guards, validation before I/O and exit streams are tested separately in F31. Legacy external formats, editor behavior and other-platform execution remain open.
 
 
 
 
 
45
  ## TXT and CLI implementation (E04)
46
 
47
  The native TXT codec and CLI cover all 58 E02 DAT outputs through exact DAT→TXT→DAT reconstruction, seven text outputs through canonical semantic round trips, 16 direct TXT-input/DAT-output pairs and nine rejected malformed/lossy text inputs. Original formatting, guessed encodings, ignored arrays and property optimization are explicit differences in F30. CLI no-clobber/force publication, alias/symlink guards, validation before I/O and exit streams are tested separately in F31. Legacy external formats, editor behavior and other-platform execution remain open.
48
+
49
+ ## Ten-task extension — 2026-10-26
50
+
51
+ [Ledger](PROGRESS_2026-10-26.md): preflight, explicit Latin-1, computed OBJ/CLI, analytic geometry, strict convex brushes, native Patch9 evaluation, bounded world computation, deterministic rays and transactional property editing. OBJ has four exact reference goldens. Geometry/property APIs use documented native contracts with analytic tests; native editor parity, visibility/selection semantics and broad numerical/performance qualification remain unverified.
docs/validation.md CHANGED
@@ -31,3 +31,9 @@ Both Apple Silicon Nix derivations built successfully: the asset-free Rust check
31
  [TXT compatibility](RUST_TXT.md), [CLI workflow](CLI.md) and [E04](experiments/txt-cli-2026-10-21.md) record the implementation. Full debug and release suites each pass **58 Rust tests**: 44 unit tests, 12 fixture/process integration tests and two doctests. All 58 DAT outputs survive exact DAT→TXT→DAT reconstruction through the real executable, seven converter TXT outputs canonicalize stably, 16 direct TXT→DAT pairs match and nine invalid/lossy TXT source fixtures are rejected. The TXT suite adds 3,000 bounded mutations and streaming/array/encoding limits; CLI tests cover publication races, aliases, symlinks, no-clobber and preserving existing destinations on encoder failure.
32
 
33
  Formatting/Clippy pass. Both Apple Silicon Nix checks and the asset-free `nanobrush` package build pass; the Rust check runs 44 unit tests/two doctests, while setup retains all 52 Python regressions. The packaged executable’s help and JSON inspection of the property fixture ran successfully. Converter fixture identity/semantic checks and all 1,082 hydrated asset hashes pass; original captures are unchanged and tests/code remain ordinary Git. Other-platform execution, native-editor interoperability, crash-durability/failure matrices and coverage-guided fuzzing remain open.
 
 
 
 
 
 
 
31
  [TXT compatibility](RUST_TXT.md), [CLI workflow](CLI.md) and [E04](experiments/txt-cli-2026-10-21.md) record the implementation. Full debug and release suites each pass **58 Rust tests**: 44 unit tests, 12 fixture/process integration tests and two doctests. All 58 DAT outputs survive exact DAT→TXT→DAT reconstruction through the real executable, seven converter TXT outputs canonicalize stably, 16 direct TXT→DAT pairs match and nine invalid/lossy TXT source fixtures are rejected. The TXT suite adds 3,000 bounded mutations and streaming/array/encoding limits; CLI tests cover publication races, aliases, symlinks, no-clobber and preserving existing destinations on encoder failure.
32
 
33
  Formatting/Clippy pass. Both Apple Silicon Nix checks and the asset-free `nanobrush` package build pass; the Rust check runs 44 unit tests/two doctests, while setup retains all 52 Python regressions. The packaged executable’s help and JSON inspection of the property fixture ran successfully. Converter fixture identity/semantic checks and all 1,082 hydrated asset hashes pass; original captures are unchanged and tests/code remain ordinary Git. Other-platform execution, native-editor interoperability, crash-durability/failure matrices and coverage-guided fuzzing remain open.
34
+
35
+ ## Ten-task native extension — 2026-10-26
36
+
37
+ See [the stage ledger](PROGRESS_2026-10-26.md), [OBJ](OBJ.md), [geometry](GEOMETRY.md) and [property edits](PROPERTIES.md). Final debug/release suites each pass 82 Rust tests: 67 unit, 13 integration and two doctests. This includes four exact OBJ goldens, existing 58 DAT/TXT CLI round trips, analytic brush/patch/ray cases, resource failures and transactional property edits. Formatting/Clippy pass. Both Apple Silicon asset-free Nix checks and the CLI package build pass; the setup suite retains 52 Python regressions. Packaged CLI computed-export preflight and explicit Latin-1 inspection ran successfully.
38
+
39
+ Converter evidence identities and all 1,082 hydrated Xet/LFS asset hashes pass unchanged. Authored code/tests/docs remain ordinary Git. Native geometry numeric/interior/seam/picking policies are documented choices, not measured editor parity. Other-platform execution and broader legacy/real-document/fuzz/hardware qualification remain open.