Download docs/experiments/txt-cli-2026-10-21.md from nmarshall/nanobrush: direct link, hf CLI and curl.
- Browser
- Download file 3.99 kB
-
https://huggingface.co/nmarshall/nanobrush/resolve/main/docs/experiments/txt-cli-2026-10-21.md
- Command line
-
hf download hf://nmarshall/nanobrush/docs/experiments/txt-cli-2026-10-21.md
-
curl -L -o txt-cli-2026-10-21.md https://huggingface.co/nmarshall/nanobrush/resolve/main/docs/experiments/txt-cli-2026-10-21.md
E04 — TXT codec and native conversion CLI
Date: 2026-10-21. Implements F30/F31: bounded TXT serialization and a native inspect/validate/convert workflow. Source anchors are the two supplied TXT grammars and E02 converter behavior, plus the shared F29 DAT model. No original binary was executed or modified by this stage; E02's existing Xet fixtures were reused.
Reproduction
In the pinned Nix development environment (Rust/Cargo 1.99.0, macOS ARM64):
analyze-converter check
cargo fmt --all --check
cargo clippy --workspace --all-targets --locked --offline -- -D warnings
cargo test --workspace --locked --offline
cargo test --workspace --release --locked --offline
nix build .#checks.aarch64-darwin.rust .#checks.aarch64-darwin.setup .#nanobrush --no-link
nix run .#nanobrush -- --help
Full integration tests require the hydrated assets/analysis/converter/probes tree. NANOBRUSH_CONVERTER_FIXTURES can select an explicitly supplied alternative. E02's SHA-256-indexed report remains the identity anchor; analyze-converter check verifies input/output hashes and original converter identities. No golden fixture payload was moved into normal Git or embedded into a source file. Synthetic tests construct their own small grammar examples and boundary inputs.
Evidence and acceptance
- 58 original DAT outputs survive native DAT→TXT→DAT with exact byte equality, both through libraries and through the real CLI process. Every successful CLI conversion also validates its text output; inspect JSON summaries identify the source format.
- All seven supplied converter TXT outputs parse and stabilize under native formatting. Sixteen non-lossy TXT→DAT fixture pairs reproduce exact reference output. Nine invalid or silently lossy TXT inputs are rejected.
- UTF-8 input preserves Unicode rather than reproducing Latin-1 guessing; TXT output includes a BOM. ASCII DAT export refuses character loss. Properties retain unused names and original index meaning, and down-conversion refuses to discard properties.
- Parser limits include UTF-8 string bytes, total logical elements, requested ownership/scratch and complete input/output bytes. Numbers are bounded to 128 token bytes and finite f32; integer fields are nonnegative i32. Arrays must agree with computed flags and dimensions. There is no recursive parse tree. Three thousand deterministic TXT mutations join the prior 5,000 DAT mutation smoke cases; neither is coverage-guided fuzzing.
- The CLI validates/encodes before destination staging, refuses accidental replacement, rejects source aliases/output symlinks, and atomically publishes through hard-link/no-clobber or explicit-force rename. A publication test exercises an existing destination after staging and verifies that existing bytes survive and the owned temp is removed.
The workspace has 44 unit tests (24 DAT, 12 TXT, eight CLI), 12 integration tests (five DAT, five TXT, two CLI) and two doctests: 58 Rust tests. Code-only Nix tests intentionally run the 44 unit tests/two doctests and compile/lint integration targets without opening assets; full fixture tests run separately. The unchanged Python suite has 52 regressions. Final observed results are recorded in validation.
Limits
This establishes codec and CLI behavior for the supplied fixture domain and authored adversarial cases, not native editor interoperability, historical byte-identical TXT printing or every possible filesystem failure. The native codec's encoding, strict computed consistency, number limits, preserved properties and canonical layout differ intentionally from the legacy converters; see F30 and F31. MAP/VMF/OBJ, native document schema, general loss reports, coverage-guided fuzzing, legacy Latin-1 recoding, independent consumer checks, native-editor tests and Linux/Intel execution remain open. The Nix package contains the native CLI only; no Bevy/Solari/editor feature is claimed by this stage.