tracegrpo / docs /PHASE1_COMPLETION.md
Dhruv1000's picture
Publish explicitly approved frozen aggregates and indexes omitted by source ignore rules
bf5d7c4 verified
|
Raw History Blame Contribute Delete
6.13 kB

Phase 1 repository cleanup — completion record

Scope and outcome

Implemented the user's approved Phase 1: presentation, navigation, small exact duplicates and future Git LFS tracking. Research execution remains closed. No TEST split, TEST dataset rows or raw TEST predictions were opened; no inference, training, data recomputation or checkpoint migration occurred.

The master evidence report and two original prompts remain byte-identical. Frozen data, results, checkpoint and tooling payloads remain at their original paths. Phase 2 large-artifact deduplication and Phase 3 HF data projection remain unapproved.

Changes

  • Rebuilt README around final findings, source-linked metrics, honest limitations, repository navigation and disaster recovery.
  • Rendered 10 scientific figures in SVG/PNG and one measured C-L1 GIF. Captions and source paths; input/output SHA-256; generator.
  • Replaced 114 decorative report-wrapper pages with one historical index. Every original scientific report stays in results/.
  • Archived five obsolete decorative images and linked current documentation to the scientific figure set. Previous figure and documentation bytes remain in the verified preservation archive.
  • Canonicalized judge-package consumers to audit/llm_judge/package. A relative local alias keeps historical root-path callers working without duplicated card payloads.
  • Preserved the two distinct old judge manifests as real, byte-identical files under docs/history/legacy_judge_package/.
  • Added the code guide, updated the documentation index and HF archive guide, and fixed maintained navigation targets.
  • Preserved all nine previously uncommitted source/test files. Eight are byte-identical; the analyzer's only approved change is its canonical package path. Original source bytes are backed up.
  • Added .gitattributes aligned with the actual parent Git root. PNG/GIF and durable binary artifacts use LFS for future commits; existing history is not migrated or rewritten. Existing HF attribute rules are retained.

Exact removals and storage accounting

Action Files Logical bytes
Exact judge payload duplicates removed after canonical re-hash 152 636,382
Redundant historical wrapper pages archived, then removed live 114 64,906
Obsolete decorative images archived, then removed live 5 31,481
Unique judge manifests relocated, not discarded 2 19,081
Total superseded live paths 273 751,850

The duplicate removal eliminates 636,382 logical bytes of redundant live payloads. This is a presentation cleanup, not a large storage-reclamation phase. New high-resolution figure exports use 1,411,328 bytes versus 60,473 bytes in the previous root figure collection; preservation archives and new assets add storage. No claim of net disk savings is made. Large checkpoint, dataset, environment and archive bytes are untouched.

Every removed live path is listed in the removal map, with a preserved destination/archive member and SHA-256. Original presentation archive: 24f049db7c5bd7379db8b1b3e2958861b6b1d967ad271f920f52d023e6f3f60f; remotely downloaded and verified before any removals, revision fd8b4790fcb94d6719a3e7ebca47ab8585354d57.

Verification

Check Result
Full suite 552 passed in 32.42 seconds; two added canonical-path/validation regressions
Frozen payload metadata 20,051 checked; zero changed payloads, zero missing files
Accepted grading cards 150 sequential cards; 75 reasoning / 75 no-reasoning; blocklist passes; content/order checked against accepted blinded deck
Existing grades 150 records validated against their frozen SHA-256 and dimension rules
Exact canonical duplicates 152 hashes match pre-cleanup originals
Original prompts Both byte-identical
Master report Byte-identical
Internal imports 490 edges resolve; no syntax errors
Figure provenance Every registered input/output hash verifies
LFS PNG/GIF, adapters, state/archive binaries match rules; SVG stays ordinary text; no history migration

Evidence: layout/integrity/test logs are included in the release bundle. A legacy test refreshes the mtime of results/phase10/human_audit/blinding_check.json; its SHA-256 remains unchanged and the side effect is recorded.

Historical reference exception: the original literature-review prompt contains five image links to assets never supplied with it. Their absence is explained in the prompt index. Original prompt bytes are preserved; no replacement experimental evidence is invented. These are not counted as working current navigation links. Historical raw reports also retain their original text.

Publication

The cleaned browsing view, all original/replaced presentation bytes, changed source and verification records are published under private HF Dhruv1000/tracegrpo. Release index describes payload manifests and the machine-readable remote verification status. Existing large bundles are retained without re-upload.

Phase 2 proposal — approval required

Review the 228 verified duplicate hash groups individually, beginning with repeated immutable tokenizer files and identical adapter payloads. Before consolidating a checkpoint, verify optimizer/scheduler/RNG and step provenance as well as weights. Preserve every expected load/restore path; test aliases or hardlinks before applying them. No dataset, unique checkpoint state, result, interrupted-run evidence or recovery archive is removed. No Git history pruning, new experiments or TEST reads.

The audit's approximately 1.56 GB repeated-byte estimate is an upper-bound candidate inventory, not guaranteed reclaimable storage. Proceed only after explicit user approval of Phase 2's exact file-level plan.