tracegrpo / docs /PHASE3_COMPLETION.md
Dhruv1000's picture
Publish verified Phase 3 cleanup journal, maintenance archive and 569-test evidence
1358401 verified
|
Raw History Blame Contribute Delete
4.22 kB

Phase 3 — HF browsing projection cleanup

Scope: approved by the user on 2026-10-12. This changes loose HF cache copies only; every local research path and all recovery archives remain intact. No training, inference, new measurements, protected dataset reads, or TEST access.

Classification and exact-byte proof

Baseline HF revision: cc8badbdd9d694d1212579a326eb5ace5df8526c; 17,641 files, including 16,006 data files.

Category Action Files Logical bytes
visible_dataset_split_provenance_or_index keep_visible 74 1,007,196,779
valid_reward_graph_cache remove_loose_remote_copy 831 1,212,018
train_reward_graph_cache remove_loose_remote_copy 5,770 7,624,131
main_teacher_batch_history remove_loose_remote_copy 166 14,351,206
supplement_teacher_batch_history remove_loose_remote_copy 26 1,461,434
topup_teacher_batch_history remove_loose_remote_copy 96 3,989,993
api_response_cache remove_loose_remote_copy 9,035 497,647,201
api_response_cache keep_visible 8 257,504

15,924 loose copies qualify for archive-only browsing; 82 data paths remain visible. The archive and every selected member were freshly SHA-256 verified; member bytes also match each current remote Git blob SHA-1 or LFS SHA-256. Protected dataset payload members were not opened, extracted, or parsed. The compressed archive was read for opaque integrity hashing and selected cache extraction only.

Existing archive: releases/2026-10-12-complete/data-01.tar.gz, SHA-256 5cea58e5a4add4cf384742891c8c303312953744a5da52c43b811ccbe907fbb4. The archive is reused without another large upload.

Eight API responses have sanitized recovery copies, so exact byte equivalence is not established for those loose paths. They remain unchanged as explicit exceptions. They are not copied into a new archive, displayed, or republished. This cleanup makes no credential-remediation claim.

What counts as clutter

The label applies to thousands of individually browsable cache and per-batch history files, not to their research value. They are retained byte-for-byte in an existing verified recovery archive. Code, fitted models, checkpoint paths, raw measurements, original prompts, composed datasets, split manifests, provenance logs, and unique results remain available. Small empty placeholders remain because removing them has no useful browsing benefit. Historical failures and infrastructure-loss artifacts remain evidence.

Dependencies and recovery

Historical runners still reference the original cache directories. Those local directories are unchanged. A fresh HF download must restore the caches before using those paths. The maintained restore tool validates archive and member hashes, accepts only six cache/batch prefixes, rejects TEST/dataset paths and links, and refuses an existing destination. It starts no jobs. See cache restore guide. Static references are recorded in the release bundle.

Verification and publication

The release directory contains the complete per-path projection plan, category summary, scope inventory, verification evidence, commit journal and publication status. Remote removal uses explicit paths, pinned object identities, reversible HF commits, and a pre-removal plan backup. All other remote files are checked for unchanged object identities, except the explicitly reviewed documentation/tool overlay. No local research files are removed, no recovery archives are deleted, and no storage-byte savings are claimed.

Final test count, remote counts and archive hash are recorded in publication status. This is a repository-maintenance result, not another scientific experiment.

HF automatically added one .gitattributes LFS rule for the 10.9 MB per-path projection plan. The final integrity guard caught it; exact text comparison verified that every existing rule was unchanged. This explicit metadata exception is recorded in hf_attributes_verification.json; no research payload was migrated.