RoboTTT: common official-base comparison
Public experiment backup, as authorized by the repository owner. Progress is updated approximately every five minutes while the backup worker runs.
See progress/comparison-status.json and backup-manifest.json for current status and durable checkpoint steps. smoke-* runs are two-update integration checks, not completed 3000-update experiments. Main runs are named comparison-lite and comparison-reference.
Both conditions use the same converted official pi05_base, Train episodes 0..59 and Dev 60..69, T32/stride1/S8, and saved schedule/noise. The comparison is between a four-layer lightweight TTT design (561,540 added trainable parameters) and the pinned OpenTau core on all 18 layers (85,319,824). This is a new common pipeline, not a reproduction of the historical full OpenTau trainer.
Extension/optimizer checkpoints, seeded protocols, preprocessing contracts, code and progress are backed up every five minutes. The converted base and exact source/processed data are mirrored separately; see progress/hf-assets-backup-status.json. Exact base provenance and hashes are in provenance/base_comparison_assets.json.
VLM prefixes are generated on demand with a bounded 2 GiB RAM cache. No large prefix tensor files are written to ephemeral storage. Sequence tensor-hash contracts are shared and checked between both methods.
The exact Python environment is stored as environment/runtime-env.tar.gz with a checksum manifest. Its runtime directory is disposable and restored from this archive after a restart, avoiding tens of thousands of persistent small files. Check progress/hf-runtime-backup-status.json for upload completion. Resume using python3 scripts/resume_comparison.py; saved optimizer/RNG state is preserved and unsaved interrupted work is archived and rolled back explicitly.
Flow-matching loss and simulator success are separate metrics; neither should be inferred from a smoke checkpoint. Final metrics appear in progress/comparison-results.json only after paired evaluations complete.