acts-sensoronly-ablation / REPRO_CHECKS.md
WitneyWW's picture
add code (configs, package, eval, scripts, README)
b1b6f13 verified
|
Raw History Blame Contribute Delete
2.05 kB
# Reproducibility checks (2026-11-13, before upload)
Setup, as on a new server: the code copied to a clean directory (no access to the original `vtm_cosmos/` tree), a **fresh**
conda env built by `scripts/setup_env.sh` (`ENV_NAME=acts_so_test`: torch 2.10.0+cu128, cuDNN 91002, imageio-ffmpeg 0.6.0),
cosmos-framework freshly cloned @ 3f94d25, `nvidia/Cosmos3-Nano` and `Wan2.2_VAE.pth` freshly downloaded with
`scripts/download.py --what nano wan_vae`, `data/` and `checkpoints/` = exactly the files uploaded to this repo. 1× H100 80 GB per job.
| check | what | result |
|---|---|---|
| training smoke, all 8 configs | `NGPU=1 scripts/train.sh configs/<cfg>.yaml run_name=smoke_<cfg> training.max_steps=30` | all rc=0, train + val pass, 30.5 GiB peak |
| windows | train / val windows per task | mb 71,386 / 109, pushT 55,199 / 92, rope 77,188 / 126 (= original runs) |
| 884599 config | action normaliser vs `runs/nano_action_causal_umi_sensoronly_noforce_v1` (job 884599); trainable params | byte-identical; 45,301,952 (view_action2llm9 frozen) |
| TODO 1a / 2a (sensor-only, force) | normaliser vs pusht_v1 / rope_v1 | identical (pushT); rope differs from rope_noforce only at `dp_y`, as intended |
| TODO 1b / 2b (sensor-only, no force) | normaliser vs the earlier `pusht_noforce_v1` / `rope_noforce_v1` runs | byte-identical / identical |
| inference pusht_v1 e019 | `eval/run_test_case.py --task pusht`: case 0 (short + 15-window long), cases 1-3 short | PSNR/SSIM/LPIPS Δ = 0 vs `eval/cases/pusht_reference_metrics.json` |
| inference rope_v1 e019 | same, `--task rope` | Δ = 0 |
| inference 884599 e013 | same, `--task umi_sensoronly_noforce` | Δ = 0 vs the original cluster eval of epoch_013 |
| summarize | `eval/summarize.py --task … --reference` | runs; paired Δ = 0.000 |
Not re-checked in this round (unchanged code, checked for the 11-08 release `WitneyWW/acts-world-model`): re-encoding latents from
`data/raw` (bit-identical) and rebuilding the dforce actions of all 166 episodes from `data/raw/parquet` (bit-identical).