acts-sensoronly-ablation / REPRO_CHECKS.md
WitneyWW's picture
add code (configs, package, eval, scripts, README)
b1b6f13 verified
|
Raw History Blame Contribute Delete
2.05 kB

Reproducibility checks (2026-11-13, before upload)

Setup, as on a new server: the code copied to a clean directory (no access to the original vtm_cosmos/ tree), a fresh conda env built by scripts/setup_env.sh (ENV_NAME=acts_so_test: torch 2.10.0+cu128, cuDNN 91002, imageio-ffmpeg 0.6.0), cosmos-framework freshly cloned @ 3f94d25, nvidia/Cosmos3-Nano and Wan2.2_VAE.pth freshly downloaded with scripts/download.py --what nano wan_vae, data/ and checkpoints/ = exactly the files uploaded to this repo. 1× H100 80 GB per job.

check what result
training smoke, all 8 configs NGPU=1 scripts/train.sh configs/<cfg>.yaml run_name=smoke_<cfg> training.max_steps=30 all rc=0, train + val pass, 30.5 GiB peak
windows train / val windows per task mb 71,386 / 109, pushT 55,199 / 92, rope 77,188 / 126 (= original runs)
884599 config action normaliser vs runs/nano_action_causal_umi_sensoronly_noforce_v1 (job 884599); trainable params byte-identical; 45,301,952 (view_action2llm9 frozen)
TODO 1a / 2a (sensor-only, force) normaliser vs pusht_v1 / rope_v1 identical (pushT); rope differs from rope_noforce only at dp_y, as intended
TODO 1b / 2b (sensor-only, no force) normaliser vs the earlier pusht_noforce_v1 / rope_noforce_v1 runs byte-identical / identical
inference pusht_v1 e019 eval/run_test_case.py --task pusht: case 0 (short + 15-window long), cases 1-3 short PSNR/SSIM/LPIPS Δ = 0 vs eval/cases/pusht_reference_metrics.json
inference rope_v1 e019 same, --task rope Δ = 0
inference 884599 e013 same, --task umi_sensoronly_noforce Δ = 0 vs the original cluster eval of epoch_013
summarize eval/summarize.py --task … --reference runs; paired Δ = 0.000

Not re-checked in this round (unchanged code, checked for the 11-08 release WitneyWW/acts-world-model): re-encoding latents from data/raw (bit-identical) and rebuilding the dforce actions of all 166 episodes from data/raw/parquet (bit-identical).