Download forgebench/code/ours/batch_equivalence.md from Ronaldo-GOAT/bert_simpson: direct link, hf CLI and curl.
- Browser
- Download file 10.3 kB
-
https://huggingface.co/Ronaldo-GOAT/bert_simpson/resolve/main/forgebench/code/ours/batch_equivalence.md
- Command line
-
hf download hf://Ronaldo-GOAT/bert_simpson/forgebench/code/ours/batch_equivalence.md
-
curl -L -o batch_equivalence.md https://huggingface.co/Ronaldo-GOAT/bert_simpson/resolve/main/forgebench/code/ours/batch_equivalence.md
Batch-equivalence check: batch of 2 vs 2 separate single inferences
Method batched: Ours (SS+SLAT) only (the only method with a batched code path). All baselines (ReconViaGen, Pixal3D, Amodal3R, Hunyuan3D-2mv, Cupid) were run UNBATCHED (one object per forward, their official per-object drivers); speed-up for them comes only from co-locating several single-object workers per GPU, which does not change any computation -> no equivalence test needed/applicable.
Runs (same ckpts SS=ssflow_seedcond_v3_20260916/step_0025000, SLAT=slatflow_prod_20260902/step_0020000, seed 42, same 2 objects FaGr_s000001_butternutsquash1 + FaGr_s000002_yellowmustardbottle5):
single= ours_combined_cell.sh (ssflow_coords.py, val_daemon single:true), GPU 4single2= identical repeat ofsingle(GPU 6) -> run-to-run NOISE FLOOR of the unbatched pathbatched= ours_combined_cell_batched.sh SSB=2 S2B=2 (ssflow_coords_batched.py --batch 2; val_daemon batch_size 2, single:false), GPU 5
1v
| object | pair | SS coords (#) | coords sym-diff (#voxels) | V/F a | V/F b | max abs vert diff | Chamfer-L1 (50k) | Chamfer sampling floor (a vs a) |
|---|---|---|---|---|---|---|---|---|
| FaGr_s000001_butternutsquash1 | single vs batched | 9443/9443 | 0 | 206160/412284 | 206152/412268 | n/a (V differ) | 0.0033 | 0.0033 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | 9443/9443 | 0 | 206160/412284 | 206164/412292 | n/a (V differ) | 0.0033 | 0.0033 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | 9766/9765 | 1 | 187950/375812 | 188114/376128 | n/a (V differ) | 0.0033 | 0.0032 |
| FaGr_s000002_yellowmustardbottle5 | single vs single2 (floor) | 9766/9766 | 0 | 187950/375812 | 187878/375664 | n/a (V differ) | 0.0032 | 0.0032 |
| object | pair | metric | a | b | abs diff |
|---|---|---|---|---|---|
| FaGr_s000001_butternutsquash1 | single vs batched | input_lpips | 0.10903 | 0.10905 | 2.14e-05 |
| FaGr_s000001_butternutsquash1 | single vs batched | input_psnr | 24.69652 | 24.70546 | 8.94e-03 |
| FaGr_s000001_butternutsquash1 | single vs batched | input_ssim | 0.90333 | 0.90334 | 8.94e-06 |
| FaGr_s000001_butternutsquash1 | single vs batched | novel_lpips | 0.07460 | 0.07455 | 4.61e-05 |
| FaGr_s000001_butternutsquash1 | single vs batched | novel_psnr | 22.07094 | 22.07439 | 3.45e-03 |
| FaGr_s000001_butternutsquash1 | single vs batched | novel_ssim | 0.94951 | 0.94951 | 7.21e-06 |
| FaGr_s000001_butternutsquash1 | single vs batched | geome_cd_l1 | 0.04028 | 0.04031 | 3.34e-05 |
| FaGr_s000001_butternutsquash1 | single vs batched | geome_f01 | 0.38985 | 0.38904 | 8.16e-04 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | input_lpips | 0.10903 | 0.10933 | 3.03e-04 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | input_psnr | 24.69652 | 24.70208 | 5.56e-03 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | input_ssim | 0.90333 | 0.90333 | 7.15e-07 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | novel_lpips | 0.07460 | 0.07454 | 5.74e-05 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | novel_psnr | 22.07094 | 22.07243 | 1.49e-03 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | novel_ssim | 0.94951 | 0.94952 | 1.32e-05 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | geome_cd_l1 | 0.04028 | 0.04028 | 2.61e-06 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | geome_f01 | 0.38985 | 0.38961 | 2.44e-04 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | input_lpips | 0.07058 | 0.07107 | 4.95e-04 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | input_psnr | 18.65362 | 18.75006 | 9.64e-02 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | input_ssim | 0.87413 | 0.87469 | 5.63e-04 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | novel_lpips | 0.05281 | 0.05350 | 6.87e-04 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | novel_psnr | 21.62769 | 21.60999 | 1.77e-02 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | novel_ssim | 0.94570 | 0.94571 | 8.44e-06 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | geome_cd_l1 | 0.01265 | 0.01258 | 6.85e-05 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | geome_f01 | 0.52419 | 0.52598 | 1.79e-03 |
| FaGr_s000002_yellowmustardbottle5 | single vs single2 (floor) | input_lpips | 0.07058 | 0.07036 | 2.18e-04 |
| FaGr_s000002_yellowmustardbottle5 | single vs single2 (floor) | input_psnr | 18.65362 | 18.65291 | 7.10e-04 |
| FaGr_s000002_yellowmustardbottle5 | single vs single2 (floor) | input_ssim | 0.87413 | 0.87409 | 4.23e-05 |
| FaGr_s000002_yellowmustardbottle5 | single vs single2 (floor) | novel_lpips | 0.05281 | 0.05280 | 1.12e-05 |
| FaGr_s000002_yellowmustardbottle5 | single vs single2 (floor) | novel_psnr | 21.62769 | 21.63569 | 7.99e-03 |
| FaGr_s000002_yellowmustardbottle5 | single vs single2 (floor) | novel_ssim | 0.94570 | 0.94571 | 1.22e-05 |
| FaGr_s000002_yellowmustardbottle5 | single vs single2 (floor) | geome_cd_l1 | 0.01265 | 0.01267 | 1.40e-05 |
| FaGr_s000002_yellowmustardbottle5 | single vs single2 (floor) | geome_f01 | 0.52419 | 0.52383 | 3.57e-04 |
4v
| object | pair | SS coords (#) | coords sym-diff (#voxels) | V/F a | V/F b | max abs vert diff | Chamfer-L1 (50k) | Chamfer sampling floor (a vs a) |
|---|---|---|---|---|---|---|---|---|
| FaGr_s000001_butternutsquash1 | single vs batched | 7778/7777 | 1 | 267276/536392 | 267868/537604 | n/a (V differ) | 0.0033 | 0.0033 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | 7778/7778 | 0 | 267276/536392 | 267108/536024 | n/a (V differ) | 0.0033 | 0.0033 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | 10256/10256 | 0 | 208384/416792 | 208240/416520 | n/a (V differ) | 0.0034 | 0.0034 |
| FaGr_s000002_yellowmustardbottle5 | single vs single2 (floor) | - | - | - | - | - | missing mesh (decode OOM in single2 run, GPU oversubscribed) |
| object | pair | metric | a | b | abs diff |
|---|---|---|---|---|---|
| FaGr_s000001_butternutsquash1 | single vs batched | input_lpips | 0.10835 | 0.10771 | 6.42e-04 |
| FaGr_s000001_butternutsquash1 | single vs batched | input_psnr | 24.20930 | 24.22512 | 1.58e-02 |
| FaGr_s000001_butternutsquash1 | single vs batched | input_ssim | 0.92051 | 0.92061 | 9.86e-05 |
| FaGr_s000001_butternutsquash1 | single vs batched | novel_lpips | 0.03706 | 0.03715 | 9.50e-05 |
| FaGr_s000001_butternutsquash1 | single vs batched | novel_psnr | 27.21692 | 27.24218 | 2.53e-02 |
| FaGr_s000001_butternutsquash1 | single vs batched | novel_ssim | 0.96570 | 0.96579 | 8.91e-05 |
| FaGr_s000001_butternutsquash1 | single vs batched | geome_cd_l1 | 0.00443 | 0.00441 | 2.08e-05 |
| FaGr_s000001_butternutsquash1 | single vs batched | geome_f01 | 0.90865 | 0.91001 | 1.37e-03 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | input_lpips | 0.10835 | 0.10829 | 5.63e-05 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | input_psnr | 24.20930 | 24.20380 | 5.49e-03 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | input_ssim | 0.92051 | 0.92050 | 8.91e-06 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | novel_lpips | 0.03706 | 0.03699 | 6.97e-05 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | novel_psnr | 27.21692 | 27.21410 | 2.82e-03 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | novel_ssim | 0.96570 | 0.96572 | 1.36e-05 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | geome_cd_l1 | 0.00443 | 0.00443 | 1.96e-06 |
| FaGr_s000001_butternutsquash1 | single vs single2 (floor) | geome_f01 | 0.90865 | 0.90868 | 3.92e-05 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | input_lpips | 0.06266 | 0.06281 | 1.52e-04 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | input_psnr | 21.96632 | 21.96070 | 5.62e-03 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | input_ssim | 0.92068 | 0.92066 | 2.88e-05 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | novel_lpips | 0.03690 | 0.03690 | 2.28e-06 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | novel_psnr | 22.77716 | 22.77316 | 4.00e-03 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | novel_ssim | 0.95298 | 0.95298 | 2.53e-06 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | geome_cd_l1 | 0.01277 | 0.01275 | 1.55e-05 |
| FaGr_s000002_yellowmustardbottle5 | single vs batched | geome_f01 | 0.72182 | 0.72286 | 1.04e-03 |
Verdict
Ours (SS+SLAT): batched path is NOT identical beyond float noise -> NOT batched; the full run uses the unbatched ours_combined_cell.sh (paper Table-1 path).
Evidence:
- Stage-1 SS coords: the unbatched path is bit-deterministic (single vs single2: 0 voxel differences on all 3 comparable pairs), but the batched ODE (B=2) flips occupancy voxels (1 voxel sym-diff in 2 of 4 object/view cells: yellowmustardbottle5 @1v 9766 vs 9765, butternutsquash1 @4v 7778 vs 7777). So batching changes the output where the single path does not.
- Stage-2 SLAT decode is itself run-to-run non-deterministic (vertex/face counts differ between two identical single runs), so exact mesh identity is unattainable even unbatched; mesh Chamfer-L1 between runs (0.0032-0.0034) equals the 50k-sample Chamfer sampling floor in both the batched and repeat pairs.
- Metric deltas, batched vs single, exceed the single-vs-single floor: input-view PSNR up to 0.096 dB (floor 0.0056), F@0.01 up to 1.8e-3 (floor 3.6e-4), CD-L1 up to 6.9e-5 (floor 1.4e-5), IV/NV LPIPS up to 6.9e-4 (floor 3.0e-4).
- These are tiny in absolute terms, but they are systematically above the unbatched noise floor, so under the rule "identical beyond float noise" the batched path fails -> run single.
Baselines (ReconViaGen, Pixal3D SV/MV, Amodal3R, Hunyuan3D-2mv, Cupid): not batched (their drivers run one object per forward pass; no batched path used). Speed-up = several independent single-object worker processes per GPU (sharded object lists, skip-existing, atomic writes), which does not alter any computation -> batch-equivalence test not applicable.
Side note: one single2 4v decode OOMed (GPU 6 was oversubscribed by co-located baseline workers at the time; before the <=2 procs/GPU cap). The full run checks for missing ours meshes and re-runs them.