RigidFormer

RigidFormer (paper, code) learns multi-object rigid-body dynamics with an object-centric Transformer. This repository holds one pretrained model per dataset of RigidFormer-MOVi, 148.4 M parameters each:

Weights	Trained on	Configuration
rigidformer_movisphere.pth	MOVi-Sphere	configs/movi_sphere.yaml
rigidformer_movia.pth	MOVi-A	configs/movi_a.yaml
rigidformer_movib.pth	MOVi-B	configs/movi_b.yaml

configs/ holds the configurations the models were trained and evaluated with; they are the files of the same name in the code's configs/ folder.

Results

Test split, step size 1, rolled out from four ground-truth frames; error at physical frames 25 to 400:

Model		@25	@50	@75	@100	@200	@300	@400
MOVi-Sphere	position RMSE (m)	0.0052	0.0215	0.0451	0.0779	0.3183	0.7439	1.1911
	rotation error (deg)	0.653	2.933	6.277	10.745	47.304	93.222	106.549
MOVi-A	position RMSE (m)	0.0109	0.0402	0.0860	0.1451	0.5181	1.0482	1.5685
	rotation error (deg)	1.369	4.809	10.637	17.568	56.553	91.524	106.223
MOVi-B	position RMSE (m)	0.0068	0.0298	0.0613	0.1080	0.4374	0.9402	1.4138
	rotation error (deg)	0.680	3.288	7.163	12.646	49.602	91.984	107.141

Protocol: position estimator hopnet; scene set complete, 120 test scenes; rotation error = RMS over objects, rotation MAE = mean absolute angle over objects. The full records, with step sizes 5 and 10 and the rotation MAE, are in eval/.

Usage

From the RigidFormer code folder, with the data prepared as in the dataset card:

hf download frankzydou/RigidFormer rigidformer_movisphere.pth rigidformer_movia.pth rigidformer_movib.pth --local-dir checkpoints
CHECKPOINT=checkpoints/rigidformer_movisphere.pth BASE_CONFIG=configs/movi_sphere.yaml TARGETS=S bash scripts/eval.sh
CHECKPOINT=checkpoints/rigidformer_movia.pth      BASE_CONFIG=configs/movi_a.yaml      TARGETS=A bash scripts/eval.sh
CHECKPOINT=checkpoints/rigidformer_movib.pth      BASE_CONFIG=configs/movi_b.yaml      TARGETS=B bash scripts/eval.sh

SHA-256 of the weights files:

• rigidformer_movisphere.pth: ffb0efc6904b5a63504a5005bb6c4df2e79e8de20641dc205885593747fcf7d9
• rigidformer_movia.pth: db2e560f132933e2169ec9f90330e6b0595b3c1656684ca7f31adf95b0bc3d19
• rigidformer_movib.pth: 7db1dacf8850939a6398028fea71d9d58e75eebd7fc5079e75bd8475e6acf1ec

Citation

@inproceedings{dou2026rigidformer,
  title     = {RigidFormer: Learning Rigid Dynamics using Transformers},
  author    = {Dou, Zhiyang and Guo, Minghao and Wu, Haixu and Roble, Doug and Stuyck, Tuur and Matusik, Wojciech},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026}
}