license: other
license_name: nvidia-open-model-license
license_link: >-
https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
tags:
- robotics
- franka
- manipulation
- video-generation
- inverse-dynamics
- lora
NT_checkpoint
Two models for a Franka cube-stacking task. A video model generates multi-view footage of the arm stacking three cubes; an inverse dynamics model reads video back and recovers the actions that would produce it.
| folder | model | size | base |
|---|---|---|---|
vgm_lora/ |
rank-32 LoRA, video generation | 288 MB | nvidia/Cosmos-Predict2-14B-Sample-GR00T-Dreams-DROID |
idm/ |
inverse dynamics, full fine-tune | 2.4 GB | seonghyeonye/IDM_franka |
Each folder has its own README with the details. Start there.
Quick orientation
vgm_lora/ — adapter weights only. Attach to the base Cosmos-Predict2 checkpoint with
rank 32, alpha 32, targets q_proj,k_proj,v_proj,output_proj,mlp.layer1,mlp.layer2.
Conditioned on one 768×432 four-view frame plus a prompt, it produces a 93-frame clip at
16 fps.
idm/ — load with IDM.from_pretrained(). One forward pass sees two frames, s and
s+16, from three cameras and emits 16 actions. Its gripper output needs
post-processing — see idm/README.md. Reading
action[12] raw gives a vote tally, not a gripper state.
Scope
One task, one scene: three 5 cm cubes on a white table, a fixed three-camera rig, a Franka arm. Both models are specific to that setup.
Licence
Both are derivatives of NVIDIA-licensed weights and the upstream terms govern.
Attribution to NVIDIA Cosmos-Predict2 and to the GR00T-Dreams IDM is required. The
cosmos-predict2 repository code is Apache-2.0, which is separate from the weights licence.