Instructions to use HaomingSong/Cosmos3-Edge with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use HaomingSong/Cosmos3-Edge with LeRobot:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
|
Download README.md from HaomingSong/Cosmos3-Edge: direct link, hf CLI and curl.
- Browser
- Download file 4.77 kB
-
https://huggingface.co/HaomingSong/Cosmos3-Edge/resolve/main/README.md
- Command line
-
hf download hf://HaomingSong/Cosmos3-Edge/README.md
-
curl -L -o README.md https://huggingface.co/HaomingSong/Cosmos3-Edge/resolve/main/README.md
4.77 kB
| library_name: lerobot | |
| pipeline_tag: robotics | |
| license: other | |
| license_name: openmdw1.1-license | |
| license_link: https://openmdw.ai/license/1-1/ | |
| base_model: nvidia/Cosmos3-Edge | |
| tags: | |
| - lerobot | |
| - cosmos3 | |
| - robotics | |
| - pretrained | |
| # Cosmos3-Edge base for LeRobot | |
| This repository contains the LeRobot policy conversion of the NVIDIA Cosmos3-Edge base checkpoint. It is a starting point for robot policy post-training. The conversion adds no task-specific fine-tuning. | |
| The policy jointly predicts robot action chunks and future video latents from language and current camera observations, with optional current-state conditioning. It contains the model components needed by the LeRobot policy interface. | |
| | Item | Value | | |
| |---|---| | |
| | Original model developer | NVIDIA | | |
| | Original checkpoint | [nvidia/Cosmos3-Edge](https://huggingface.co/nvidia/Cosmos3-Edge/tree/344d602b128d1bbdacb43b08d0a3626f46343e29) | | |
| | Exact source revision | `344d602b128d1bbdacb43b08d0a3626f46343e29` | | |
| | LeRobot compatibility baseline | `b8f060837ced0a4ce4b136ea65b4d56bc7894bed` | | |
| | Policy type | `cosmos3_edge` | | |
| | Backbone | Nemotron 3 Dense VL MoT | | |
| | Decoder layers / hidden width | 28 / 2048 | | |
| | Stored scalar weights, including frozen components and VAE | 4,563,688,396 | | |
| | Stored tensors | 1188, comprising 1184 BF16 and 4 FP32 | | |
| The stored scalar count is not a count of trainable or active parameters. The package includes a Wan2.2 VAE, tokenizer files, and serialized LeRobot processors. | |
| ## Loading and post-training | |
| Use a LeRobot version containing the Cosmos3 policy types. Conversion and CPU loading checks used the exact commit above. | |
| The saved observation and action features are empty. Frame rate and camera size are unresolved. A target dataset must provide these values before constructing the policy. Read the configuration with: | |
| ```python | |
| from lerobot.configs.policies import PreTrainedConfig | |
| config = PreTrainedConfig.from_pretrained("HaomingSong/Cosmos3-Edge") | |
| ``` | |
| The training entry points select different configurations: | |
| - `--policy.path=HaomingSong/Cosmos3-Edge` preserves the saved DROID post-training recipe, including domain 8, 32-step chunks, state conditioning, identity normalization, UniPC with 4 inference steps, and guidance 3. State and action must already use the same layout and width. | |
| - `--policy.type=cosmos3_edge --policy.pretrained_path=HaomingSong/Cosmos3-Edge` uses these weights with the size class's default LIBERO recipe, including domain 5, 16-step chunks, no state conditioning, action quantile normalization, UniPC with 30 steps, and guidance 1. The dataset must match that recipe's camera and action layout. | |
| `load_action_modules` defaults to `true`, preserving the released action modules. For native-style post-training, `--policy.load_action_modules=false` reinitializes the action input projection, action output projection, and action modality embedding. The setting returns to `true` after loading so subsequent reloads preserve trained action modules. | |
| The saved `dtype=float32` supports FP32 master weights during training. Most stored weights retain upstream BF16 precision. The four time-embedder tensors are FP32. For inference set `--policy.dtype=bfloat16`. For evaluation also set `--tf32=false`. | |
| ## Scope and evaluation | |
| The LeRobot implementation supports joint video-and-action `wam` mode, a single observation step, and the UniPC sampler. Actual action width comes from the target dataset. This package does not expose the upstream reasoning, audio, or standalone video-generation APIs. | |
| CPU checks verified conversion and strict loading with resolved dataset features. No LIBERO closed-loop success rate is reported for this base conversion. Results for separately fine-tuned LIBERO checkpoints or NVIDIA Policy-DROID models do not measure this checkpoint. | |
| ## License and provenance | |
| The NVIDIA model materials use [OpenMDW 1.1](LICENSE). Retain the license and applicable copyright and origin notices when redistributing them. [NOTICE](NOTICE) records the original model and conversion attribution. Separately installed runtime software retains its own licenses. | |
| Consult the original [NVIDIA model card](https://huggingface.co/nvidia/Cosmos3-Edge/blob/344d602b128d1bbdacb43b08d0a3626f46343e29/README.md) and its [safety](https://huggingface.co/nvidia/Cosmos3-Edge/blob/344d602b128d1bbdacb43b08d0a3626f46343e29/SAFETY.md), [bias](https://huggingface.co/nvidia/Cosmos3-Edge/blob/344d602b128d1bbdacb43b08d0a3626f46343e29/BIAS.md), [privacy](https://huggingface.co/nvidia/Cosmos3-Edge/blob/344d602b128d1bbdacb43b08d0a3626f46343e29/PRIVACY.md), and [explainability](https://huggingface.co/nvidia/Cosmos3-Edge/blob/344d602b128d1bbdacb43b08d0a3626f46343e29/EXPLAINABILITY.md) notes for upstream limitations. | |