Initial model release

Co-authored-by: zhengfei-nvidia <zhengfei-nvidia@users.noreply.huggingface.co>
Co-authored-by: yaoxuhf <yaoxuhf@users.noreply.huggingface.co>
Co-authored-by: qmiao <qmiao@users.noreply.huggingface.co>
Co-authored-by: yzhautouskay <yzhautouskay@users.noreply.huggingface.co>
Co-authored-by: mbalaNV <mbalaNV@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,65 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+text_tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/agibot/images/agibot_001_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/agibot/images/agibot_006_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/agibot/images/agibot_034_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/agibot/images/agibot_idx0000009_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/agibot/images/agibot_mixed_005_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/agibot/images/agibot_mixed_007_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/agibot/images/agibot_mixed_028_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/yam/images/yam_005_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/yam/images/yam_019_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/yam/images/yam_023_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/yam/images/yam_idx0000014_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/gt/agibot_001_gt.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/gt/agibot_006_gt.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/gt/agibot_034_gt.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/gt/agibot_mixed_005_gt.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/gt/agibot_mixed_007_gt.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/gt/agibot_mixed_028_gt.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/images/agibot_001_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/images/agibot_006_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/images/agibot_034_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/images/agibot_mixed_005_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/images/agibot_mixed_007_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/training/agibot/images/agibot_mixed_028_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/agibot/images/agibot_original_long60_012_initial.png filter=lfs diff=lfs merge=lfs -text
+assets/output/cosmos3-nano-sim-bimanual-quickstart.mp4 filter=lfs diff=lfs merge=lfs -text
+assets/sim_bimanual/agibot/images/agibot_original_long60_019_initial.png filter=lfs diff=lfs merge=lfs -text
+images/cosmos3-nano-sim-bimanual-graph.png filter=lfs diff=lfs merge=lfs -text
+images/cosmos3-nano-sim-bimanual-overview.gif filter=lfs diff=lfs merge=lfs -text
+images/cosmos3-nano-sim-bimanual-overview.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,571 @@
+---
+license: other
+license_name: openmdw1.1-license
+license_link: >-
+  https://openmdw.ai/license/1-1/
+library_name: cosmos
+tags:
+  - nvidia
+  - cosmos
+  - cosmos3
+  - omnimodel
+---
+
+# **Interactive World Simulation with Cosmos for Physical AI**
+
+**[Cosmos3 Examples Collection](https://huggingface.co/collections/nvidia/cosmos3-examples)** | **[Code](https://github.com/nvidia/cosmos)** | **[Technical Report](https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation/technical-report.pdf)** (coming soon) | **[Website](https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation/)**
+
+# Model Overview: Cosmos3-Nano-Sim-Bimanual
+
+![Cosmos3 product and examples lineage highlighting Cosmos3-Nano-Sim-Bimanual](images/cosmos3-nano-sim-bimanual-graph.png)
+
+The highlighted checkpoint, Cosmos3-Nano-Sim-Bimanual, is a post-trained, causal-distilled variant of the Cosmos3-Nano base model.
+
+## Description
+
+<p align="center">
+  <video src="https://huggingface.co/nvidia/Cosmos3-Nano-Sim-Bimanual/resolve/main/images/cosmos3-nano-sim-bimanual-overview.mp4" controls autoplay loop muted playsinline width="640" aria-label="Generated results only: AgiBot 008 and 006, seed 2026, on the top row; YAM 001, seed 2025, and YAM 008, seed 2027, on the bottom row. First 15 seconds at original speed."></video>
+  <br>
+</p>
+
+Cosmos3-Nano-Sim-Bimanual simulates how an environment evolves in response to a bimanual robot’s actions. It supports interleaved camera navigation and robot manipulation within a single action sequence in both open-loop and closed-loop settings, enabling continuous simulation across changes in camera pose and robot arm or gripper state. Given an initial image and a sequence of control inputs, the model generates video in small latent-frame chunks conditioned on cached history and incoming robot actions or camera trajectories. Control inputs can be supplied at any point during a rollout by a policy model, such as a vision-language-action (VLA) model or a world-action model, or by a human-operated joystick. These capabilities support robotic policy evaluation and verification, reinforcement learning (RL) training, and synthetic demonstration data generation without teleoperation when control inputs are supplied by a policy model.
+
+Cosmos3-Nano-Sim-Bimanual was developed by NVIDIA as part of the Cosmos example model family.
+
+For a quick start, see the [Cosmos3-Nano-Sim-Bimanual skill](https://github.com/NVIDIA/cosmos-framework/tree/main/.agents/skills/cosmos3-nano-sim-bimanual).
+
+This model is ready for commercial or non-commercial use.
+
+## License
+
+This model is released under [OpenMDW1.1](https://openmdw.ai/license/1-1/).
+
+## Deployment Geography
+
+Global
+
+## Use Case
+
+Physical AI for robotics: policy evaluation and verification, reinforcement learning (RL) training, and synthetic demonstration data generation with interleaved camera navigation and bimanual manipulation actions. Policy-generated control inputs enable demonstration generation without teleoperation.
+
+## Release Date
+
+- Hugging Face 10/20/2026 via [https://huggingface.co/collections/nvidia/cosmos3-examples](https://huggingface.co/collections/nvidia/cosmos3-examples)
+- GitHub 10/20/2026 via [https://github.com/nvidia/cosmos](https://github.com/nvidia/cosmos)
+
+## Model Architecture
+
+**Architecture Type:** Transformer
+
+**Network Architecture:** Mixture-of-Transformers (MoT)
+
+Cosmos3-Nano-Sim-Bimanual is derived from [Cosmos3-Nano](https://huggingface.co/nvidia/Cosmos3-Nano) through post-training on a mixture of robot manipulation videos paired with robot actions and camera navigation videos paired with camera trajectories, followed by causal distillation. It generates video in small latent-frame chunks conditioned on cached history and incoming control inputs. The model supports interleaved robot manipulation and camera navigation within a single action sequence, maintaining scene continuity across robot motion and viewpoint changes. Additional post-training on user-provided data can adapt the model to different robot embodiments and arm configurations; see the [Cosmos3-Nano-Sim-Bimanual skill](https://github.com/NVIDIA/cosmos-framework/tree/main/.agents/skills/cosmos3-nano-sim-bimanual) for details.
+
+This model was developed using the [Cosmos Framework](https://github.com/NVIDIA/cosmos-framework).
+
+**Number of trainable model parameters:** 16B
+
+## Inputs
+
+- **Input Type(s):** Image, robot actions, camera trajectories, text (optional scene description)
+- **Input Format(s):**
+  - **Image:** JPG, PNG, JPEG, or WebP (initial frame)
+  - **Robot actions:** JSON containing a sequence of robot state or control values with shape `(T, D)`, where `T` is the number of action steps and `D` is the embodiment-specific action dimensionality
+  - **Camera trajectories:** Sequence of camera poses (for example, a 9D camera-motion representation)
+  - **Text:** String
+- **Input Parameters:**
+  - **Image:** Two-dimensional (2D)
+  - **Robot actions / Camera trajectories:** Two-dimensional sequences (time steps × control dimensions)
+  - **Text:** One-dimensional (1D)
+- **Other Properties Related to Input:**
+  - Actions are aligned with the generated video.
+  - Supported embodiment: AgiBot
+
+## Outputs
+
+- **Output Type(s):** Video
+- **Output Format(s):**
+  - **Video:** MP4 or streamed video frames
+- **Output Parameters:**
+  - **Video:** Three-dimensional (3D)
+- **Other Properties Related to Output:**
+  - Generates video at 480p with an output frame rate of 30 frames per second (fps).
+  - Output video frame rates describe playback timing; inference throughput is reported separately.
+
+Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g., GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
+
+## Software Integration
+
+**Runtime Engine(s):**
+
+- [PyTorch](https://pytorch.org/)
+- [vLLM-Omni](https://github.com/vllm-project/vllm-omni)
+
+**Supported Hardware Microarchitecture Compatibility:**
+
+- NVIDIA Blackwell
+- NVIDIA Hopper
+
+**Operating System(s):**
+
+- Linux (other operating systems have not been tested).
+
+The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
+
+## Model Version(s)
+
+**Cosmos3-Nano-Sim-Bimanual — V1.0 (10/20/2026)**
+
+Cosmos3-Nano-Sim-Bimanual is a post-trained, causal-distilled variant of [Cosmos3-Nano](https://huggingface.co/nvidia/Cosmos3-Nano).
+
+It can be integrated into a closed-loop robotic policy evaluation or training pipeline that supplies an initial image and a sequence of robot actions and camera trajectories from a policy model or human controller. The model supports this interaction by returning generated video frames as visual observations for subsequent policy or controller inputs.
+
+## Training and Evaluation Datasets
+
+### Post-Training Datasets
+
+**Data Modality:** Video, Text, and paired robot-action or camera-trajectory data.
+
+**Post-Training Data Size:**
+- **Text:** 1 Billion to 10 Trillion Tokens
+- **Video with paired robot-action or camera-trajectory data:** 10,000 to 1 Million Hours
+
+**Data Collection Method:** Hybrid: Automatic/Sensors, Synthetic, Automated.
+
+**Labeling Method:** Hybrid: Manually-Labeled, Automated.
+
+**Properties:** The post-training datasets consist of robot manipulation and camera navigation videos, associated English captions, robot actions, and camera trajectories. The data includes real and synthetic content from vendor-provided, public, and NVIDIA-created datasets.
+
+### Dataset Overview
+
+- **Total Size:** 1.3B data points
+- **Total Number of Datasets:** 393 dataset entries
+- **Dataset partition:** Training [100%], Testing [N/A — evaluation benchmarks used separately], Validation [N/A — evaluation benchmarks used separately]
+- **Time period for training data collection:** 2024–2026
+- **Time period for validation data collection:** N/A (standard public benchmarks)
+
+Raw data from internal and external sources is transformed into training-ready data through multiple stages of curation, filtering, and quality review. Data acquisition spans diverse multimodal sources — robotics, autonomous driving, industrial environments, indoor and outdoor scenes, varied lighting and weather conditions, camera viewpoints, object categories, and human activities — to broaden coverage across Physical AI operating environments. Automated filtering pipelines remove corrupted, duplicate, low-quality, and restricted content. Metadata analysis, heuristic rules, and model-assisted classifiers are applied during preprocessing to flag anomalous distributions and low-diversity subsets. Human review supplements automated filtering for selected datasets, benchmark construction, and targeted quality analysis. Datasets are balanced across modalities and task categories — visual reasoning, text-to-image, text-to-video, image-to-video, audio generation, video transfer, action-conditioned generation, and action command generation — to reduce overrepresentation of narrow domains. Synthetic and simulation-based augmentation supplements coverage of rare physical interactions and edge-case scenarios. Deduplication and provenance tracking are applied across the corpus. The resulting processed data is converted into model-ready tokenized or encoded representations through modality-specific preprocessors before training begins.
+
+Training datasets passed through multiple layers of automated and manual safeguards designed to reduce the presence of harmful or policy-violating content across categories including weapons and weapons-related instructional content, criminal planning, child sexual abuse material (CSAM), non-consensual intimate imagery (NCII), sexual content involving minors, harassment, hate speech, profanity, threats and incitement to violence, self-harm or suicide-related content, and graphic violence. Data sources are reviewed for licensing compatibility, provenance, and alignment with internal data governance and safety policies before admission into training corpora. Automated filtering pipelines combine multiple detection strategies: hash-matching against known CSAM and NCII reference databases; classifier-based moderation models trained for explicit sexual content, hate speech, violence, weapons imagery, and other restricted categories; keyword and regex-based screening for criminal-planning, threats, and self-harm phrases in text data; metadata and provenance heuristics for source-level risk signals; and embedding-based anomaly detection to surface samples that fall outside expected distributions. Human review and targeted audits supplement automated filtering for selected datasets, benchmark construction, and safety-sensitive evaluation. For multimodal Physical AI data (robotics, autonomous driving, industrial scenes), additional filtering targets invalid action trajectories, physically implausible interactions, and unsafe control sequences. Synthetic and simulation-generated data are evaluated through internal validation before inclusion. Benchmark evaluations and red-team testing are applied post-training to surface remaining safety gaps across world generation, reasoning, audio, and action tasks. No large-scale data-filtering process can guarantee complete removal of all harmful content; residual risks may remain, particularly in rare edge cases or open-world deployment settings. Ongoing monitoring and dataset review continue post-release.
+
+For more information about the datasets used to train this model, please see the [Public Summary of Training Content](https://docs.nvidia.com/cosmos/latest/cosmos3/index.html).
+
+**Data Modality and Training Data Size**
+
+| Modality | Reasoning Data Sample Count | Generation Data Sample Count |
+| -------- | ------------------- | -------------------- |
+| Text     | 22M                 | Not Applicable       |
+| Image    | 19M                 | 767M                 |
+| Video    | 1M                  | 348M                 |
+| Audio    | Not Applicable      | 139M                 |
+| Action   | Not Applicable      | 7M                   |
+
+**Data Collection Method by dataset**
+
+- Hybrid: Automatic/Sensors, Synthetic, Automated
+
+**Labeling Method by dataset**
+
+- Hybrid: Manually-Labeled, Automated
+
+**Properties:** The training, testing, and evaluation datasets consist of diverse multimodal video, image, audio, action, synthetic, and sensor-conditioned data sourced from NVIDIA-owned data and publicly available, commercially permissive datasets. These datasets are curated to exclude known restricted content and to support building an Omni model that learns to generate and reason about dynamic physical environments across world reasoning and generation tasks.
+
+### Public Datasets
+
+| Dataset                   | Samples     |
+|---------------------------|-------------|
+| OpenImage                 | 1.2M        |
+| Coyo700M                  | 100M        |
+| YouTube Video             | 340M        |
+| UMI                       | 4.5M        |
+
+### Private Datasets
+
+| Dataset                   | Samples     |
+|---------------------------|-------------|
+| Egocentric                | 7M          |
+| Nexar                     | 0.6M        |
+| AgiBot                    | 0.2M        |
+| HOI                       | 0.3M        |
+
+### Synthetic Datasets
+
+| Dataset                                 | Samples     |
+|-----------------------------------------|-------------|
+| synthetic images generated using HiDream-I1       | 15M         |
+| synthetic images generated using Qwen-Image-2512  | 14M         |
+| synthetic captions generated using Qwen3-VL       | 1115M       |
+
+## Evaluation Datasets
+
+**Data Collection Method by dataset**
+
+- Hybrid: Automatic/Sensors, Synthetic, Automated
+
+**Labeling Method by dataset**
+
+- Hybrid: Manually-Labeled, Automated
+
+**Properties:** The training, testing, and evaluation datasets consist of diverse multimodal video, image, audio, action, synthetic, and sensor-conditioned data sourced from NVIDIA-owned data and publicly available, commercially permissive datasets. These datasets are curated to exclude known restricted content and to support building an Omni model that learns to generate and reason about dynamic physical environments across world reasoning and generation tasks.
+
+## Quality Benchmark(s)
+
+### Camera Navigation Benchmark
+
+#### [PAI-Bench-G](https://arxiv.org/abs/2512.01989) Image-to-Video (I2V)
+
+We evaluate **Cosmos3-Nano-Sim-Bimanual** against LingBot-World 2.0 and Sana-WM on sustained viewpoint control. Each example supplies an initial image, a scene caption, and a camera trajectory; captions exclude camera motion. The [PAI-Bench-G](https://arxiv.org/abs/2512.01989) quality score measures video fidelity and consistency. [ViPE](https://research.nvidia.com/labs/toronto-ai/vipe/) estimates camera positions and orientations from the generated video. We fit the estimated trajectory to the input trajectory by rotation, translation, and uniform scaling (Umeyama alignment), allowing comparison independent of coordinate frame and scale. Absolute translation error (ATE) measures camera-position error after alignment; relative translation error (RTE) and relative rotation error (RRE) measure errors in camera displacement and orientation change between poses. Perceptual drift evaluates changes in video quality over time.
+
+<table>
+  <thead>
+    <tr>
+      <th scope="col" style="text-align: left;">Model</th>
+      <th scope="col" style="text-align: right;">Video Quality ↑</th>
+      <th scope="col" style="text-align: right;">ATE ↓</th>
+      <th scope="col" style="text-align: right;">RTE ↓</th>
+      <th scope="col" style="text-align: right;">RRE ↓</th>
+      <th scope="col" style="text-align: right;">Perceptual Drift ↓</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left;">LingBot-World 2.0</td>
+      <td style="text-align: right;">65.3577</td>
+      <td style="text-align: right;"><strong>4.1698</strong></td>
+      <td style="text-align: right;">0.0596</td>
+      <td style="text-align: right;">0.1781</td>
+      <td style="text-align: right;"><strong style="color: inherit;">0.0209</strong></td>
+    </tr>
+    <tr>
+      <td style="text-align: left;">Sana-WM</td>
+      <td style="text-align: right;">65.2090</td>
+      <td style="text-align: right;">6.4823</td>
+      <td style="text-align: right;">0.1284</td>
+      <td style="text-align: right;">0.1601</td>
+      <td style="text-align: right;">0.0299</td>
+    </tr>
+    <tr>
+      <td style="text-align: left; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">Cosmos3-Nano-Sim-Bimanual</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">73.9099</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;">4.7072</td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.0545</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.1002</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;">0.0210</td>
+    </tr>
+  </tbody>
+</table>
+
+#### Sana-WM simple
+
+The simple split of [Sana-WM-Bench](https://arxiv.org/abs/2605.15178) reports visual quality, camera alignment, and drift. We pair benchmark runs by source inputs and trajectories, using the same metrics as the [PAI-Bench-G](https://arxiv.org/abs/2512.01989) I2V evaluation.
+
+<table>
+  <thead>
+    <tr>
+      <th scope="col" style="text-align: left;">Model</th>
+      <th scope="col" style="text-align: right;">Video Quality ↑</th>
+      <th scope="col" style="text-align: right;">ATE ↓</th>
+      <th scope="col" style="text-align: right;">RTE ↓</th>
+      <th scope="col" style="text-align: right;">RRE ↓</th>
+      <th scope="col" style="text-align: right;">Perceptual Drift ↓</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left;">LingBot-World 2.0</td>
+      <td style="text-align: right;">65.9748</td>
+      <td style="text-align: right;">1.4254</td>
+      <td style="text-align: right;">0.0185</td>
+      <td style="text-align: right;">0.2591</td>
+      <td style="text-align: right;">0.0371</td>
+    </tr>
+    <tr>
+      <td style="text-align: left;">Sana-WM</td>
+      <td style="text-align: right;">66.0599</td>
+      <td style="text-align: right;">1.2159</td>
+      <td style="text-align: right;">0.0192</td>
+      <td style="text-align: right;">0.2356</td>
+      <td style="text-align: right;">0.0223</td>
+    </tr>
+    <tr>
+      <td style="text-align: left; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">Cosmos3-Nano-Sim-Bimanual</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">77.2887</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">1.0040</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.0124</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.0551</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.0118</strong></td>
+    </tr>
+  </tbody>
+</table>
+
+#### Sana-WM hard
+
+The hard split of [Sana-WM-Bench](https://arxiv.org/abs/2605.15178) reports visual quality, camera alignment, and drift. We pair benchmark runs by source inputs and trajectories, using the same metrics as the simple split.
+
+<table>
+  <thead>
+    <tr>
+      <th scope="col" style="text-align: left;">Model</th>
+      <th scope="col" style="text-align: right;">Video Quality ↑</th>
+      <th scope="col" style="text-align: right;">ATE ↓</th>
+      <th scope="col" style="text-align: right;">RTE ↓</th>
+      <th scope="col" style="text-align: right;">RRE ↓</th>
+      <th scope="col" style="text-align: right;">Perceptual Drift ↓</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left;">LingBot-World 2.0</td>
+      <td style="text-align: right;">64.5462</td>
+      <td style="text-align: right;">1.3423</td>
+      <td style="text-align: right;"><strong>0.0161</strong></td>
+      <td style="text-align: right;">0.3264</td>
+      <td style="text-align: right;">0.0372</td>
+    </tr>
+    <tr>
+      <td style="text-align: left;">Sana-WM</td>
+      <td style="text-align: right;">65.5699</td>
+      <td style="text-align: right;">1.3165</td>
+      <td style="text-align: right;">0.0178</td>
+      <td style="text-align: right;">0.1826</td>
+      <td style="text-align: right;">0.0251</td>
+    </tr>
+    <tr>
+      <td style="text-align: left; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">Cosmos3-Nano-Sim-Bimanual</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">76.3971</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">1.3085</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.0161</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.1774</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.0161</strong></td>
+    </tr>
+  </tbody>
+</table>
+
+### Robot Manipulation Benchmark
+
+#### AgiBot-World
+
+We evaluate **Cosmos3-Nano-Sim-Bimanual** on observation prediction conditioned on AgiBot-World action sequences, an initial image, and a caption, comparing against the bidirectional DreamDojo 2B and 14B teacher models. The [PAI-Bench-G](https://arxiv.org/abs/2512.01989) quality score measures video fidelity and consistency. PSNR measures pixel-level agreement, SSIM measures image-structure agreement, and [LPIPS](https://arxiv.org/abs/1801.03924) measures differences in learned visual features. These metrics compare generated and reference frames and serve as proxies for action following. Perceptual drift evaluates changes in video quality over time.
+
+<table>
+  <thead>
+    <tr>
+      <th scope="col" style="text-align: left;">Model</th>
+      <th scope="col" style="text-align: right;">Video Quality ↑</th>
+      <th scope="col" style="text-align: right;">PSNR (dB) ↑</th>
+      <th scope="col" style="text-align: right;">SSIM ↑</th>
+      <th scope="col" style="text-align: right;">LPIPS ↓</th>
+      <th scope="col" style="text-align: right;">Perceptual Drift ↓</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left;">DreamDojo 2B</td>
+      <td style="text-align: right;">74.5327</td>
+      <td style="text-align: right;">12.1016</td>
+      <td style="text-align: right;">0.6405</td>
+      <td style="text-align: right;">0.3766</td>
+      <td style="text-align: right;">0.0149</td>
+    </tr>
+    <tr>
+      <td style="text-align: left;">DreamDojo 14B</td>
+      <td style="text-align: right;">74.9390</td>
+      <td style="text-align: right;">12.3561</td>
+      <td style="text-align: right;">0.6635</td>
+      <td style="text-align: right;">0.3586</td>
+      <td style="text-align: right;">0.0133</td>
+    </tr>
+    <tr>
+      <td style="text-align: left; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">Cosmos3-Nano-Sim-Bimanual</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">76.0884</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">14.4138</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.6839</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.3454</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.0118</strong></td>
+    </tr>
+  </tbody>
+</table>
+
+#### AgiBot-GEAR
+
+We evaluate **Cosmos3-Nano-Sim-Bimanual** on observation prediction conditioned on AgiBot-GEAR action sequences, an initial image, and a caption, comparing against the bidirectional DreamDojo 2B and 14B teacher models. We use the same video quality, reference-frame agreement, and drift metrics as the AgiBot-World evaluation.
+
+<table>
+  <thead>
+    <tr>
+      <th scope="col" style="text-align: left;">Model</th>
+      <th scope="col" style="text-align: right;">Video Quality ↑</th>
+      <th scope="col" style="text-align: right;">PSNR (dB) ↑</th>
+      <th scope="col" style="text-align: right;">SSIM ↑</th>
+      <th scope="col" style="text-align: right;">LPIPS ↓</th>
+      <th scope="col" style="text-align: right;">Perceptual Drift ↓</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td style="text-align: left;">DreamDojo 2B</td>
+      <td style="text-align: right;">72.4586</td>
+      <td style="text-align: right;">11.5709</td>
+      <td style="text-align: right;">0.5437</td>
+      <td style="text-align: right;">0.4638</td>
+      <td style="text-align: right;">0.0232</td>
+    </tr>
+    <tr>
+      <td style="text-align: left;">DreamDojo 14B</td>
+      <td style="text-align: right;">73.0033</td>
+      <td style="text-align: right;">12.1113</td>
+      <td style="text-align: right;">0.5818</td>
+      <td style="text-align: right;">0.4326</td>
+      <td style="text-align: right;">0.0218</td>
+    </tr>
+    <tr>
+      <td style="text-align: left; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">Cosmos3-Nano-Sim-Bimanual</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">74.4912</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">14.0158</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.6172</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.4057</strong></td>
+      <td style="text-align: right; background-color: #e8f4df; color: #18202b;"><strong style="color: inherit;">0.0188</strong></td>
+    </tr>
+  </tbody>
+</table>
+
+## Inference
+
+**Acceleration Engine:** [PyTorch](https://pytorch.org/), [vLLM-Omni](https://github.com/vllm-project/vllm-omni)
+
+**Test Hardware:** NVIDIA Blackwell GB200, NVIDIA Hopper H100, H200, H20, RTX PRO 6000
+
+### Inference Performance
+
+Numbers are based on vLLM-Omni, using a single GPU. Empty table cells indicate that measurements are not reported.
+
+|  | 480p inference throughput (frames/s) |
+| --- | --- |
+| GB200 | 54.1 |
+| H100 | 32.6 |
+| H200 | 32.4 |
+| H20 | 10.7 |
+| RTX Pro 6000 | 20.6 |
+
+## Usage Example
+
+### Quick Start
+
+This example serves the model with
+[vLLM-Omni](https://github.com/vllm-project/vllm-omni) and generates the mixed-action AgiBot example: 901 frames at 30 FPS from one initial
+image and 900 normalized 59D actions combining robot actions and camera motion.
+The checkpoint uses a fixed four-step sampler.
+
+#### 1. Start the server
+
+On a Linux host with an NVIDIA GPU (tested on one GB200), [Docker](https://docs.docker.com/engine/install/), and the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html):
+
+```bash
+docker run --runtime nvidia --gpus all --ipc=host \
+  -v ~/.cache/huggingface:/root/.cache/huggingface \
+  --env "HF_TOKEN=$HF_TOKEN" \
+  --env VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
+  -p 127.0.0.1:8000:8000 \
+  vllm/vllm-omni:cosmos3 \
+  vllm serve nvidia/Cosmos3-Nano-Sim-Bimanual \
+    --omni --model-class-name Cosmos3NanoSimBimanualPipeline \
+    --deploy-config /app/vllm-omni/vllm_omni/deploy/cosmos3_nano_sim_bimanual_windowed.yaml \
+    --no-guardrails \
+    --host 0.0.0.0 --port 8000 --init-timeout 1800
+```
+
+The container publishes the server on `127.0.0.1:8000` of the host.
+`--deploy-config` selects the Bimanual deployment: one request at a time, eager
+execution, and a 64-latent-frame rolling key/value cache with eight attention
+sinks. `--no-guardrails` disables the Cosmos3 text and video safety checks. The
+first start downloads the checkpoint. For optimized server settings and
+additional examples, see the
+[Bimanual vLLM-Omni notebook](https://github.com/NVIDIA/cosmos-framework/blob/main/.agents/skills/cookbooks/cosmos3/simulation/bimanual/run_bimanual_with_vllm_omni.ipynb).
+
+#### 2. Generate the mixed-action video
+
+Install the client dependencies in your Python environment:
+
+```bash
+pip install requests huggingface_hub
+```
+
+Run the client on the same machine as the server. The example metadata supplies
+the prompt, frame count, frame rate, action domain, resolution, and seed 2026:
+
+```python
+import json
+from pathlib import Path
+from uuid import uuid4
+
+import requests
+from huggingface_hub import snapshot_download
+
+MODEL = "nvidia/Cosmos3-Nano-Sim-Bimanual"
+repo_dir = Path(snapshot_download(MODEL, allow_patterns=[
+    "examples/inference/sim_bimanual_mixed_action_hf.jsonl",
+    "assets/sim_bimanual/agibot/images/agibot_mixed_028_initial.png",
+    "assets/sim_bimanual/agibot/actions/agibot_mixed_028_normalized_mixed_action.json",
+    "cosmos3_nano_sim_bimanual_action_sources/action_schema.json",
+]))
+manifest_path = repo_dir / "examples/inference/sim_bimanual_mixed_action_hf.jsonl"
+image_path = repo_dir / "assets/sim_bimanual/agibot/images/agibot_mixed_028_initial.png"
+action_path = repo_dir / "assets/sim_bimanual/agibot/actions/agibot_mixed_028_normalized_mixed_action.json"
+schema_path = repo_dir / "cosmos3_nano_sim_bimanual_action_sources/action_schema.json"
+
+examples = [json.loads(line) for line in manifest_path.read_text().splitlines()]
+example = next(item for item in examples if item["action_path"].endswith("/" + action_path.name))
+action = json.loads(action_path.read_text())
+schema = json.loads(schema_path.read_text())
+domain_ids = {
+    name: values["domain_id"]
+    for name, values in schema["embodiments"].items()
+}
+
+extra_params = {
+    "session_id": uuid4().hex,
+    "reset": True,
+    "close_session": True,
+    "action_mode": "forward_dynamics",
+    "action_space": action["action_space"],
+    "action": action["action"],
+    "domain_name": example["domain_name"],
+    "domain_id": [domain_ids[name] for name in action["domain_names"]],
+    "resolution": int(example["resolution"]),
+}
+
+with image_path.open("rb") as image_file:
+    response = requests.post(
+        "http://127.0.0.1:8000/v1/videos/sync",
+        data={
+            "model": MODEL,
+            "prompt": example["prompt"],
+            "num_frames": str(example["num_frames"]),
+            "fps": str(example["fps"]),
+            "seed": str(example["seed"]),
+            "extra_params": json.dumps(extra_params),
+        },
+        files={"input_reference": (image_path.name, image_file, "image/png")},
+        headers={"Accept": "video/mp4"},
+        timeout=1860,
+    )
+response.raise_for_status()
+Path("cosmos3-nano-sim-bimanual-mixed-028.mp4").write_bytes(response.content)
+```
+
+Generated output:
+
+<video src="https://huggingface.co/nvidia/Cosmos3-Nano-Sim-Bimanual/resolve/main/assets/output/cosmos3-nano-sim-bimanual-quickstart.mp4" controls autoplay loop muted playsinline width="100%"></video>
+
+The initial image is the only visual observation sent to the server. The action
+values are already normalized for this checkpoint.
+
+## Ethical Considerations
+
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
+
+Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.
+
+Users are responsible for model inputs and outputs. Users are responsible for ensuring safe integration of this model, including implementing guardrails as well as other safety mechanisms, prior to deployment.
+
+For more detailed information on ethical considerations for this model, please see the Model Card++ [Explainability](explainability-Cosmos3-Nano-Sim-Bimanual.md), [Bias](bias.md), [Safety & Security](safety.md), and [Privacy](privacy.md) subcards. Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

assets/output/cosmos3-nano-sim-bimanual-quickstart.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c04c1c883d135e3a3d6d76ccff406aae1e36bff7b8ee637dc5ec274c23cb217f
+size 9417890

assets/training/agibot/gt/agibot_001_gt.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b520df89e6f881bf3a5a57116032d62cf11dbec0d0ae3a86d869f26b5a022aa
+size 2544736

assets/training/agibot/gt/agibot_006_gt.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f905b8e4963d7a705b16d3b0c5e5adfac40fa4ce37c7b34f960df4caf9be5089
+size 1847124

assets/training/agibot/gt/agibot_034_gt.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:047b191903bfbfe8f79c6dbeeb518e0497ee772cad8010284147dac8c6dc7bad
+size 2301040

bias.md

@@ -0,0 +1,11 @@
+## Bias
+
+| Field | Response |
+| :---- | :---- |
+| Participation considerations from adversely impacted groups [protected classes](https://www.senate.ca.gov/content/protected-classes) in model design and testing | None. |
+| Measures taken to mitigate against unwanted bias | Training, evaluation, and testing data are curated before release to filter restricted content, including content relating to protected classes. Downstream evaluation should assess coverage and model behavior across the environments and operating conditions relevant to the intended application. |
+| Which characteristic (feature) show(s) the greatest difference in performance?: | Performance may vary with scene complexity, rollout length, and the input conditions used for generation. Long-horizon temporal consistency and fine-grained physical interactions are known challenges described for the Cosmos3-Nano base model and are relevant considerations for these simulation models. |
+| Which feature(s) have the worst performance overall? | Challenging scenarios include long rollouts and complex physical interactions, where temporal consistency and fidelity to the supplied controls may degrade. Users should evaluate these aspects under their intended simulation conditions. |
+| If using internal data, description of methods implemented in data acquisition or processing, if any, to address the prevalence of identifiable biases in the training, testing, and validation data: | Bias-specific methods applied during data processing include person-presence screening, demographic-taxonomy classification (age, gender, ethnicity), embedding-based diversity analysis, and dataset balancing across sources. Internal analysis of the Cosmos3-Nano base model's training data surfaced: non-person scenes are more prevalent than person-centric content; demographic-taxonomy outputs on person-present samples are most frequently "uncertain" across age, gender, and ethnicity dimensions; and source-type variation, with people-centric image and video datasets showing higher demographic signal than document-, object-, robotics-, or scene-focused datasets. *(Quantitative details are provided below.)* Downstream deployments should add bias audits, fairness evaluation, red-teaming, demographically balanced fine-tuning, or counterfactual augmentation as mitigations. |
+| Tools used to assess statistical imbalances and highlight patterns that may introduce bias into AI models: | Dataset analytics pipelines, metadata distribution analysis, heuristic quality checks, embedding-based clustering, model-assisted filtering systems, and benchmark evaluation suites are used to assess statistical imbalances and identify patterns that may introduce bias into model behavior. |
+| Tools used to assess statistical imbalances and highlight patterns that may introduce bias into AI models: | The following analysis covers training data used for the Cosmos3-Nano base model. These datasets, such as OpenImages-derived detection-to-NLP datasets, visual grounding and VQA datasets, document/image understanding datasets, video/action understanding datasets, and NVIDIA-created or curated visual datasets, do not collectively or exhaustively represent all demographic groups (and proportionally therein). For instance, automated person-presence screening did not identify a person in approximately 58% of visual samples analyzed across approximately 400 datasets, while person-present signals were identified in approximately 42% of analyzed samples. In the subset where person-present signals were identified, these datasets contain uneven representation splits across the measured visual taxonomies: age outputs were most frequently uncertain, followed by child and adult; gender outputs were most frequently uncertain, followed by male and female; and ethnicity outputs were most frequently uncertain, followed by Hispanic and White as the most frequent identified categories. Dataset-level results vary by source type, with people-centric image and video datasets containing higher person-present and demographic-taxonomy signals than document-, object-, robotics-, or scene-focused datasets. To mitigate these imbalances, we recommend considering evaluation techniques such as bias audits, task-specific fairness evaluation, and red-teaming, along with fine-tuning with demographically balanced datasets and counterfactual data augmentation to align with the desired model behavior. This evaluation used a baseline of 200 samples across all datasets, with larger subsets of up to 3,000 samples utilized for certain in-depth analyses, identified as optimal thresholds for maximizing embedder accuracy. |

chat_template.json

@@ -0,0 +1,3 @@
+{
+    "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {%- if messages[0].content is string %}\n            {{- messages[0].content }}\n        {%- else %}\n            {%- for content in messages[0].content %}\n                {%- if 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' }}\n        {%- if messages[0].content is string %}\n            {{- messages[0].content }}\n        {%- else %}\n            {%- for content in messages[0].content %}\n                {%- if 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set image_count = namespace(value=0) %}\n{%- set video_count = namespace(value=0) %}\n{%- for message in messages %}\n    {%- if message.role == \"user\" %}\n        {{- '<|im_start|>' + message.role + '\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content in message.content %}\n                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}\n                    {%- set image_count.value = image_count.value + 1 %}\n                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}\n                    <|vision_start|><|image_pad|><|vision_end|>\n                {%- elif content.type == 'video' or 'video' in content %}\n                    {%- set video_count.value = video_count.value + 1 %}\n                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}\n                    <|vision_start|><|video_pad|><|vision_end|>\n                {%- elif 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role + '\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content_item in message.content %}\n                {%- if 'text' in content_item %}\n                    {{- content_item.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and message.content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {%- if message.content is string %}\n            {{- message.content }}\n        {%- else %}\n            {%- for content in message.content %}\n                {%- if content.type == 'image' or 'image' in content or 'image_url' in content %}\n                    {%- set image_count.value = image_count.value + 1 %}\n                    {%- if add_vision_id %}Picture {{ image_count.value }}: {% endif -%}\n                    <|vision_start|><|image_pad|><|vision_end|>\n                {%- elif content.type == 'video' or 'video' in content %}\n                    {%- set video_count.value = video_count.value + 1 %}\n                    {%- if add_vision_id %}Video {{ video_count.value }}: {% endif -%}\n                    <|vision_start|><|video_pad|><|vision_end|>\n                {%- elif 'text' in content %}\n                    {{- content.text }}\n                {%- endif %}\n            {%- endfor %}\n        {%- endif %}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n"
+}

checkpoint.json

@@ -0,0 +1,249 @@
+{
+  "checkpoint_type": "hf",
+  "cosmos3_nano_sim_bimanual": {
+    "attention_mode": "three_way",
+    "checkpoint_hash": "cc3075bfc7ffbf1ce361d0da01ee1348cc11d4b94aab980e648435306a6be53d",
+    "checkpoint_id": "simulation_sfdmd_bimanual_prod_64N_chunk2_v002_step4@iter_000001380",
+    "checkpoint_iteration": 1380,
+    "chunk_partition": [
+      1,
+      2
+    ],
+    "chunk_size": 2,
+    "conditioning": {
+      "action_tokens_per_frame": 4,
+      "contract_sha256": "bb69c3678ea0a17e0685933606b44d5b9fe946d1f657fa945d7b4dce321723df",
+      "default_embodiment": "agibotworld",
+      "embodiments": {
+        "abc_yam": {
+          "domain_id": 16,
+          "input_action_dim": 59
+        },
+        "agibot_gear_gripper": {
+          "domain_id": 15,
+          "input_action_dim": 59
+        },
+        "agibot_gear_gripper_ext": {
+          "domain_id": 15,
+          "input_action_dim": 59
+        },
+        "agibotworld": {
+          "domain_id": 15,
+          "input_action_dim": 59
+        },
+        "behavior1k_lerobot": {
+          "domain_id": 22,
+          "input_action_dim": 59
+        },
+        "camera_pose": {
+          "domain_id": 2,
+          "input_action_dim": 59
+        },
+        "hand_pose": {
+          "domain_id": 3,
+          "input_action_dim": 59
+        },
+        "molmoact2_yam": {
+          "domain_id": 16,
+          "input_action_dim": 59
+        },
+        "xdof_yam": {
+          "domain_id": 16,
+          "input_action_dim": 59
+        }
+      },
+      "input_contract": {
+        "action_dim": 59,
+        "action_space": "normalized_unified_v1",
+        "domain_routing": "scalar_or_per_action_row",
+        "model_mode": "forward_dynamics",
+        "runtime_normalization": false,
+        "validity": "masked_to_zero_after_normalization_by_source"
+      },
+      "layout": {
+        "fields": [
+          {
+            "name": "ego_pose",
+            "offset": 0,
+            "size": 9
+          },
+          {
+            "name": "right_wrist_pose",
+            "offset": 9,
+            "size": 9
+          },
+          {
+            "name": "right_fingertips",
+            "offset": 18,
+            "size": 15
+          },
+          {
+            "name": "right_gripper_width_m",
+            "offset": 33,
+            "size": 1
+          },
+          {
+            "name": "left_wrist_pose",
+            "offset": 34,
+            "size": 9
+          },
+          {
+            "name": "left_fingertips",
+            "offset": 43,
+            "size": 15
+          },
+          {
+            "name": "left_gripper_width_m",
+            "offset": 58,
+            "size": 1
+          }
+        ],
+        "id": "unified_v1",
+        "pose_convention": "backward_chunk_anchored_16f",
+        "rotation_representation": "rot6d_columns"
+      },
+      "mode": "action",
+      "model_action_dim": 64,
+      "normalizer": {
+        "method": "global_asinh_unified_v1",
+        "runtime_application": false,
+        "source": {
+          "artifact_path": "cosmos3_nano_sim_bimanual_action_sources/ae695e2c643a5626b3c3564f00705aa3a83ab0da7640549c7340db71172594b0.json",
+          "path": "projects/cosmos3/cosmos3/datasets/action/normalizers/global_asinh_v1_backward_anchored_gripper_metric.json",
+          "pose_convention_authority": "resolved_canonical_lance_training_config",
+          "repository_revision": "970970f3a3024563973d4114e7c60cfda62c3af0",
+          "sha256": "ae695e2c643a5626b3c3564f00705aa3a83ab0da7640549c7340db71172594b0"
+        }
+      },
+      "num_embodiment_domains": 32,
+      "padding": {
+        "stage": "after_normalization_and_validity_mask",
+        "value": 0.0
+      },
+      "schema_version": 5,
+      "training_config_excerpt": {
+        "datasets": [
+          {
+            "camera_mask": false,
+            "embodiment": "camera_pose",
+            "name": "camera_lance_480_chunk_anchored",
+            "pose_convention": "backward_chunk_anchored_16f",
+            "rotation_format": "rot6d",
+            "target": "projects.cosmos3.cosmos3.datasets.action.lance.lance_camera_dataset.LanceCameraDataset",
+            "translation_scale": 1.0
+          },
+          {
+            "camera_mask": true,
+            "embodiment": "agibotworld",
+            "name": "agibotworld_beta",
+            "pose_convention": "backward_chunk_anchored_16f",
+            "rotation_format": "rot6d",
+            "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+            "translation_scale": 1.0
+          },
+          {
+            "camera_mask": true,
+            "embodiment": "agibot_gear_gripper",
+            "name": "agibotgeargripper_offshelf",
+            "pose_convention": "backward_chunk_anchored_16f",
+            "rotation_format": "rot6d",
+            "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+            "translation_scale": 1.0
+          },
+          {
+            "camera_mask": true,
+            "embodiment": "agibot_gear_gripper",
+            "name": "agibotgeargripper_custom",
+            "pose_convention": "backward_chunk_anchored_16f",
+            "rotation_format": "rot6d",
+            "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+            "translation_scale": 1.0
+          },
+          {
+            "camera_mask": true,
+            "embodiment": "agibot_gear_gripper_ext",
+            "name": "agibotgeargripper_customext",
+            "pose_convention": "backward_chunk_anchored_16f",
+            "rotation_format": "rot6d",
+            "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+            "translation_scale": 1.0
+          },
+          {
+            "camera_mask": true,
+            "embodiment": "molmoact2_yam",
+            "name": "molmoact2_yam",
+            "pose_convention": "backward_chunk_anchored_16f",
+            "rotation_format": "rot6d",
+            "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceYAMDataset",
+            "translation_scale": 1.0
+          },
+          {
+            "camera_mask": true,
+            "embodiment": "xdof_yam",
+            "name": "xdofyam",
+            "pose_convention": "backward_chunk_anchored_16f",
+            "rotation_format": "rot6d",
+            "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceYAMDataset",
+            "translation_scale": 1.0
+          },
+          {
+            "camera_mask": true,
+            "embodiment": "abc_yam",
+            "name": "abcyam",
+            "pose_convention": "backward_chunk_anchored_16f",
+            "rotation_format": "rot6d",
+            "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceYAMDataset",
+            "translation_scale": 1.0
+          },
+          {
+            "camera_mask": true,
+            "embodiment": "behavior1k_lerobot",
+            "name": "behavior1k",
+            "pose_convention": "backward_chunk_anchored_16f",
+            "rotation_format": "rot6d",
+            "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceBehavior1KDataset",
+            "translation_scale": 1.0
+          }
+        ],
+        "experiment": "simulation_sfdmd_bimanual_prod_64N_chunk2_v002_step4",
+        "loader": {
+          "action_schema": "unified_v1",
+          "append_action_caption_semantics": false,
+          "append_duration_fps_timestamps": false,
+          "append_resolution_info": false,
+          "append_viewpoint_info": false,
+          "enable_mode_specific_prompt": false,
+          "format_prompt_as_json": false,
+          "global_asinh": true,
+          "metric_gripper": true
+        }
+      }
+    },
+    "fixed_step_sampler_config": {
+      "num_train_timesteps": 1000,
+      "sample_type": "sde",
+      "t_list": [
+        1.0,
+        0.9375,
+        0.8333333333333334,
+        0.625
+      ]
+    },
+    "latent_patch_size": 2,
+    "mrope": {
+      "base_fps": 24,
+      "enable_fps_modulation": true,
+      "reset_spatial_ids": true,
+      "temporal_modality_margin": 15000
+    },
+    "schema_version": 1,
+    "sink_frames": 0,
+    "temporal_compression_factor": 4,
+    "text_cache_max_len": 512,
+    "vae_spatial_compression_factor": 16,
+    "video_temporal_causal": true,
+    "window_frames": null
+  },
+  "source_weights": "ema",
+  "student_only": true
+}

config.json

@@ -0,0 +1,424 @@
+{
+  "allow_patterns_overrides": [
+    "*/*.safetensors"
+  ],
+  "architectures": [
+    "Cosmos3ForConditionalGeneration"
+  ],
+  "cosmos3_nano_sim_bimanual": {
+    "attention_mode": "three_way",
+    "base_fps": 24,
+    "checkpoint_hash": "cc3075bfc7ffbf1ce361d0da01ee1348cc11d4b94aab980e648435306a6be53d",
+    "checkpoint_id": "simulation_sfdmd_bimanual_prod_64N_chunk2_v002_step4@iter_000001380",
+    "checkpoint_iteration": 1380,
+    "chunk_size": 2,
+    "enable_fps_modulation": true,
+    "fixed_step_sampler_config": {
+      "num_train_timesteps": 1000,
+      "sample_type": "sde",
+      "t_list": [
+        1.0,
+        0.9375,
+        0.8333333333333334,
+        0.625
+      ]
+    },
+    "latent_patch_size": 2,
+    "schema_version": 1,
+    "sink_frames": 0,
+    "temporal_compression_factor": 4,
+    "temporal_modality_margin": 15000,
+    "text_cache_max_len": 1024,
+    "unified_3d_mrope_reset_spatial_ids": true,
+    "vae_spatial_compression_factor": 16,
+    "video_temporal_causal": true,
+    "window_frames": null
+  },
+  "image_token_id": 151655,
+  "model": {
+    "_recursive_": false,
+    "_target": "omni_mot_causal_model",
+    "config": {
+      "_type": "omni_mot_causal_model_config",
+      "action_gen": true,
+      "action_global_asinh": false,
+      "action_io_projector_type": "domain_aware",
+      "action_metric_gripper": false,
+      "action_schema": "native",
+      "activation_checkpointing": {
+        "_type": "activation_checkpointing_config",
+        "determinism_check": "default",
+        "min_cgroup_memory_free_fraction": 0.1,
+        "mode": "selective",
+        "offload_to_cpu": false,
+        "preserve_rng_state": true,
+        "save_only_marked_ops": false,
+        "save_ops_regex": [
+          "fmha_forward",
+          "_flash_attn_varlen_forward",
+          "_flash_attn_forward",
+          "fmha_fwd"
+        ]
+      },
+      "ar_distilled_sample_type": null,
+      "ar_static_und_cache_max_len": 512,
+      "attention_io_layout": "sequence_sharded",
+      "attention_sink_size": 0,
+      "causal_training_strategy": "none",
+      "clamp_empty_varlen_kv": true,
+      "compile": {
+        "_type": "compile_config",
+        "ar_post_saturation_mode": "default",
+        "compile_dynamic": true,
+        "compiled_region": "language",
+        "coordinate_descent_tuning": false,
+        "cuda_graph_scope": "block",
+        "enabled": false,
+        "mark_unbacked": false,
+        "max_autotune_pointwise": false,
+        "use_cuda_graphs": false
+      },
+      "correct_cp_gradients": false,
+      "diffusion_expert_config": {
+        "_type": "diffusion_expert_config",
+        "base_fps": 24,
+        "enable_action_modality_embedding": true,
+        "enable_fps_modulation": true,
+        "enable_media_modality_embedding": false,
+        "enable_sound_modality_embedding": true,
+        "enable_vision_modality_embeddings": false,
+        "lidar_patch_spatial_hw": null,
+        "load_weights_from_pretrained": false,
+        "max_vae_latent_side_after_patchify": 52,
+        "num_view_embeddings": 0,
+        "patch_spatial": 2,
+        "radar_patch_spatial_hw": null,
+        "sound_base_temporal_compression_factor": null,
+        "timestep_range": 1.0,
+        "unified_3d_mrope_reset_spatial_ids": true,
+        "unified_3d_mrope_temporal_modality_margin": 15000,
+        "vision_temporal_position_mode": "latent_index"
+      },
+      "ema": {
+        "_type": "ema_config",
+        "enabled": false,
+        "iteration_shift": 0,
+        "rate": 0.1
+      },
+      "enable_input_bias": true,
+      "enable_moba": false,
+      "exclude_reasoner_weights_from_checkpoint": false,
+      "fixed_step_sampler_config": {
+        "_type": "fixed_step_sampler_config",
+        "sample_type": "sde",
+        "t_list": [
+          1.0,
+          0.9375,
+          0.8333333333333334,
+          0.625
+        ]
+      },
+      "flex_attention": {
+        "_type": "causal_flex_attention_config",
+        "backend": "auto",
+        "deduplicate_cross_view": false,
+        "geometry_position_encoding": "baseline",
+        "mask": {
+          "_type": "causal_flex_attention_mask_config",
+          "attention_scope": "all_views",
+          "control_attends_sensor": false,
+          "control_to_control_window": null,
+          "control_to_sensor_window": null,
+          "decomposed_temporal_window_includes_first_frame": false,
+          "decomposed_temporal_window_seconds": null,
+          "lidar_attends_captions": true,
+          "radar_attends_captions": true,
+          "sensor_to_control_window": null,
+          "sensor_to_sensor_window": null
+        },
+        "maskless_cross_view_band_keys": "all_views",
+        "maskless_cross_view_band_radius": 0,
+        "maskless_cross_view_include_frame_zero": false,
+        "prope_intrinsics": false,
+        "rigrope_canonical_frame": false,
+        "rigrope_channels": "all",
+        "rigrope_head_fraction": 1.0,
+        "rigrope_include_time": false,
+        "rigrope_max_freq_exponent": 3.0,
+        "rigrope_min_freq_exponent": 0.0,
+        "rigrope_missing_geometry": "error",
+        "rigrope_moment_normalization": "fixed",
+        "rigrope_moment_scale_floor_m": 0.05,
+        "rigrope_moment_scale_m": 25.0,
+        "rigrope_pose_world_frame": "scene",
+        "rigrope_time_scale_s": 1.0
+      },
+      "grad_clip": false,
+      "input_caption_key": "ai_caption",
+      "input_image_key": "images",
+      "input_video_key": "video",
+      "joint_attn_implementation": "three_way",
+      "kv_cache_dtype": null,
+      "kv_cache_inference_size": null,
+      "kv_cache_kernel_impl": "triton",
+      "latent_downsample_factor": 16,
+      "lbl": {
+        "_type": "lbl_config",
+        "coeff_gen": null,
+        "coeff_und": null,
+        "method": "local"
+      },
+      "lidar_fps": null,
+      "lidar_state_ch": null,
+      "lidar_tokenizer": null,
+      "load_vision_tokenizer": true,
+      "lora_alpha": 32,
+      "lora_enabled": false,
+      "lora_rank": 16,
+      "lora_target_modules": "q_proj_moe_gen,k_proj_moe_gen,v_proj_moe_gen,o_proj_moe_gen",
+      "max_action_dim": 64,
+      "max_num_tokens_after_packing": 500000,
+      "moba_bidirectional_steps": 1,
+      "moba_causal_steps": 1,
+      "multiview_action_conditioning": null,
+      "multiview_attention": {
+        "_type": "flex_attention_config",
+        "backend": "auto",
+        "deduplicate_cross_view": false,
+        "geometry_position_encoding": "baseline",
+        "mask": {
+          "_type": "flex_attention_mask_config",
+          "attention_scope": "all_views",
+          "control_attends_sensor": false,
+          "control_to_control_window": null,
+          "control_to_sensor_window": null,
+          "decomposed_temporal_window_includes_first_frame": false,
+          "decomposed_temporal_window_seconds": null,
+          "lidar_attends_captions": true,
+          "radar_attends_captions": true,
+          "sensor_to_control_window": null,
+          "sensor_to_sensor_window": null
+        },
+        "maskless_cross_view_band_keys": "all_views",
+        "maskless_cross_view_band_radius": 0,
+        "maskless_cross_view_include_frame_zero": false,
+        "prope_intrinsics": false,
+        "rigrope_canonical_frame": false,
+        "rigrope_channels": "all",
+        "rigrope_head_fraction": 1.0,
+        "rigrope_include_time": false,
+        "rigrope_max_freq_exponent": 3.0,
+        "rigrope_min_freq_exponent": 0.0,
+        "rigrope_missing_geometry": "error",
+        "rigrope_moment_normalization": "fixed",
+        "rigrope_moment_scale_floor_m": 0.05,
+        "rigrope_moment_scale_m": 25.0,
+        "rigrope_pose_world_frame": "scene",
+        "rigrope_time_scale_s": 1.0
+      },
+      "natten_parameter_list": null,
+      "net": null,
+      "num_embodiment_domains": 32,
+      "nvfp4_linears": null,
+      "parallelism": {
+        "_type": "parallelism_config",
+        "cfg_parallel_shard_degree": 1,
+        "context_parallel_shard_degree": 1,
+        "data_parallel_replicate_degree": 1,
+        "data_parallel_shard_degree": 1,
+        "enable_inference_mode": false,
+        "fsdp_cpu_offload": false,
+        "fsdp_master_dtype": "float32",
+        "fsdp_reduce_dtype": null,
+        "vae_load_balance_group_size": 1
+      },
+      "precision": "bfloat16",
+      "radar_fps": null,
+      "radar_state_ch": null,
+      "radar_tokenizer": null,
+      "rectified_flow_inference_config": {
+        "_type": "rectified_flow_inference_config",
+        "num_train_timesteps": 1000,
+        "scheduler_type": "unipc",
+        "shift": 1,
+        "use_dynamic_shifting": false
+      },
+      "rectified_flow_training_config": {
+        "_type": "rectified_flow_training_config",
+        "action_loss_weight": 10.0,
+        "exclude_fully_conditioned_items": null,
+        "image_loss_scale": null,
+        "independent_action_schedule": false,
+        "independent_sound_schedule": false,
+        "lidar_loss_scale": null,
+        "loss_reduction_mode": "local_item_mean",
+        "loss_scale": 10.0,
+        "normalize_loss_by_active": false,
+        "radar_loss_scale": null,
+        "shift": {
+          "256": 3,
+          "480": 5
+        },
+        "shift_action": null,
+        "shift_image": null,
+        "shift_lidar": null,
+        "shift_sound": null,
+        "sound_loss_scale": null,
+        "train_time_action_distribution": "logitnormal",
+        "train_time_image_distribution": "logitnormal",
+        "train_time_sound_distribution": "logitnormal",
+        "train_time_video_distribution": "uniform",
+        "train_time_weight": "uniform",
+        "use_discrete_rf": false,
+        "use_dynamic_shift": false
+      },
+      "release_cp_raw_batch_after_preprocessing": false,
+      "resolution": "480",
+      "rolling_kv_cache_chunks": null,
+      "rolling_kv_sink_chunks": 1,
+      "rolling_media_rope_scale": 1.0,
+      "rolling_media_sink_age_cap_seconds": null,
+      "rolling_media_text_extent": null,
+      "rolling_prompt_context_after": 0,
+      "rolling_prompt_context_before": 0,
+      "rolling_prompt_policy": null,
+      "rolling_static_caption_max_tokens": 576,
+      "rolling_text_position_mode": "fixed",
+      "rolling_text_sink_tokens": 4,
+      "sigma_diffusion_forcing": 0.02,
+      "sound_dim": null,
+      "sound_gen": false,
+      "sound_latent_fps": 25,
+      "sound_tokenizer": null,
+      "sr_latent_condition_noise": null,
+      "state_ch": 48,
+      "state_t": 300,
+      "teacher_forcing_detach_clean_kv": false,
+      "teacher_forcing_frames_per_chunk": 2,
+      "teacher_forcing_kv_implementation": "singleview_threeway_kv",
+      "teacher_forcing_replay_policy": {
+        "_type": "teacher_forcing_replay_policy_config",
+        "clean_pass_causality": "frame",
+        "control_visibility": "global",
+        "controls_read_strict_past_clean_rgb": false,
+        "decomposed_temporal_window_seconds": null,
+        "deduplicate_cross_view": false,
+        "multiview_attention_scope": "all_views"
+      },
+      "teacher_forcing_target_only_no_text_pass2": false,
+      "teacher_forcing_transfer_control_dropout_rate": 0.0,
+      "timestep_embedding_chunk_size": 128000,
+      "tokenizer": {
+        "_target": "wan2pt2_vae_interface",
+        "bucket_name": "bucket",
+        "causal": true,
+        "chunk_duration": 93,
+        "encode_bucket_multiple": null,
+        "encode_chunk_frames": {
+          "256": 68,
+          "480": 24
+        },
+        "encode_exact_durations": null,
+        "keep_decoder_cache": false,
+        "object_store_credential_path_pretrained": null,
+        "spatial_compression_factor": 16,
+        "temporal_compression_factor": 4,
+        "temporal_window": null,
+        "use_streaming_encode": false,
+        "vae_path": "pretrained/tokenizers/video/wan2pt2/Wan2.2_VAE.pth"
+      },
+      "video_temporal_causal": true,
+      "vision_gen": true,
+      "vlm_config": {
+        "_type": "vlm_config",
+        "layer_module": null,
+        "model_instance": {
+          "_target": "qwen3_vl_text_for_causal_lm",
+          "config": {
+            "_target": "create_vlm_config",
+            "base_config": {
+              "_target": "qwen3_vl_mot_config_from_json_file",
+              "json_file": "cosmos3://vfm/models/vlm/qwen3_vl/configs/Qwen3-VL-8B-Instruct.json"
+            },
+            "qk_norm_for_text": true
+          }
+        },
+        "model_name": "nvidia/Cosmos3-Nano-Reasoner",
+        "pretrained_weights": {
+          "_type": "pretrained_weights_config",
+          "backbone_path": "",
+          "checkpoint_format": null,
+          "credentials_path": "",
+          "enable_gcs_patch_in_boto3": false,
+          "enabled": false
+        },
+        "qk_norm": false,
+        "safetensors_path": "",
+        "tie_word_embeddings": false,
+        "tokenizer": {
+          "_target": "create_qwen2_tokenizer_with_download",
+          "config_variant": "hf",
+          "pretrained_model_name": "Qwen/Qwen3-VL-8B-Instruct"
+        },
+        "use_system_prompt": false
+      }
+    }
+  },
+  "model_type": "cosmos3_omni",
+  "per_view_captions": false,
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 151643,
+    "dtype": "bfloat16",
+    "eos_token_id": 151645,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 4096,
+    "initializer_range": 0.02,
+    "intermediate_size": 12288,
+    "max_position_embeddings": 262144,
+    "model_type": "qwen3_vl_text",
+    "num_attention_heads": 32,
+    "num_hidden_layers": 36,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 1e-06,
+    "rope_scaling": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        24,
+        20,
+        20
+      ],
+      "rope_type": "default"
+    },
+    "rope_theta": 5000000,
+    "use_cache": true,
+    "vocab_size": 151936
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.0.dev0",
+  "video_token_id": 151656,
+  "vision_config": {
+    "deepstack_visual_indexes": [
+      8,
+      16,
+      24
+    ],
+    "depth": 27,
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4304,
+    "model_type": "qwen3_vl",
+    "num_heads": 16,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 4096,
+    "patch_size": 16,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 151653,
+  "vision_start_token_id": 151652
+}

cosmos3_nano_sim_bimanual_action_sources/4d38b6321a92741c93e8cf1e96cba18979399ddb5c037708d3aaeffd666faa10.json

@@ -0,0 +1,41 @@
+{
+  "metadata": {
+    "embodiment_type":    "agibot",
+    "pose_convention":    "backward_framewise",
+    "rotation_format":    "rot6d",
+    "action_dim":         29,
+    "skip_rotation_dims": [3, 4, 5, 6, 7, 8, 12, 13, 14, 15, 16, 17, 22, 23, 24, 25, 26, 27],
+    "chunk_length":       16,
+    "sample_stride":      16,
+    "dataset_name":       "agibot_shared_gripper_beta",
+    "dataset_class":      "AgibotGEARGripperDataset+AgibotGEARGripperExtDataset+AgiBotWorldBetaDataset",
+    "dataset_root":       "AgiBotWorld-GEAR + AgiBotWorld-Beta",
+    "split":              "train",
+    "num_samples_stats":  200000,
+    "reservoir_size":     200000,
+    "max_samples":        200000,
+    "sampling_seed":      42,
+    "dataset_lengths":    {
+      "agibot_gear_gripper": 10723406,
+      "agibot_gear_gripper_ext": 209875,
+      "agibotworld_beta": 16688991
+    },
+    "normalizer_note":    "Shared AgiBot 29D FK-pose normalizer for GEAR gripper/ext and AgiBotWorld-Beta. Intended for action_normalization=asinh_rot to compress valid heavy-tailed base motion without clamping."
+  },
+  "global": {
+    "mean": [ 0.000007, -0.000028,  0.000017,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.000054,  0.000055, -0.000004,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.461416, -0.000065,  0.000066,  0.000026,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.511525],
+    "std":  [ 0.001146,  0.003775,  0.006236,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  0.005170,  0.005957,  0.005775,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  0.365715,  0.004577,  0.006320,  0.004809,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  0.368419],
+    "min":  [-0.332076, -0.461419, -1.287811, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.181742, -1.008569, -0.964748, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000,  0.000000, -1.086141, -1.194795, -0.704462, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000,  0.000000],
+    "max":  [ 0.332075,  0.461429,  1.287816,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.181742,  1.008569,  0.968047,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.077497,  1.194157,  0.696745,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000],
+    "q01":  [-0.000474, -0.004285, -0.006603, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -0.011330, -0.012649, -0.014966, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000,  0.000000, -0.010683, -0.011185, -0.012754, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000,  0.000000],
+    "q99":  [ 0.000472,  0.003180,  0.007315,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  0.011657,  0.012629,  0.013074,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  0.010122,  0.011385,  0.011604,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000]
+  },
+  "global_raw": {
+    "mean": [ 0.000007, -0.000028,  0.000017,  0.999997,  0.000012, -0.000011, -0.000013,  0.999992,  0.000013,  0.000054,  0.000055, -0.000004,  0.999755,  0.000162, -0.000206, -0.000163,  0.999770, -0.000068,  0.461416, -0.000065,  0.000066,  0.000026,  0.999821, -0.000156,  0.000218,  0.000156,  0.999840, -0.000065,  0.511525],
+    "std":  [ 0.001146,  0.003775,  0.006236,  0.000050,  0.001322,  0.001990,  0.001321,  0.000091,  0.003722,  0.005170,  0.005957,  0.005775,  0.001184,  0.016934,  0.014178,  0.016919,  0.001112,  0.013136,  0.365715,  0.004577,  0.006320,  0.004809,  0.001140,  0.014271,  0.012406,  0.014263,  0.000987,  0.010745,  0.368419],
+    "min":  [-0.332076, -0.461419, -1.287811,  0.992475, -0.047897, -0.115986, -0.048200,  0.986519, -0.163639, -1.181742, -1.008569, -0.964748,  0.834839, -0.535909, -0.508249, -0.417918,  0.835656, -0.360748,  0.000000, -1.086141, -1.194795, -0.704462,  0.885655, -0.422226, -0.392999, -0.404670,  0.847586, -0.267267,  0.000000],
+    "max":  [ 0.332075,  0.461429,  1.287816,  1.000000,  0.048245,  0.111197,  0.047794,  1.000000,  0.130982,  1.181742,  1.008569,  0.968047,  1.000000,  0.413419,  0.506131,  0.545759,  1.000000,  0.405109,  1.000000,  1.077497,  1.194157,  0.696745,  1.000000,  0.405554,  0.460652,  0.428101,  1.000000,  0.423156,  1.000000],
+    "q01":  [-0.000474, -0.004285, -0.006603,  0.999969, -0.000524, -0.000606, -0.000540,  0.999801, -0.000404, -0.011330, -0.012649, -0.014966,  0.995746, -0.055268, -0.044579, -0.057030,  0.995879, -0.041778,  0.000000, -0.010683, -0.011185, -0.012754,  0.996474, -0.048846, -0.038892, -0.044784,  0.996874, -0.034269,  0.000000],
+    "q99":  [ 0.000472,  0.003180,  0.007315,  1.000000,  0.000532,  0.000603,  0.000514,  1.000000,  0.000454,  0.011657,  0.012629,  0.013074,  1.000000,  0.057221,  0.045662,  0.055355,  1.000000,  0.042508,  1.000000,  0.010122,  0.011385,  0.011604,  1.000000,  0.044671,  0.039433,  0.048886,  1.000000,  0.036261,  1.000000]
+  }
+}

cosmos3_nano_sim_bimanual_action_sources/action_schema.json

@@ -0,0 +1,208 @@
+{
+  "action_tokens_per_frame": 4,
+  "contract_sha256": "bb69c3678ea0a17e0685933606b44d5b9fe946d1f657fa945d7b4dce321723df",
+  "default_embodiment": "agibotworld",
+  "embodiments": {
+    "abc_yam": {
+      "domain_id": 16,
+      "input_action_dim": 59
+    },
+    "agibot_gear_gripper": {
+      "domain_id": 15,
+      "input_action_dim": 59
+    },
+    "agibot_gear_gripper_ext": {
+      "domain_id": 15,
+      "input_action_dim": 59
+    },
+    "agibotworld": {
+      "domain_id": 15,
+      "input_action_dim": 59
+    },
+    "behavior1k_lerobot": {
+      "domain_id": 22,
+      "input_action_dim": 59
+    },
+    "camera_pose": {
+      "domain_id": 2,
+      "input_action_dim": 59
+    },
+    "hand_pose": {
+      "domain_id": 3,
+      "input_action_dim": 59
+    },
+    "molmoact2_yam": {
+      "domain_id": 16,
+      "input_action_dim": 59
+    },
+    "xdof_yam": {
+      "domain_id": 16,
+      "input_action_dim": 59
+    }
+  },
+  "input_contract": {
+    "action_dim": 59,
+    "action_space": "normalized_unified_v1",
+    "domain_routing": "scalar_or_per_action_row",
+    "model_mode": "forward_dynamics",
+    "runtime_normalization": false,
+    "validity": "masked_to_zero_after_normalization_by_source"
+  },
+  "layout": {
+    "fields": [
+      {
+        "name": "ego_pose",
+        "offset": 0,
+        "size": 9
+      },
+      {
+        "name": "right_wrist_pose",
+        "offset": 9,
+        "size": 9
+      },
+      {
+        "name": "right_fingertips",
+        "offset": 18,
+        "size": 15
+      },
+      {
+        "name": "right_gripper_width_m",
+        "offset": 33,
+        "size": 1
+      },
+      {
+        "name": "left_wrist_pose",
+        "offset": 34,
+        "size": 9
+      },
+      {
+        "name": "left_fingertips",
+        "offset": 43,
+        "size": 15
+      },
+      {
+        "name": "left_gripper_width_m",
+        "offset": 58,
+        "size": 1
+      }
+    ],
+    "id": "unified_v1",
+    "pose_convention": "backward_chunk_anchored_16f",
+    "rotation_representation": "rot6d_columns"
+  },
+  "model_action_dim": 64,
+  "normalizer": {
+    "method": "global_asinh_unified_v1",
+    "runtime_application": false,
+    "source": {
+      "artifact_path": "cosmos3_nano_sim_bimanual_action_sources/ae695e2c643a5626b3c3564f00705aa3a83ab0da7640549c7340db71172594b0.json",
+      "path": "projects/cosmos3/cosmos3/datasets/action/normalizers/global_asinh_v1_backward_anchored_gripper_metric.json",
+      "pose_convention_authority": "resolved_canonical_lance_training_config",
+      "repository_revision": "970970f3a3024563973d4114e7c60cfda62c3af0",
+      "sha256": "ae695e2c643a5626b3c3564f00705aa3a83ab0da7640549c7340db71172594b0"
+    }
+  },
+  "num_embodiment_domains": 32,
+  "padding": {
+    "stage": "after_normalization_and_validity_mask",
+    "value": 0.0
+  },
+  "schema_version": 5,
+  "training_config_excerpt": {
+    "datasets": [
+      {
+        "camera_mask": false,
+        "embodiment": "camera_pose",
+        "name": "camera_lance_480_chunk_anchored",
+        "pose_convention": "backward_chunk_anchored_16f",
+        "rotation_format": "rot6d",
+        "target": "projects.cosmos3.cosmos3.datasets.action.lance.lance_camera_dataset.LanceCameraDataset",
+        "translation_scale": 1.0
+      },
+      {
+        "camera_mask": true,
+        "embodiment": "agibotworld",
+        "name": "agibotworld_beta",
+        "pose_convention": "backward_chunk_anchored_16f",
+        "rotation_format": "rot6d",
+        "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+        "translation_scale": 1.0
+      },
+      {
+        "camera_mask": true,
+        "embodiment": "agibot_gear_gripper",
+        "name": "agibotgeargripper_offshelf",
+        "pose_convention": "backward_chunk_anchored_16f",
+        "rotation_format": "rot6d",
+        "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+        "translation_scale": 1.0
+      },
+      {
+        "camera_mask": true,
+        "embodiment": "agibot_gear_gripper",
+        "name": "agibotgeargripper_custom",
+        "pose_convention": "backward_chunk_anchored_16f",
+        "rotation_format": "rot6d",
+        "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+        "translation_scale": 1.0
+      },
+      {
+        "camera_mask": true,
+        "embodiment": "agibot_gear_gripper_ext",
+        "name": "agibotgeargripper_customext",
+        "pose_convention": "backward_chunk_anchored_16f",
+        "rotation_format": "rot6d",
+        "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+        "translation_scale": 1.0
+      },
+      {
+        "camera_mask": true,
+        "embodiment": "molmoact2_yam",
+        "name": "molmoact2_yam",
+        "pose_convention": "backward_chunk_anchored_16f",
+        "rotation_format": "rot6d",
+        "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceYAMDataset",
+        "translation_scale": 1.0
+      },
+      {
+        "camera_mask": true,
+        "embodiment": "xdof_yam",
+        "name": "xdofyam",
+        "pose_convention": "backward_chunk_anchored_16f",
+        "rotation_format": "rot6d",
+        "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceYAMDataset",
+        "translation_scale": 1.0
+      },
+      {
+        "camera_mask": true,
+        "embodiment": "abc_yam",
+        "name": "abcyam",
+        "pose_convention": "backward_chunk_anchored_16f",
+        "rotation_format": "rot6d",
+        "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceYAMDataset",
+        "translation_scale": 1.0
+      },
+      {
+        "camera_mask": true,
+        "embodiment": "behavior1k_lerobot",
+        "name": "behavior1k",
+        "pose_convention": "backward_chunk_anchored_16f",
+        "rotation_format": "rot6d",
+        "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceBehavior1KDataset",
+        "translation_scale": 1.0
+      }
+    ],
+    "experiment": "simulation_sfdmd_bimanual_prod_64N_chunk2_v002_step4",
+    "loader": {
+      "action_schema": "unified_v1",
+      "append_action_caption_semantics": false,
+      "append_duration_fps_timestamps": false,
+      "append_resolution_info": false,
+      "append_viewpoint_info": false,
+      "enable_mode_specific_prompt": false,
+      "format_prompt_as_json": false,
+      "global_asinh": true,
+      "metric_gripper": true
+    }
+  }
+}

cosmos3_nano_sim_bimanual_action_sources/ae695e2c643a5626b3c3564f00705aa3a83ab0da7640549c7340db71172594b0.json

@@ -0,0 +1,162 @@
+{
+  "metadata": {
+    "name": "global_asinh_v1_backward_anchored_gripper_metric",
+    "method": "global_asinh",
+    "version": 3,
+    "action_schema": "unified_v1",
+    "dim": 59,
+    "allowed_pose_conventions": [
+      "backward_anchored"
+    ],
+    "rotation_format": "rot6d",
+    "gripper_representation": "metric_width_m",
+    "metric_gripper_reference_width_m": 0.082,
+    "description": "q01 and q99 encode offset=(q01+q99)/2 and scale=(q99-q01)/2 for the canonical 59D action.",
+    "provenance": {
+      "ego_translation_override": "Set zero offset and the same 0.2822-meter scale for x, y, and z.",
+      "ego_translation_rationale": "AV motion is primarily along z while general camera motion is directionally unconstrained, so there is no robust cross-domain rule for axis-specific centers or scales.",
+      "ego_rotation": "hand_pose_backward_anchored_rot6d.json global_raw[3:9]",
+      "right_wrist_pose_and_openness": "yam_ee_backward_anchored_rot6d.json global_raw[10:20]",
+      "right_fingertips": "hand_pose_backward_anchored_rot6d.json global_raw[18:33]",
+      "left_wrist_pose_and_openness": "yam_ee_backward_anchored_rot6d.json global_raw[0:10]",
+      "left_fingertips": "hand_pose_backward_anchored_rot6d.json global_raw[42:57]",
+      "metric_gripper_override": "Set both gripper slots to YAM's zero-closed, 0.082-meter fully-open reference range."
+    },
+    "layout": {
+      "0:3": "ego translation xyz in meters; zero-centered with scale 0.2822 meters per axis",
+      "3:9": "ego rotation as rot6d from HandPose",
+      "9:12": "right wrist translation xyz in meters from YAM",
+      "12:18": "right wrist rotation as rot6d from YAM",
+      "18:33": "right fingertips: thumb, index, middle, ring, pinky xyz in the wrist frame from HandPose",
+      "33:34": "right gripper width in meters, where 0 is closed and 0.082 is the YAM fully-open reference width",
+      "34:37": "left wrist translation xyz in meters from YAM",
+      "37:43": "left wrist rotation as rot6d from YAM",
+      "43:58": "left fingertips: thumb, index, middle, ring, pinky xyz in the wrist frame from HandPose",
+      "58:59": "left gripper width in meters, where 0 is closed and 0.082 is the YAM fully-open reference width"
+    }
+  },
+  "global": {
+    "q01": [
+      -0.2822,
+      -0.2822,
+      -0.2822,
+      0.709793,
+      -0.528049,
+      -0.211989,
+      -0.526309,
+      0.717806,
+      -0.2239,
+      -0.121196,
+      -0.106861,
+      -0.161145,
+      0.604712,
+      -0.500095,
+      -0.529701,
+      -0.484055,
+      0.689496,
+      -0.384133,
+      -0.078661,
+      0.030485,
+      0.058749,
+      -0.037953,
+      0.011371,
+      0.063185,
+      -0.007476,
+      0.01086,
+      0.047803,
+      0.00622,
+      0.00845,
+      0.037389,
+      0.019229,
+      0.003273,
+      0.03427,
+      0,
+      -0.104546,
+      -0.095492,
+      -0.149185,
+      0.666787,
+      -0.419017,
+      -0.473956,
+      -0.444951,
+      0.744433,
+      -0.35486,
+      -0.0134,
+      0.024862,
+      0.053364,
+      -0.011772,
+      0.002479,
+      0.066106,
+      -0.037275,
+      0.000915,
+      0.053206,
+      -0.058168,
+      -0.002249,
+      0.043841,
+      -0.076916,
+      -0.005779,
+      0.039363,
+      0
+    ],
+    "q99": [
+      0.2822,
+      0.2822,
+      0.2822,
+      1,
+      0.521286,
+      0.203338,
+      0.529919,
+      0.999999,
+      0.259182,
+      0.124457,
+      0.12985,
+      0.123286,
+      1,
+      0.485177,
+      0.515304,
+      0.504032,
+      1,
+      0.440837,
+      0.016207,
+      0.105381,
+      0.129947,
+      0.014021,
+      0.090293,
+      0.187483,
+      0.037809,
+      0.088395,
+      0.189717,
+      0.058805,
+      0.084619,
+      0.175622,
+      0.073382,
+      0.069707,
+      0.143841,
+      0.082,
+      0.103266,
+      0.11501,
+      0.114157,
+      1,
+      0.440519,
+      0.501329,
+      0.416844,
+      1,
+      0.408429,
+      0.086612,
+      0.10718,
+      0.12828,
+      0.038691,
+      0.089021,
+      0.190503,
+      0.008158,
+      0.087324,
+      0.194924,
+      -0.006426,
+      0.083202,
+      0.181171,
+      -0.020051,
+      0.068746,
+      0.149127,
+      0.082
+    ]
+  }
+}

cosmos3_nano_sim_bimanual_action_sources/cosmos3_nano_sim_bimanual_action_schema.schema.json

@@ -0,0 +1,300 @@
+{
+  "type": "object",
+  "additionalProperties": false,
+  "required": [
+    "schema_version",
+    "action_tokens_per_frame",
+    "model_action_dim",
+    "num_embodiment_domains",
+    "default_embodiment",
+    "embodiments",
+    "padding",
+    "input_contract",
+    "layout",
+    "normalizer",
+    "training_config_excerpt",
+    "contract_sha256"
+  ],
+  "properties": {
+    "schema_version": {
+      "const": 5
+    },
+    "action_tokens_per_frame": {
+      "const": 4
+    },
+    "model_action_dim": {
+      "const": 64
+    },
+    "num_embodiment_domains": {
+      "const": 32
+    },
+    "default_embodiment": {
+      "enum": [
+        "agibotworld",
+        "agibot_gear_gripper",
+        "agibot_gear_gripper_ext",
+        "molmoact2_yam",
+        "xdof_yam",
+        "abc_yam",
+        "hand_pose",
+        "behavior1k_lerobot",
+        "camera_pose"
+      ]
+    },
+    "embodiments": {
+      "type": "object",
+      "minProperties": 1,
+      "additionalProperties": false,
+      "properties": {
+        "agibotworld": {
+          "const": {
+            "domain_id": 15,
+            "input_action_dim": 59
+          }
+        },
+        "agibot_gear_gripper": {
+          "const": {
+            "domain_id": 15,
+            "input_action_dim": 59
+          }
+        },
+        "agibot_gear_gripper_ext": {
+          "const": {
+            "domain_id": 15,
+            "input_action_dim": 59
+          }
+        },
+        "molmoact2_yam": {
+          "const": {
+            "domain_id": 16,
+            "input_action_dim": 59
+          }
+        },
+        "xdof_yam": {
+          "const": {
+            "domain_id": 16,
+            "input_action_dim": 59
+          }
+        },
+        "abc_yam": {
+          "const": {
+            "domain_id": 16,
+            "input_action_dim": 59
+          }
+        },
+        "hand_pose": {
+          "const": {
+            "domain_id": 3,
+            "input_action_dim": 59
+          }
+        },
+        "behavior1k_lerobot": {
+          "const": {
+            "domain_id": 22,
+            "input_action_dim": 59
+          }
+        },
+        "camera_pose": {
+          "const": {
+            "domain_id": 2,
+            "input_action_dim": 59
+          }
+        }
+      }
+    },
+    "padding": {
+      "const": {
+        "stage": "after_normalization_and_validity_mask",
+        "value": 0.0
+      }
+    },
+    "input_contract": {
+      "const": {
+        "action_space": "normalized_unified_v1",
+        "action_dim": 59,
+        "domain_routing": "scalar_or_per_action_row",
+        "validity": "masked_to_zero_after_normalization_by_source",
+        "runtime_normalization": false,
+        "model_mode": "forward_dynamics"
+      }
+    },
+    "layout": {
+      "const": {
+        "id": "unified_v1",
+        "pose_convention": "backward_chunk_anchored_16f",
+        "rotation_representation": "rot6d_columns",
+        "fields": [
+          {
+            "name": "ego_pose",
+            "offset": 0,
+            "size": 9
+          },
+          {
+            "name": "right_wrist_pose",
+            "offset": 9,
+            "size": 9
+          },
+          {
+            "name": "right_fingertips",
+            "offset": 18,
+            "size": 15
+          },
+          {
+            "name": "right_gripper_width_m",
+            "offset": 33,
+            "size": 1
+          },
+          {
+            "name": "left_wrist_pose",
+            "offset": 34,
+            "size": 9
+          },
+          {
+            "name": "left_fingertips",
+            "offset": 43,
+            "size": 15
+          },
+          {
+            "name": "left_gripper_width_m",
+            "offset": 58,
+            "size": 1
+          }
+        ]
+      }
+    },
+    "normalizer": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": [
+        "method",
+        "runtime_application",
+        "source"
+      ],
+      "properties": {
+        "method": {
+          "const": "global_asinh_unified_v1"
+        },
+        "runtime_application": {
+          "const": false
+        },
+        "source": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": [
+            "path",
+            "artifact_path",
+            "sha256",
+            "repository_revision",
+            "pose_convention_authority"
+          ],
+          "properties": {
+            "path": {
+              "const": "projects/cosmos3/cosmos3/datasets/action/normalizers/global_asinh_v1_backward_anchored_gripper_metric.json"
+            },
+            "artifact_path": {
+              "type": "string",
+              "pattern": "^cosmos3_nano_sim_bimanual_action_sources/[0-9a-f]{64}\\.json$"
+            },
+            "sha256": {
+              "type": "string",
+              "pattern": "^[0-9a-f]{64}$"
+            },
+            "repository_revision": {
+              "type": "string",
+              "pattern": "^[0-9a-f]{40}$"
+            },
+            "pose_convention_authority": {
+              "const": "resolved_canonical_lance_training_config"
+            }
+          }
+        }
+      }
+    },
+    "training_config_excerpt": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": [
+        "experiment",
+        "loader",
+        "datasets"
+      ],
+      "properties": {
+        "experiment": {
+          "type": "string",
+          "minLength": 1
+        },
+        "loader": {
+          "const": {
+            "action_schema": "unified_v1",
+            "global_asinh": true,
+            "metric_gripper": true,
+            "format_prompt_as_json": false,
+            "append_viewpoint_info": false,
+            "append_action_caption_semantics": false,
+            "append_duration_fps_timestamps": false,
+            "append_resolution_info": false,
+            "enable_mode_specific_prompt": false
+          }
+        },
+        "datasets": {
+          "type": "array",
+          "minItems": 1,
+          "items": {
+            "type": "object",
+            "additionalProperties": false,
+            "required": [
+              "name",
+              "target",
+              "embodiment",
+              "pose_convention",
+              "rotation_format",
+              "translation_scale",
+              "camera_mask"
+            ],
+            "properties": {
+              "name": {
+                "type": "string",
+                "minLength": 1
+              },
+              "target": {
+                "type": "string",
+                "minLength": 1
+              },
+              "embodiment": {
+                "enum": [
+                  "agibotworld",
+                  "agibot_gear_gripper",
+                  "agibot_gear_gripper_ext",
+                  "molmoact2_yam",
+                  "xdof_yam",
+                  "abc_yam",
+                  "hand_pose",
+                  "behavior1k_lerobot",
+                  "camera_pose"
+                ]
+              },
+              "pose_convention": {
+                "const": "backward_chunk_anchored_16f"
+              },
+              "rotation_format": {
+                "const": "rot6d"
+              },
+              "translation_scale": {
+                "const": 1.0
+              },
+              "camera_mask": {
+                "type": "boolean"
+              }
+            }
+          }
+        }
+      }
+    },
+    "contract_sha256": {
+      "type": "string",
+      "pattern": "^[0-9a-f]{64}$"
+    }
+  },
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "title": "Cosmos3 Sim Bimanual model-ready unified-v1 contract (schema 5)"
+}

cosmos3_nano_sim_bimanual_action_sources/resolved_action_config.json

@@ -0,0 +1,97 @@
+{
+  "datasets": [
+    {
+      "camera_mask": false,
+      "embodiment": "camera_pose",
+      "name": "camera_lance_480_chunk_anchored",
+      "pose_convention": "backward_chunk_anchored_16f",
+      "rotation_format": "rot6d",
+      "target": "projects.cosmos3.cosmos3.datasets.action.lance.lance_camera_dataset.LanceCameraDataset",
+      "translation_scale": 1.0
+    },
+    {
+      "camera_mask": true,
+      "embodiment": "agibotworld",
+      "name": "agibotworld_beta",
+      "pose_convention": "backward_chunk_anchored_16f",
+      "rotation_format": "rot6d",
+      "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+      "translation_scale": 1.0
+    },
+    {
+      "camera_mask": true,
+      "embodiment": "agibot_gear_gripper",
+      "name": "agibotgeargripper_offshelf",
+      "pose_convention": "backward_chunk_anchored_16f",
+      "rotation_format": "rot6d",
+      "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+      "translation_scale": 1.0
+    },
+    {
+      "camera_mask": true,
+      "embodiment": "agibot_gear_gripper",
+      "name": "agibotgeargripper_custom",
+      "pose_convention": "backward_chunk_anchored_16f",
+      "rotation_format": "rot6d",
+      "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+      "translation_scale": 1.0
+    },
+    {
+      "camera_mask": true,
+      "embodiment": "agibot_gear_gripper_ext",
+      "name": "agibotgeargripper_customext",
+      "pose_convention": "backward_chunk_anchored_16f",
+      "rotation_format": "rot6d",
+      "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceAgiBotDataset",
+      "translation_scale": 1.0
+    },
+    {
+      "camera_mask": true,
+      "embodiment": "molmoact2_yam",
+      "name": "molmoact2_yam",
+      "pose_convention": "backward_chunk_anchored_16f",
+      "rotation_format": "rot6d",
+      "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceYAMDataset",
+      "translation_scale": 1.0
+    },
+    {
+      "camera_mask": true,
+      "embodiment": "xdof_yam",
+      "name": "xdofyam",
+      "pose_convention": "backward_chunk_anchored_16f",
+      "rotation_format": "rot6d",
+      "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceYAMDataset",
+      "translation_scale": 1.0
+    },
+    {
+      "camera_mask": true,
+      "embodiment": "abc_yam",
+      "name": "abcyam",
+      "pose_convention": "backward_chunk_anchored_16f",
+      "rotation_format": "rot6d",
+      "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceYAMDataset",
+      "translation_scale": 1.0
+    },
+    {
+      "camera_mask": true,
+      "embodiment": "behavior1k_lerobot",
+      "name": "behavior1k",
+      "pose_convention": "backward_chunk_anchored_16f",
+      "rotation_format": "rot6d",
+      "target": "projects.cosmos3.cosmos3.datasets.action.canonical_lance_action_datasets.CanonicalLanceBehavior1KDataset",
+      "translation_scale": 1.0
+    }
+  ],
+  "experiment": "simulation_sfdmd_bimanual_prod_64N_chunk2_v002_step4",
+  "loader": {
+    "action_schema": "unified_v1",
+    "append_action_caption_semantics": false,
+    "append_duration_fps_timestamps": false,
+    "append_resolution_info": false,
+    "append_viewpoint_info": false,
+    "enable_mode_specific_prompt": false,
+    "format_prompt_as_json": false,
+    "global_asinh": true,
+    "metric_gripper": true
+  }
+}

cosmos_dreams_action_sources/4d38b6321a92741c93e8cf1e96cba18979399ddb5c037708d3aaeffd666faa10.json

@@ -0,0 +1,41 @@
+{
+  "metadata": {
+    "embodiment_type":    "agibot",
+    "pose_convention":    "backward_framewise",
+    "rotation_format":    "rot6d",
+    "action_dim":         29,
+    "skip_rotation_dims": [3, 4, 5, 6, 7, 8, 12, 13, 14, 15, 16, 17, 22, 23, 24, 25, 26, 27],
+    "chunk_length":       16,
+    "sample_stride":      16,
+    "dataset_name":       "agibot_shared_gripper_beta",
+    "dataset_class":      "AgibotGEARGripperDataset+AgibotGEARGripperExtDataset+AgiBotWorldBetaDataset",
+    "dataset_root":       "AgiBotWorld-GEAR + AgiBotWorld-Beta",
+    "split":              "train",
+    "num_samples_stats":  200000,
+    "reservoir_size":     200000,
+    "max_samples":        200000,
+    "sampling_seed":      42,
+    "dataset_lengths":    {
+      "agibot_gear_gripper": 10723406,
+      "agibot_gear_gripper_ext": 209875,
+      "agibotworld_beta": 16688991
+    },
+    "normalizer_note":    "Shared AgiBot 29D FK-pose normalizer for GEAR gripper/ext and AgiBotWorld-Beta. Intended for action_normalization=asinh_rot to compress valid heavy-tailed base motion without clamping."
+  },
+  "global": {
+    "mean": [ 0.000007, -0.000028,  0.000017,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.000054,  0.000055, -0.000004,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.461416, -0.000065,  0.000066,  0.000026,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.000000,  0.511525],
+    "std":  [ 0.001146,  0.003775,  0.006236,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  0.005170,  0.005957,  0.005775,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  0.365715,  0.004577,  0.006320,  0.004809,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  0.368419],
+    "min":  [-0.332076, -0.461419, -1.287811, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.181742, -1.008569, -0.964748, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000,  0.000000, -1.086141, -1.194795, -0.704462, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000,  0.000000],
+    "max":  [ 0.332075,  0.461429,  1.287816,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.181742,  1.008569,  0.968047,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.077497,  1.194157,  0.696745,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000],
+    "q01":  [-0.000474, -0.004285, -0.006603, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -0.011330, -0.012649, -0.014966, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000,  0.000000, -0.010683, -0.011185, -0.012754, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000, -1.000000,  0.000000],
+    "q99":  [ 0.000472,  0.003180,  0.007315,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  0.011657,  0.012629,  0.013074,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  0.010122,  0.011385,  0.011604,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000,  1.000000]
+  },
+  "global_raw": {
+    "mean": [ 0.000007, -0.000028,  0.000017,  0.999997,  0.000012, -0.000011, -0.000013,  0.999992,  0.000013,  0.000054,  0.000055, -0.000004,  0.999755,  0.000162, -0.000206, -0.000163,  0.999770, -0.000068,  0.461416, -0.000065,  0.000066,  0.000026,  0.999821, -0.000156,  0.000218,  0.000156,  0.999840, -0.000065,  0.511525],
+    "std":  [ 0.001146,  0.003775,  0.006236,  0.000050,  0.001322,  0.001990,  0.001321,  0.000091,  0.003722,  0.005170,  0.005957,  0.005775,  0.001184,  0.016934,  0.014178,  0.016919,  0.001112,  0.013136,  0.365715,  0.004577,  0.006320,  0.004809,  0.001140,  0.014271,  0.012406,  0.014263,  0.000987,  0.010745,  0.368419],
+    "min":  [-0.332076, -0.461419, -1.287811,  0.992475, -0.047897, -0.115986, -0.048200,  0.986519, -0.163639, -1.181742, -1.008569, -0.964748,  0.834839, -0.535909, -0.508249, -0.417918,  0.835656, -0.360748,  0.000000, -1.086141, -1.194795, -0.704462,  0.885655, -0.422226, -0.392999, -0.404670,  0.847586, -0.267267,  0.000000],
+    "max":  [ 0.332075,  0.461429,  1.287816,  1.000000,  0.048245,  0.111197,  0.047794,  1.000000,  0.130982,  1.181742,  1.008569,  0.968047,  1.000000,  0.413419,  0.506131,  0.545759,  1.000000,  0.405109,  1.000000,  1.077497,  1.194157,  0.696745,  1.000000,  0.405554,  0.460652,  0.428101,  1.000000,  0.423156,  1.000000],
+    "q01":  [-0.000474, -0.004285, -0.006603,  0.999969, -0.000524, -0.000606, -0.000540,  0.999801, -0.000404, -0.011330, -0.012649, -0.014966,  0.995746, -0.055268, -0.044579, -0.057030,  0.995879, -0.041778,  0.000000, -0.010683, -0.011185, -0.012754,  0.996474, -0.048846, -0.038892, -0.044784,  0.996874, -0.034269,  0.000000],
+    "q99":  [ 0.000472,  0.003180,  0.007315,  1.000000,  0.000532,  0.000603,  0.000514,  1.000000,  0.000454,  0.011657,  0.012629,  0.013074,  1.000000,  0.057221,  0.045662,  0.055355,  1.000000,  0.042508,  1.000000,  0.010122,  0.011385,  0.011604,  1.000000,  0.044671,  0.039433,  0.048886,  1.000000,  0.036261,  1.000000]
+  }
+}

cosmos_dreams_action_sources/action_schema.json

@@ -0,0 +1,95 @@
+{
+  "action_tokens_per_frame": 4,
+  "contract_sha256": "2c75f43813f507ab5d382d5d64c41955b3871ee164c093fa098c8429cb8547db",
+  "default_embodiment": "camera_pose",
+  "embodiments": {
+    "camera_pose": {
+      "domain_id": 2,
+      "layout": {
+        "delta_equation": "T_i^-1 @ T_{i+1}",
+        "fields": [
+          {
+            "name": "camera_translation",
+            "offset": 0,
+            "size": 3,
+            "unit": "meter"
+          },
+          {
+            "name": "camera_rotation",
+            "offset": 3,
+            "representation": "rot6d_columns",
+            "size": 6,
+            "unit": "dimensionless"
+          }
+        ],
+        "id": "camera_pose_backward_framewise_rot6d_v1",
+        "pose_convention": "backward_framewise",
+        "rotation_representation": "rot6d_columns"
+      },
+      "normalizer": {
+        "derivation": {
+          "rotation_scale": 1.0,
+          "translation_scale": 10.0
+        },
+        "method": "pose_scale",
+        "schema_version": 1,
+        "training_config": {
+          "experiment": "causal_8b_sf_dmd_max_4step_cam_chunk4_480p_961f",
+          "repository_revision": "483375f79279832b52d9e5c3598345a86f7137d5",
+          "resolved_sha256": "0e7b2be45c82b13c6f1454cdafd7e75d1941a6565e3b72b15a03a99d65177f29"
+        },
+        "transform": {
+          "forward_clamp": false,
+          "offset": [
+            0.0,
+            0.0,
+            0.0,
+            0.0,
+            0.0,
+            0.0,
+            0.0,
+            0.0,
+            0.0
+          ],
+          "scale": [
+            0.10000000149011612,
+            0.10000000149011612,
+            0.10000000149011612,
+            1.0,
+            1.0,
+            1.0,
+            1.0,
+            1.0,
+            1.0
+          ],
+          "type": "affine"
+        },
+        "transform_sha256": "bb8102c686b4813c1cdcc82b78cb174225fa35f8bfafd07d36b81a767bb6e698"
+      },
+      "raw_action_dim": 9
+    }
+  },
+  "model_action_dim": 64,
+  "num_embodiment_domains": 32,
+  "padding": {
+    "stage": "after_normalization",
+    "value": 0.0
+  },
+  "schema_version": 3,
+  "training_config_excerpt": {
+    "datasets": [
+      {
+        "apply_forward_clamp": false,
+        "dataset_class": "CameraDatasetSharded",
+        "embodiment": "camera_pose",
+        "method": "pose_scale",
+        "mode": "forward_dynamics",
+        "pose_convention": "backward_framewise",
+        "rotation_format": "rot6d",
+        "rotation_scale": 1.0,
+        "translation_scale": 10.0
+      }
+    ],
+    "experiment": "causal_8b_sf_dmd_max_4step_cam_chunk4_480p_961f"
+  }
+}

cosmos_dreams_action_sources/cosmos_dreams_action_schema.schema.json

@@ -0,0 +1,201 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://developer.nvidia.com/cosmos-dreams/action-schema-v3.schema.json",
+  "title": "Cosmos-Dreams Per-Embodiment Action Schema v3",
+  "type": "object",
+  "additionalProperties": false,
+  "required": ["schema_version", "action_tokens_per_frame", "model_action_dim", "num_embodiment_domains", "default_embodiment", "embodiments", "padding", "training_config_excerpt", "contract_sha256"],
+  "properties": {
+    "schema_version": {"const": 3},
+    "action_tokens_per_frame": {"const": 4},
+    "model_action_dim": {"const": 64},
+    "num_embodiment_domains": {"const": 32},
+    "default_embodiment": {"enum": ["abc_yam", "agibotworld", "agibot_gear_gripper", "agibot_gear_gripper_ext", "camera_pose", "molmoact2_yam", "xdof_yam"]},
+    "embodiments": {
+      "type": "object",
+      "minProperties": 1,
+      "additionalProperties": false,
+      "properties": {
+        "abc_yam": {"$ref": "#/$defs/yam_embodiment"},
+        "agibotworld": {"$ref": "#/$defs/agibot_embodiment"},
+        "agibot_gear_gripper": {"$ref": "#/$defs/agibot_embodiment"},
+        "agibot_gear_gripper_ext": {"$ref": "#/$defs/agibot_embodiment"},
+        "camera_pose": {"$ref": "#/$defs/camera_embodiment"},
+        "molmoact2_yam": {"$ref": "#/$defs/yam_embodiment"},
+        "xdof_yam": {"$ref": "#/$defs/yam_embodiment"}
+      }
+    },
+    "padding": {"const": {"stage": "after_normalization", "value": 0.0}},
+    "training_config_excerpt": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["datasets", "experiment"],
+      "properties": {
+        "datasets": {"type": "array", "minItems": 1, "items": {"oneOf": [{"$ref": "#/$defs/agibot_dataset_descriptor"}, {"$ref": "#/$defs/yam_dataset_descriptor"}, {"$ref": "#/$defs/camera_dataset_descriptor"}]}},
+        "experiment": {"type": "string", "minLength": 1}
+      }
+    },
+    "contract_sha256": {"$ref": "#/$defs/sha256"}
+  },
+  "$defs": {
+    "sha256": {"type": "string", "pattern": "^[0-9a-f]{64}$"},
+    "repository_revision": {"type": "string", "pattern": "^[0-9a-f]{40}$"},
+    "training_config": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["experiment", "resolved_sha256", "repository_revision"],
+      "properties": {
+        "experiment": {"type": "string", "minLength": 1},
+        "resolved_sha256": {"$ref": "#/$defs/sha256"},
+        "repository_revision": {"$ref": "#/$defs/repository_revision"}
+      }
+    },
+    "agibot_layout": {
+      "const": {"id": "agibot_backward_framewise_rot6d_v1", "pose_convention": "backward_framewise", "delta_equation": "T_i^-1 @ T_{i+1}", "rotation_representation": "rot6d_columns", "fields": [{"name": "head_translation", "offset": 0, "size": 3, "unit": "meter"}, {"name": "head_rotation", "offset": 3, "size": 6, "unit": "dimensionless", "representation": "rot6d_columns"}, {"name": "right_translation", "offset": 9, "size": 3, "unit": "meter"}, {"name": "right_rotation", "offset": 12, "size": 6, "unit": "dimensionless", "representation": "rot6d_columns"}, {"name": "right_gripper", "offset": 18, "size": 1, "unit": "open_fraction", "closed_value": 0.0, "open_value": 1.0}, {"name": "left_translation", "offset": 19, "size": 3, "unit": "meter"}, {"name": "left_rotation", "offset": 22, "size": 6, "unit": "dimensionless", "representation": "rot6d_columns"}, {"name": "left_gripper", "offset": 28, "size": 1, "unit": "open_fraction", "closed_value": 0.0, "open_value": 1.0}]}
+    },
+    "yam_layout": {
+      "const": {"id": "legacy_yam_fk_backward_framewise_rot6d_v1", "pose_convention": "backward_framewise", "delta_equation": "T_i^-1 @ T_{i+1}", "rotation_representation": "rot6d_columns", "fields": [{"name": "left_translation", "offset": 0, "size": 3, "unit": "meter"}, {"name": "left_rotation", "offset": 3, "size": 6, "unit": "dimensionless", "representation": "rot6d_columns"}, {"name": "left_gripper", "offset": 9, "size": 1, "unit": "open_fraction", "closed_value": 0.0, "open_value": 1.0}, {"name": "right_translation", "offset": 10, "size": 3, "unit": "meter"}, {"name": "right_rotation", "offset": 13, "size": 6, "unit": "dimensionless", "representation": "rot6d_columns"}, {"name": "right_gripper", "offset": 19, "size": 1, "unit": "open_fraction", "closed_value": 0.0, "open_value": 1.0}]}
+    },
+    "camera_layout": {
+      "const": {"id": "camera_pose_backward_framewise_rot6d_v1", "pose_convention": "backward_framewise", "delta_equation": "T_i^-1 @ T_{i+1}", "rotation_representation": "rot6d_columns", "fields": [{"name": "camera_translation", "offset": 0, "size": 3, "unit": "meter"}, {"name": "camera_rotation", "offset": 3, "size": 6, "unit": "dimensionless", "representation": "rot6d_columns"}]}
+    },
+    "agibot_embodiment": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["domain_id", "raw_action_dim", "layout", "normalizer"],
+      "properties": {"domain_id": {"const": 15}, "raw_action_dim": {"const": 29}, "layout": {"$ref": "#/$defs/agibot_layout"}, "normalizer": {"$ref": "#/$defs/quantile_rot"}}
+    },
+    "yam_embodiment": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["domain_id", "raw_action_dim", "layout", "normalizer"],
+      "properties": {"domain_id": {"const": 16}, "raw_action_dim": {"const": 20}, "layout": {"$ref": "#/$defs/yam_layout"}, "normalizer": {"$ref": "#/$defs/yam_quantile_rot"}}
+    },
+    "camera_embodiment": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["domain_id", "raw_action_dim", "layout", "normalizer"],
+      "properties": {"domain_id": {"const": 2}, "raw_action_dim": {"const": 9}, "layout": {"$ref": "#/$defs/camera_layout"}, "normalizer": {"$ref": "#/$defs/pose_scale"}}
+    },
+    "agibot_dataset_descriptor": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["dataset_class", "embodiment", "method", "apply_forward_clamp", "pose_convention", "rotation_format", "stats_filename"],
+      "properties": {
+        "dataset_class": {"enum": ["AgiBotWorldBetaDataset", "AgibotGEARGripperDataset", "AgibotGEARGripperExtDataset"]},
+        "embodiment": {"enum": ["agibotworld", "agibot_gear_gripper", "agibot_gear_gripper_ext"]},
+        "method": {"const": "quantile_rot"},
+        "apply_forward_clamp": {"const": false},
+        "pose_convention": {"const": "backward_framewise"},
+        "rotation_format": {"const": "rot6d"},
+        "stats_filename": {"enum": ["agibot_backward_framewise_rot6d.json", "agibot_gear_gripper_backward_framewise_rot6d.json"]}
+      },
+      "allOf": [
+        {"if": {"properties": {"dataset_class": {"const": "AgiBotWorldBetaDataset"}}}, "then": {"properties": {"embodiment": {"const": "agibotworld"}, "stats_filename": {"const": "agibot_backward_framewise_rot6d.json"}}}},
+        {"if": {"properties": {"dataset_class": {"const": "AgibotGEARGripperDataset"}}}, "then": {"properties": {"embodiment": {"const": "agibot_gear_gripper"}}}},
+        {"if": {"properties": {"dataset_class": {"const": "AgibotGEARGripperExtDataset"}}}, "then": {"properties": {"embodiment": {"const": "agibot_gear_gripper_ext"}}}}
+      ]
+    },
+    "yam_dataset_descriptor": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["dataset_class", "embodiment", "method", "apply_forward_clamp", "pose_convention", "rotation_format", "stats_filename"],
+      "properties": {
+        "dataset_class": {"enum": ["ABCYAMLeRobotDataset", "MolmoAct2YAMDataset", "XDOFYAMDataset"]},
+        "embodiment": {"enum": ["abc_yam", "molmoact2_yam", "xdof_yam"]},
+        "method": {"const": "quantile_rot"},
+        "apply_forward_clamp": {"const": false},
+        "pose_convention": {"const": "backward_framewise"},
+        "rotation_format": {"const": "rot6d"},
+        "stats_filename": {"enum": ["abc_yam_backward_framewise_rot6d.json", "molmoact2_yam_backward_framewise_rot6d.json", "xdof_yam_backward_framewise_rot6d.json"]}
+      },
+      "allOf": [
+        {"if": {"properties": {"dataset_class": {"const": "ABCYAMLeRobotDataset"}}}, "then": {"properties": {"embodiment": {"const": "abc_yam"}, "stats_filename": {"const": "abc_yam_backward_framewise_rot6d.json"}}}},
+        {"if": {"properties": {"dataset_class": {"const": "MolmoAct2YAMDataset"}}}, "then": {"properties": {"embodiment": {"const": "molmoact2_yam"}, "stats_filename": {"const": "molmoact2_yam_backward_framewise_rot6d.json"}}}},
+        {"if": {"properties": {"dataset_class": {"const": "XDOFYAMDataset"}}}, "then": {"properties": {"embodiment": {"const": "xdof_yam"}, "stats_filename": {"const": "xdof_yam_backward_framewise_rot6d.json"}}}}
+      ]
+    },
+    "camera_dataset_descriptor": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["dataset_class", "embodiment", "method", "apply_forward_clamp", "mode", "pose_convention", "rotation_format", "translation_scale", "rotation_scale"],
+      "properties": {
+        "dataset_class": {"const": "CameraDatasetSharded"},
+        "embodiment": {"const": "camera_pose"},
+        "method": {"const": "pose_scale"},
+        "apply_forward_clamp": {"const": false},
+        "mode": {"const": "forward_dynamics"},
+        "pose_convention": {"const": "backward_framewise"},
+        "rotation_format": {"const": "rot6d"},
+        "translation_scale": {"type": "number", "exclusiveMinimum": 0},
+        "rotation_scale": {"type": "number", "exclusiveMinimum": 0}
+      }
+    },
+    "quantile_rot": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["schema_version", "method", "transform", "derivation", "source", "training_config", "transform_sha256"],
+      "properties": {
+        "schema_version": {"const": 1},
+        "method": {"const": "quantile_rot"},
+        "transform": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": ["type", "offset", "scale", "forward_clamp"],
+          "properties": {"type": {"const": "affine"}, "offset": {"type": "array", "minItems": 29, "maxItems": 29, "items": {"type": "number"}}, "scale": {"type": "array", "minItems": 29, "maxItems": 29, "items": {"type": "number", "exclusiveMinimum": 0}}, "forward_clamp": {"const": false}}
+        },
+        "derivation": {"type": "object", "additionalProperties": false, "required": ["statistics_block", "low_key", "high_key", "range_floor"], "properties": {"statistics_block": {"const": "global_raw"}, "low_key": {"const": "q01"}, "high_key": {"const": "q99"}, "range_floor": {"const": 9.99999993922529e-9}}},
+        "source": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": ["path", "artifact_path", "sha256", "repository_revision"],
+          "properties": {"path": {"enum": ["projects/cosmos3/cosmos3/datasets/action/normalizers/agibot_backward_framewise_rot6d.json", "projects/cosmos3/cosmos3/datasets/action/normalizers/agibot_gear_gripper_backward_framewise_rot6d.json"]}, "artifact_path": {"type": "string", "pattern": "^cosmos_dreams_action_sources/[0-9a-f]{64}\\.json$"}, "sha256": {"$ref": "#/$defs/sha256"}, "repository_revision": {"$ref": "#/$defs/repository_revision"}}
+        },
+        "training_config": {"$ref": "#/$defs/training_config"},
+        "transform_sha256": {"$ref": "#/$defs/sha256"}
+      }
+    },
+    "yam_quantile_rot": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["schema_version", "method", "transform", "derivation", "source", "training_config", "transform_sha256"],
+      "properties": {
+        "schema_version": {"const": 1},
+        "method": {"const": "quantile_rot"},
+        "transform": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": ["type", "offset", "scale", "forward_clamp"],
+          "properties": {"type": {"const": "affine"}, "offset": {"type": "array", "minItems": 20, "maxItems": 20, "items": {"type": "number"}}, "scale": {"type": "array", "minItems": 20, "maxItems": 20, "items": {"type": "number", "exclusiveMinimum": 0}}, "forward_clamp": {"const": false}}
+        },
+        "derivation": {"type": "object", "additionalProperties": false, "required": ["statistics_block", "low_key", "high_key", "range_floor"], "properties": {"statistics_block": {"const": "global_raw"}, "low_key": {"const": "q01"}, "high_key": {"const": "q99"}, "range_floor": {"const": 9.99999993922529e-9}}},
+        "source": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": ["path", "artifact_path", "sha256", "repository_revision"],
+          "properties": {"path": {"enum": ["projects/cosmos3/cosmos3/datasets/action/legacy/normalizers/abc_yam_backward_framewise_rot6d.json", "projects/cosmos3/cosmos3/datasets/action/legacy/normalizers/molmoact2_yam_backward_framewise_rot6d.json", "projects/cosmos3/cosmos3/datasets/action/legacy/normalizers/xdof_yam_backward_framewise_rot6d.json"]}, "artifact_path": {"type": "string", "pattern": "^cosmos_dreams_action_sources/[0-9a-f]{64}\\.json$"}, "sha256": {"$ref": "#/$defs/sha256"}, "repository_revision": {"$ref": "#/$defs/repository_revision"}}
+        },
+        "training_config": {"$ref": "#/$defs/training_config"},
+        "transform_sha256": {"$ref": "#/$defs/sha256"}
+      }
+    },
+    "pose_scale": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["schema_version", "method", "transform", "derivation", "training_config", "transform_sha256"],
+      "properties": {
+        "schema_version": {"const": 1},
+        "method": {"const": "pose_scale"},
+        "transform": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": ["type", "offset", "scale", "forward_clamp"],
+          "properties": {"type": {"const": "affine"}, "offset": {"type": "array", "minItems": 9, "maxItems": 9, "items": {"const": 0.0}}, "scale": {"type": "array", "minItems": 9, "maxItems": 9, "items": {"type": "number", "exclusiveMinimum": 0}}, "forward_clamp": {"const": false}}
+        },
+        "derivation": {"type": "object", "additionalProperties": false, "required": ["translation_scale", "rotation_scale"], "properties": {"translation_scale": {"type": "number", "exclusiveMinimum": 0}, "rotation_scale": {"type": "number", "exclusiveMinimum": 0}}},
+        "training_config": {"$ref": "#/$defs/training_config"},
+        "transform_sha256": {"$ref": "#/$defs/sha256"}
+      }
+    }
+  }
+}

cosmos_dreams_action_sources/resolved_action_config.json

@@ -0,0 +1,16 @@
+{
+  "datasets": [
+    {
+      "apply_forward_clamp": false,
+      "dataset_class": "CameraDatasetSharded",
+      "embodiment": "camera_pose",
+      "method": "pose_scale",
+      "mode": "forward_dynamics",
+      "pose_convention": "backward_framewise",
+      "rotation_format": "rot6d",
+      "rotation_scale": 1.0,
+      "translation_scale": 10.0
+    }
+  ],
+  "experiment": "causal_8b_sf_dmd_max_4step_cam_chunk4_480p_961f"
+}

cosmos_dreams_manifest.json

@@ -0,0 +1,131 @@
+{
+  "attention_mode": "three_way",
+  "checkpoint_hash": "9540afba2ac43d36b213a847f227b8011f5419feb670be722b518debdc22f63e",
+  "checkpoint_id": "causal_8b_sf_dmd_max_4step_cam_chunk4_480p_961f@iter_000002000",
+  "checkpoint_iteration": 2000,
+  "chunk_partition": [
+    1,
+    4
+  ],
+  "chunk_size": 4,
+  "conditioning": {
+    "action_tokens_per_frame": 4,
+    "contract_sha256": "2c75f43813f507ab5d382d5d64c41955b3871ee164c093fa098c8429cb8547db",
+    "default_embodiment": "camera_pose",
+    "embodiments": {
+      "camera_pose": {
+        "domain_id": 2,
+        "layout": {
+          "delta_equation": "T_i^-1 @ T_{i+1}",
+          "fields": [
+            {
+              "name": "camera_translation",
+              "offset": 0,
+              "size": 3,
+              "unit": "meter"
+            },
+            {
+              "name": "camera_rotation",
+              "offset": 3,
+              "representation": "rot6d_columns",
+              "size": 6,
+              "unit": "dimensionless"
+            }
+          ],
+          "id": "camera_pose_backward_framewise_rot6d_v1",
+          "pose_convention": "backward_framewise",
+          "rotation_representation": "rot6d_columns"
+        },
+        "normalizer": {
+          "derivation": {
+            "rotation_scale": 1.0,
+            "translation_scale": 10.0
+          },
+          "method": "pose_scale",
+          "schema_version": 1,
+          "training_config": {
+            "experiment": "causal_8b_sf_dmd_max_4step_cam_chunk4_480p_961f",
+            "repository_revision": "483375f79279832b52d9e5c3598345a86f7137d5",
+            "resolved_sha256": "0e7b2be45c82b13c6f1454cdafd7e75d1941a6565e3b72b15a03a99d65177f29"
+          },
+          "transform": {
+            "forward_clamp": false,
+            "offset": [
+              0.0,
+              0.0,
+              0.0,
+              0.0,
+              0.0,
+              0.0,
+              0.0,
+              0.0,
+              0.0
+            ],
+            "scale": [
+              0.10000000149011612,
+              0.10000000149011612,
+              0.10000000149011612,
+              1.0,
+              1.0,
+              1.0,
+              1.0,
+              1.0,
+              1.0
+            ],
+            "type": "affine"
+          },
+          "transform_sha256": "bb8102c686b4813c1cdcc82b78cb174225fa35f8bfafd07d36b81a767bb6e698"
+        },
+        "raw_action_dim": 9
+      }
+    },
+    "mode": "action",
+    "model_action_dim": 64,
+    "num_embodiment_domains": 32,
+    "padding": {
+      "stage": "after_normalization",
+      "value": 0.0
+    },
+    "schema_version": 3,
+    "training_config_excerpt": {
+      "datasets": [
+        {
+          "apply_forward_clamp": false,
+          "dataset_class": "CameraDatasetSharded",
+          "embodiment": "camera_pose",
+          "method": "pose_scale",
+          "mode": "forward_dynamics",
+          "pose_convention": "backward_framewise",
+          "rotation_format": "rot6d",
+          "rotation_scale": 1.0,
+          "translation_scale": 10.0
+        }
+      ],
+      "experiment": "causal_8b_sf_dmd_max_4step_cam_chunk4_480p_961f"
+    }
+  },
+  "fixed_step_sampler_config": {
+    "num_train_timesteps": 1000,
+    "sample_type": "sde",
+    "t_list": [
+      1.0,
+      0.9375,
+      0.8333333333333334,
+      0.625
+    ]
+  },
+  "latent_patch_size": 2,
+  "mrope": {
+    "base_fps": 24.0,
+    "enable_fps_modulation": true,
+    "reset_spatial_ids": true,
+    "temporal_modality_margin": 15000
+  },
+  "schema_version": 1,
+  "sink_frames": 0,
+  "temporal_compression_factor": 4,
+  "text_cache_max_len": 512,
+  "vae_spatial_compression_factor": 16,
+  "video_temporal_causal": true,
+  "window_frames": 96
+}

examples/inference/README.md

@@ -0,0 +1,96 @@
+# Sim-Bimanual inference examples
+
+These manifests provide 30 FPS examples for the `normalized_unified_v1`
+action contract used by Cosmos3 Nano Sim Bimanual. Each row contains an
+initial image, a plain-text prompt, and normalized 59D action rows. The
+30-second examples use 900 actions for 901 frames; the 60-second AgiBot
+example uses 1800 actions for 1801 frames. The initial image is frame 0 of
+its source sequence; target future frames are not included.
+
+Image and action paths use Hugging Face `/resolve/main/` URLs. Keep `HF_TOKEN`
+available to the Framework inference process when the gated repository
+requires authentication. Store the token outside the checkout and do not put
+its value in manifests, command arguments, or logs.
+
+| Manifest | Samples | Action routing |
+| --- | --- | --- |
+| `sim_bimanual_unifiedv1_hf.jsonl` | Five AgiBot and two YAM trajectories | Scalar or per-row unified-action domains |
+| `sim_bimanual_mixed_action_hf.jsonl` | Two AgiBot examples | Slow camera loop at rows 300-599 plus source camera masking in the surrounding robot spans |
+| `sim_bimanual_agibot_long60_hf.jsonl` | One 60-second AgiBot-scene example | All 1800 rows use `agibotworld` (domain 15) with nonzero, varying robot actions; the source 900-row program is repeated twice |
+
+The mixed-action rows are `agibot_mixed_007` and `agibot_mixed_028`. Their
+first and last 300-row spans preserve the source production camera masking:
+stationary robot rows use `camera_pose` (domain 2), retain the first nine
+camera values, and zero robot-only slots 9-58. The middle 300 rows retain the
+slow camera loop. The complete sidecars contain 385 camera rows for
+`agibot_mixed_007` and 402 for `agibot_mixed_028`; all other rows retain their
+original AgiBot actions.
+
+Every PNG stores the unpadded content region. Framework reconstructs the model
+canvas and spatial metadata during preprocessing.
+
+## Run inference
+
+From a current [Cosmos Framework](https://github.com/nvidia/cosmos-framework)
+checkout with the inference environment activated:
+
+```bash
+export COSMOS_TRAINING=0
+CHECKPOINT_DIR=/path/to/Cosmos3-Nano-Sim-Bimanual
+INPUT_MANIFEST="$CHECKPOINT_DIR/examples/inference/sim_bimanual_unifiedv1_hf.jsonl"
+OUTPUT_DIR=outputs/sim_bimanual_examples
+
+torchrun --standalone --nproc-per-node=1 \
+  -m cosmos_framework.scripts.inference \
+  -i "$INPUT_MANIFEST" -o "$OUTPUT_DIR" \
+  --checkpoint-path "$CHECKPOINT_DIR" \
+  --model-mode=FORWARD_DYNAMICS --resolution=480 --image-size=480 \
+  --num-steps=4 --seed=2026 \
+  --autoregressive=True --kv-cache-inference-size=64 --attention-sink-size=8 \
+  --parallelism-preset=latency --max-num-seqs=1 \
+  --no-use-torch-compile --no-diffusion-cache --no-guardrails \
+  --video-save-quality=4
+```
+
+Select another manifest by changing `INPUT_MANIFEST` and using a new output
+directory. Run one active sequence at a time with `--max-num-seqs=1`.
+The checkpoint stores the four-step SDE schedule
+`[1.0, 0.9375, 0.8333333333333334, 0.625]`; keep `--num-steps=4`.
+
+## 60-second AgiBot example
+
+`agibot_original_long60_019_bulk_seed2026` is included in the combined manifest
+and in `sim_bimanual_agibot_long60_hf.jsonl` for a focused run. It uses the exact
+initial observation and production v3 prompt from the selected website case.
+Its 1800 normalized action rows repeat the audited 900-row source program
+twice. All rows use `agibotworld` (domain 15), with nonzero, varying wrist and
+gripper actions preserved from the frozen production input. No camera-only
+substitution, renormalization or prompt rewriting is applied during export.
+This is an open-loop action-conditioned rollout, not a new 60-second source
+episode or a closed-loop policy evaluation.
+
+To run this example alone, select:
+
+```bash
+INPUT_MANIFEST="$CHECKPOINT_DIR/examples/inference/sim_bimanual_agibot_long60_hf.jsonl"
+OUTPUT_DIR=outputs/sim_bimanual_agibot_long60
+```
+
+Then run the command above. It reads `action_chunk_size` and `num_frames`
+from each manifest row, allowing 901- and 1801-frame examples in the same
+manifest. Do not override these with `--action-chunk-size=900 --num-frames=901`
+when running the 60-second example. If setting them explicitly for the focused
+manifest, use `--action-chunk-size=1800 --num-frames=1801`.
+
+## Optional frustum history retrieval
+
+`bimanual_frustum.json` is a reusable KV-history configuration. Copy a
+manifest row and add a `kv_cache_frustum_path` field pointing to that file.
+Replace its estimated 90-degree horizontal FOV with measured camera
+intrinsics when available. Run frustum retrieval with:
+
+```bash
+--no-use-torch-compile --kv-cache-inference-size=0 --attention-sink-size=0
+```
+
+This mode uses an unbounded physical KV archive.

examples/inference/bimanual_frustum.json

@@ -0,0 +1,14 @@
+{
+  "pose_source": "bimanual_unified_v1_chunk16",
+  "image_size": [
+    832,
+    480
+  ],
+  "fps": 30,
+  "horizontal_fov_degrees": 90.0,
+  "near": 0.1,
+  "far": 10.0,
+  "sink_size": 8,
+  "retrieved_size": 32,
+  "recent_size": 32
+}

examples/inference/sim_bimanual_agibot_long60_hf.jsonl

@@ -0,0 +1 @@
+{"name":"agibot_original_long60_019_bulk_seed2026","model_mode":"forward_dynamics","prompt":"A landscape, high oblique overhead medium-wide shot of an indoor tabletop robotics station, sharply focused. A white rectangular table with rounded corners runs diagonally through most of the frame over a pale beige tiled floor; a pale green wall or partition stands behind it, a tan perforated pegboard with evenly spaced dark holes is cropped along frame-left, and a partial light gray chair back sits beyond the table in the upper-right background. A matte black vase with green leafy stems and small pale blossoms rests near the upper center of the tabletop. On the right side of the table, a crumpled pink cloth with a small white tag bearing unreadable markings lies below and slightly right of several brown liquid spots, while a small crumpled translucent paper or tissue rests nearer the middle-right area. A black open trash bin lined with a black plastic bag sits below the table’s left-front edge in the lower-left/center foreground, partly under the tabletop, with discarded items already visible inside, including a clear plastic cup with a lid and straw and another container-like item with unreadable labeling. Soft, diffuse indoor lighting, mild shadows, moderate contrast, and a muted palette of white, black, pale green, beige, pink, brown, and plant green give the setting a controlled laboratory demonstration mood. A left robotic arm, cropped by frame-left, has a white cylindrical segment, black wrist and gripper assembly, visible wiring, a small bracket, and a cyan-green illuminated ring; its gripper points toward the bin rim and remains close to it. A larger right robotic arm, cropped by frame-right, has a white arm body, black cable loops, metal fasteners, a small rectangular wrist-mounted module, a black multi-finger gripper, and a cyan-green illuminated ring. The right arm occupies the right half and appears larger and nearer than the left arm; the left arm occupies the left third. The right gripper holds a clear plastic cup with a white lid and a tan straw protruding upward, with light residue or liquid visible at the bottom.","vision_path":"https://huggingface.co/nvidia/Cosmos3-Nano-Sim-Bimanual/resolve/main/assets/sim_bimanual/agibot/images/agibot_original_long60_019_initial.png","domain_name":"agibotworld","action_condition":true,"raw_action_dim":59,"resolution":"480","aspect_ratio":"16,9","fps":30,"num_frames":1801,"action_chunk_size":1800,"num_steps":4,"guidance":3.0,"shift":5.0,"seed":2026,"autoregressive":true,"action_path":"https://huggingface.co/nvidia/Cosmos3-Nano-Sim-Bimanual/resolve/main/assets/sim_bimanual/agibot/actions/agibot_original_long60_019_normalized_unifiedv1.json","format_prompt_as_json":false}

examples/inference/sim_bimanual_mixed_action_hf.jsonl

@@ -0,0 +1,2 @@
+{"name":"agibot_mixed_007_bulk_seed2026","model_mode":"forward_dynamics","prompt":"The scene opens directly into a high-overhead landscape shot centered on a round white marble-pattern table with subtle gray veining and a black circular rim, sharply focused under bright, even overhead lighting that casts soft shadows and gives the clean indoor setup a controlled demonstration-lab mood. Gray-brown wood-look flooring surrounds the table; dark brown cabinet fronts line the upper background, with an open recess beneath one cabinet showing loose cables; a tall dark vertical appliance or cabinet face stands at frame-right; a pale wall or vertical panel borders frame-left; and a small brown bag-like object rests on the floor near the left side of the table, partly hidden by the left robot. A clear glass vase at the back/top-center of the tabletop holds cream-white flowers, green leaves, and small orange-red sprigs. A dark blue ribbed cup with a light interior and dark handle sits near the lower-left/center of the table, a folded deep-blue rectangular cloth lies on the right half, and a small beige oval stain sits below the cloth in the lower-right quadrant. Two large industrial-style robotic arms flank the table: the left arm is cropped by the left edge and has a white/silver structural segment, black cylindrical wrist housing, black articulated opposing gripper fingers, exposed pivots and screws, black cables, and a small yellow connector; the right arm is cropped by the right edge and has a white/silver structural segment, black wrist and gripper assembly, black cable loops, small blue/green indicator lights, and a small yellow triangular marking. Small physical markings on the robot housings are not clearly readable. The table fills most of the central frame, the vase stands behind the cup and cloth, the cup is smaller than either gripper but larger than the beige stain, and both robots face inward across the tabletop without touching each other.","vision_path":"https://huggingface.co/nvidia/Cosmos3-Nano-Sim-Bimanual/resolve/main/assets/sim_bimanual/agibot/images/agibot_mixed_007_initial.png","domain_name":"agibotworld","action_condition":true,"raw_action_dim":59,"resolution":"480","aspect_ratio":"16,9","fps":30,"num_frames":901,"action_chunk_size":900,"num_steps":4,"guidance":3.0,"shift":5.0,"seed":2026,"autoregressive":true,"action_path":"https://huggingface.co/nvidia/Cosmos3-Nano-Sim-Bimanual/resolve/main/assets/sim_bimanual/agibot/actions/agibot_mixed_007_normalized_mixed_action.json","format_prompt_as_json":false}
+{"name":"agibot_mixed_028_bulk_seed2026","model_mode":"forward_dynamics","prompt":"The scene opens in a continuous landscape shot from a high oblique overhead angle, sharply focused across a glossy white kitchen countertop with faint gray streaking, soft reflections, and diffuse cool-neutral indoor light that casts gentle moving shadows and gives the workspace a clinical, controlled demonstration quality. A stainless-steel sink basin with a rounded rectangular rim, inner metal bowl, drain area, and a cropped vertical faucet assembly occupies frame-left and bottom-left; a pink textured scrub pad rests near the back-left corner beside it. A pale gray-white marble-pattern backsplash spans the rear, with a white-framed dark reflective window in the upper-left background showing indistinct reflected or exterior shapes. Along the rear counter sit a white pump bottle with an orange pump and a red physical label reading “SODA,” a small yellow rectangular container with a green-and-white label whose smaller text is unreadable, and two tall clear bottles of amber liquid, one capped yellow and one capped brown, with colored labels containing unreadable non-Latin characters. A white appliance or cabinet face with a dark inset panel is cropped at far frame-right. Centered between the arms, a soft mint-green terry-like cloth with stitched edging and a small white tag lies folded into overlapping triangular layers, occupying roughly one fifth of the frame. A left robotic arm, partially cropped by the left edge, extends from the sink side with a white cylindrical segment, silver joint ring, glowing blue circular indicator, black looped cabling, gray wrist module, black cylindrical wrist, and black angular multi-link pincer gripper bearing small unreadable white markings. A matching right robotic arm, partially cropped by the right edge, has white and silver sections, black cabling, gray wrist module, black cylindrical wrist, black articulated pincer gripper, small yellow cable ties or connectors, and glowing blue-green indicator lights. The arms fill the lower-left and lower-right portions of the frame, balance the composition, and face inward toward each other and the cloth; the rear bottles sit behind the cloth along the backsplash, while the sink lies nearer the left foreground under the left arm.","vision_path":"https://huggingface.co/nvidia/Cosmos3-Nano-Sim-Bimanual/resolve/main/assets/sim_bimanual/agibot/images/agibot_mixed_028_initial.png","domain_name":"agibotworld","action_condition":true,"raw_action_dim":59,"resolution":"480","aspect_ratio":"16,9","fps":30,"num_frames":901,"action_chunk_size":900,"num_steps":4,"guidance":3.0,"shift":5.0,"seed":2026,"autoregressive":true,"action_path":"https://huggingface.co/nvidia/Cosmos3-Nano-Sim-Bimanual/resolve/main/assets/sim_bimanual/agibot/actions/agibot_mixed_028_normalized_mixed_action.json","format_prompt_as_json":false}