Make the WorldDiT model card demo first

README.md

@@ -12,11 +12,9 @@ tags:
   - robotic-manipulation
   - imitation-learning
   - diffusion-transformer
+  - diffusion-policy
   - flow-matching
-  - pareto-frontier
-  - parameter-efficiency
 inference: false
-license: cc-by-4.0
 widget:
   - example_title: "LIBERO Spatial, task 5"
     text: "Successful rollout, front view."
@@ -40,96 +38,91 @@ widget:
   <img src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/paris2/model-card/v1/bagel_labs_logo.png" alt="Bagel Labs">
 </p>
 
-<h1 align="center">WorldDiT: A Unified Diffusion Backbone for<br>World and Action Modeling</h1>
+# WorldDiT
+
+## One diffusion backbone learns what to do and what comes next.
 
 <p align="center">
-  <a href="https://arxiv.org/abs/2607.23909">
-    <img src="https://img.shields.io/badge/arXiv-2607.23909-B31B1B.svg?logo=arxiv&logoColor=white" alt="WorldDiT paper on arXiv">
+  <a href="https://huggingface.co/bageldotcom/worlddit" target="_blank">
+    <img src="https://img.shields.io/badge/🤗_DOWNLOAD_WORLDDIT_WEIGHTS-FFD21E?style=for-the-badge&logoColor=000000" alt="Download WorldDiT Weights">
+  </a>
+  <a href="https://github.com/Lifelong-Robot-Learning/LIBERO" target="_blank">
+    <img src="https://img.shields.io/badge/🤖_LIBERO_BENCHMARK-FF6B6B?style=for-the-badge&logoColor=white" alt="LIBERO Benchmark">
   </a>
 </p>
 
-<p align="center">
-WorldDiT couples continuous action generation with auxiliary future normalized
-RGB patch prediction in one diffusion transformer. The architecture is designed
-as a general backbone for world and action modeling, while the current release
-evaluates it on LIBERO and provides four checkpoints, a self contained
-inference runtime, and an evaluator.
-</p>
+WorldDiT learns continuous robot action chunks and a future visual target
+through one shared diffusion transformer. Deployment keeps only the action
+path.
+
+This release includes four LIBERO checkpoints, a self contained inference
+runtime, and an evaluator for reproducing the reported suite results.
 
 ## See WorldDiT act
 
 The four clips below show successful rollouts from the released checkpoints.
 Each clip covers a different LIBERO suite and camera view.
 
-<table>
-  <tr>
-    <td width="50%" valign="top">
-      <video width="100%" controls muted loop playsinline preload="metadata"
-        src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_spatial_frontview_task05_episode01.mp4"></video>
-      <strong>LIBERO Spatial</strong><br>
-      Task 5, front view.
-    </td>
-    <td width="50%" valign="top">
-      <video width="100%" controls muted loop playsinline preload="metadata"
-        src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_object_agentview_task08_episode01.mp4"></video>
-      <strong>LIBERO Object</strong><br>
-      Task 8, agent view.
-    </td>
-  </tr>
-  <tr>
-    <td width="50%" valign="top">
-      <video width="100%" controls muted loop playsinline preload="metadata"
-        src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_goal_sideview_task10_episode01.mp4"></video>
-      <strong>LIBERO Goal</strong><br>
-      Task 10, side view.
-    </td>
-    <td width="50%" valign="top">
-      <video width="100%" controls muted loop playsinline preload="metadata"
-        src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_10_frontview_task06_episode01.mp4"></video>
-      <strong>LIBERO Long</strong><br>
-      Task 6, front view.
-    </td>
-  </tr>
-</table>
-
-[Spatial MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_spatial_frontview_task05_episode01.mp4)
-· [Object MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_object_agentview_task08_episode01.mp4)
-· [Goal MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_goal_sideview_task10_episode01.mp4)
-· [Long MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_10_frontview_task06_episode01.mp4)
-
-## Release snapshot
-
-| Reported LIBERO result | Released model |
-|---|---|
-| **94.9 percent** mean success<br>**1,898 of 2,000** successful episodes | **399.084 million** total parameters<br>**135.107 million** trainable parameters |
-| **98.0 percent** Spatial<br>**97.0 percent** Object | **Three** observation steps<br>**Seven** predicted actions |
-| **92.8 percent** Goal<br>**91.8 percent** Long | **Three** actions executed before replanning<br>**Seven** action dimensions |
+<Gallery />
 
-| Checkpoints | Runtime | Encoders and environment |
+| Suite | View | Video |
 |---|---|---|
-| Spatial<br>Object<br>Goal<br>Long | `inference.py`<br>`eval.py`<br>`config.json` | MAE ViT B<br>OpenAI CLIP ViT B 32<br>SafeTensors and pinned requirements |
+| LIBERO Spatial | Front view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_spatial_frontview_task05_episode01.mp4) |
+| LIBERO Object | Agent view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_object_agentview_task08_episode01.mp4) |
+| LIBERO Goal | Side view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_goal_sideview_task10_episode01.mp4) |
+| LIBERO Long | Front view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_10_frontview_task06_episode01.mp4) |
 
-The repository is self contained for WorldDiT inference. LIBERO provides the
-benchmark environments, assets, task definitions, and initial states.
+## What is in this release
 
-The released runtime and checkpoints were revalidated from a clean installation
-on eight RTX Pro 6000 Blackwell GPUs. The reported aggregate covers five
-hundred simulator episodes per suite. Three hundred episodes per suite informed
-staged checkpoint selection, while two hundred episodes per suite were disjoint
-from selection.
+| Release component | Included artifact |
+|---|---|
+| LIBERO Spatial policy | SafeTensors checkpoint |
+| LIBERO Object policy | SafeTensors checkpoint |
+| LIBERO Goal policy | SafeTensors checkpoint |
+| LIBERO Long policy | SafeTensors checkpoint |
+| Model runtime | `inference.py` |
+| Evaluation runtime | `eval.py` |
+| Frozen encoders | CLIP ViT B 32 and MAE ViT B |
+| Configuration | `config.json` |
+| Environment | Pinned Python requirements |
+
+The repository is self contained for WorldDiT inference. LIBERO still provides
+the benchmark environments, assets, task definitions, and initial states.
+
+## Reported LIBERO results
+
+Across the four released suite checkpoints, WorldDiT records 1,898 successful
+episodes out of 2,000 under the selection aware evaluation protocol.
+
+| Suite | Successful episodes | Success rate |
+|---|---:|---:|
+| LIBERO Spatial | 490 of 500 | 98.0 percent |
+| LIBERO Object | 485 of 500 | 97.0 percent |
+| LIBERO Goal | 464 of 500 | 92.8 percent |
+| LIBERO Long | 459 of 500 | 91.8 percent |
+| Selection aware mean | 1,898 of 2,000 | 94.9 percent |
 
-### Parameter count and reported success
+The released runtime and checkpoints were revalidated from a clean installation
+on eight RTX Pro 6000 Blackwell GPUs.
 
-Among methods with complete four suite averages, WorldDiT lies on the reported
-Pareto frontier for total model parameters and mean LIBERO success.
+The result is selection aware because three hundred episodes per suite informed
+staged checkpoint selection before the final five hundred episode score was
+assembled.
 
-<p align="center">
-  <img src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v2/worlddit_parameter_efficiency.png" alt="Reported mean LIBERO success against total instantiated model parameters">
-</p>
+## Model at a glance
 
-<p align="center">
-  <sub>Reported LIBERO success against total model parameters for 24 methods. The line connects methods on the Pareto frontier with complete four suite averages. Because the methods follow different published evaluation protocols, the figure summarizes published results rather than a direct comparison under one common evaluation protocol.</sub>
-</p>
+| Property | Released configuration |
+|---|---|
+| Total parameters | 399.084 million |
+| Trainable parameters | 135.107 million |
+| Observation context | Three frames |
+| Predicted action horizon | Seven actions |
+| Executed before replanning | Three actions |
+| Action dimension | Seven |
+| Visual encoder | MAE ViT B |
+| Language encoder | OpenAI CLIP ViT B 32 |
+| Checkpoint format | SafeTensors |
+| Evaluation environment | Headless LIBERO with EGL |
 
 ## Run a smoke test
 
@@ -166,40 +159,27 @@ python eval.py \
 ```
 
 A successful smoke test confirms that the environment, checkpoint, visual
-encoders, simulator, and rendering path load together. Full benchmark reporting
-uses complete suite evaluations.
+encoders, simulator, and rendering path load together. It is not a benchmark
+result.
 
 ## How WorldDiT works
 
-Each of three recent observation steps contributes primary and wrist images
-together with robot state, while one language instruction conditions the
-sequence. During training, one diffusion transformer backbone learns a seven
-step action chunk together with an auxiliary future normalized RGB patch
-target. At deployment, the encoded history conditions the action velocity field
-directly. RGB patch token construction and RGB prediction head evaluation
-remain outside the inference graph, concentrating computation on action
-generation. The controller executes the first three predicted actions, observes
-again, and replans.
+WorldDiT uses three recent observations, robot state, and language as context.
+During training, one diffusion transformer learns a seven step action chunk and
+an auxiliary future visual target. During deployment, the future visual path is
+absent. The policy executes the first three predicted actions, observes again,
+and replans.
 
-<p align="center">
-  <img src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v2/worlddit_inference_pipeline.png?v=09d6508" alt="WorldDiT inference pipeline">
-</p>
-
-<p align="center">
-  <sub>WorldDiT inference pipeline. The encoded observation history conditions action generation through twenty flow steps. The controller executes the first three actions from each seven action chunk, then updates the window and replans.</sub>
-</p>
+> Future visual prediction is a training signal, not a deployment path.
 
 | Training | Deployment |
 |---|---|
-| Action and future normalized RGB patch targets are learned by one backbone | Encoded history conditions the action velocity field |
+| Action and future visual targets share one backbone | Only the action path remains |
 | Seven action steps are supervised | Seven actions are predicted |
-| Future normalized RGB patch supervision is present | RGB patch tokens and the RGB prediction head remain outside the inference graph |
+| Future visual supervision is present | No future visual output is requested |
 | The complete training objective is active | Three actions execute before replanning |
 
-## Reference
-
-<details>
-<summary><strong>Full evaluation commands</strong></summary>
+## Evaluation
 
 ### One GPU
 
@@ -221,8 +201,8 @@ CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python eval.py \
 
 Each GPU receives an independent progress bar. After all workers finish, rank 0
 prints per task and overall success rates and writes a structured
-`results.json`. Use a new output directory for each evaluation to preserve
-earlier results.
+`results.json`. Output directories must be new so an earlier evaluation is
+never overwritten.
 
 Supported suites.
 
@@ -233,10 +213,7 @@ libero_goal
 libero_10
 ```
 
-</details>
-
-<details>
-<summary><strong>Repository contents</strong></summary>
+## What this repository contains
 
 ```text
 .
@@ -255,13 +232,9 @@ libero_10
 ```
 
 `dependencies/` contains the frozen visual and language encoder weights needed
-by the released WorldDiT runtime. The repository contains every model weight
-required for inference.
+by the released policy. No additional model downloads are required.
 
-</details>
-
-<details>
-<summary><strong>Inference API and tensor shapes</strong></summary>
+## Inference API
 
 ```python
 from inference import load_model
@@ -280,15 +253,12 @@ actions = model(primary_images, wrist_images, robot_state, text_tokens)
 
 Evaluation uses the final temporal slot of the predicted action tensor.
 
-</details>
-
-<details>
-<summary><strong>Architecture details</strong></summary>
+## Architecture details
 
 | Component | Specification |
 |---|---|
-| Backbone | WorldDiT diffusion transformer |
-| Observation context | 3 observation steps |
+| Policy | WorldDiT diffusion transformer |
+| Observation context | 3 frames |
 | Action horizon | 7 actions |
 | Action dimension | 7 |
 | Action aggregation | Temporal ensembling |
@@ -297,38 +267,21 @@ Evaluation uses the final temporal slot of the predicted action tensor.
 | Evaluation | Headless LIBERO with EGL |
 | Checkpoint format | SafeTensors |
 
-</details>
+## Intended use
 
-## Use and scope
-
-| Intended use | Scope of the release |
-|---|---|
-| Research on world and action modeling for language conditioned robot manipulation. The architecture supports continuous action generation with auxiliary future normalized RGB patch prediction. | The current release evaluates WorldDiT in LIBERO simulation under the released protocol and provides checkpoints for all four suites. |
-| Architecture research, reproduction, and evaluation of multimodal diffusion backbones for robot manipulation. | Real robot reliability, safety, and transfer across embodiments require dedicated future evaluation. The present experiments evaluate the integrated WorldDiT system. Targeted ablations are required to attribute performance to the future normalized RGB patch objective. Total instantiated parameter count characterizes model scale. Training cost, deployment latency, and runtime efficiency require dedicated measurements. |
+WorldDiT is intended for research on language conditioned robot manipulation in
+the LIBERO simulator. The released checkpoints support reproduction,
+evaluation, and architecture research across the four released suites.
 
-## Citation
-
-If you use WorldDiT in your research, please cite the paper.
-
-```bibtex
-@article{260723909,
-  title={{WorldDiT: A Unified Diffusion Architecture for World and Action Modeling}},
-  author={Sen Wang and R. Gnana Praveen and Bidhan Roy and Marcos Villagra},
-  year={{2026}},
-  eprint={{2607.23909}},
-  archivePrefix={{arXiv}}
-}
-```
+## Scope of the release
 
-## License
+The reported results describe LIBERO simulation under the released evaluation
+protocol. They do not establish real robot reliability, safety, or transfer
+across embodiments.
 
-The WorldDiT checkpoints, model card, and original release materials are
-licensed under [Creative Commons Attribution 4.0
-International](https://creativecommons.org/licenses/by/4.0/). You may copy,
-redistribute, and adapt them, including commercially, with appropriate credit
-to Bagel Labs and the WorldDiT authors, a link to the license, and an indication
-of any changes. Third party dependencies and assets remain governed by their
-upstream licenses.
+The present release does not isolate the causal contribution of the future
+visual target. Total parameter count also does not measure training cost,
+deployment latency, or runtime efficiency.
 
 ## Authors and contact