Align the WorldDiT model card with the paper

README.md

@@ -12,9 +12,8 @@ tags:
   - robotic-manipulation
   - imitation-learning
   - diffusion-transformer
+  - diffusion-policy
   - flow-matching
-  - pareto-frontier
-  - parameter-efficiency
 inference: false
 license: cc-by-4.0
 widget:
@@ -42,18 +41,18 @@ widget:
 
 <h1 align="center">WorldDiT: A Unified Diffusion Backbone for<br>World and Action Modeling</h1>
 
+<h2 align="center">One diffusion backbone generates continuous actions and predicts future normalized RGB patches.</h2>
+
 <p align="center">
-  <a href="https://arxiv.org/abs/2607.23909">
-    <img src="https://img.shields.io/badge/arXiv-2607.23909-B31B1B.svg?logo=arxiv&logoColor=white" alt="WorldDiT paper on arXiv">
-  </a>
+WorldDiT uses one shared diffusion transformer to generate continuous robot
+action chunks and predict normalized RGB patches selected from future camera
+frames, without using a large pretrained vision language model as its action
+backbone. At deployment, the auxiliary RGB patch prediction branch is omitted.
 </p>
 
 <p align="center">
-WorldDiT couples continuous action generation with auxiliary future normalized
-RGB patch prediction in one diffusion transformer. The architecture is designed
-as a general backbone for world and action modeling, while the current release
-evaluates it on LIBERO and provides four checkpoints, a self contained
-inference runtime, and an evaluator.
+This release includes four LIBERO checkpoints, a self contained inference
+runtime, and an evaluator for reproducing the reported suite results.
 </p>
 
 ## See WorldDiT act
@@ -101,8 +100,8 @@ Each clip covers a different LIBERO suite and camera view.
 
 | Reported LIBERO result | Released model |
 |---|---|
-| **94.9 percent** mean success<br>**1,898 of 2,000** successful episodes | **399.084 million** total parameters<br>**135.107 million** trainable parameters |
-| **98.0 percent** Spatial<br>**97.0 percent** Object | **Three** observation steps<br>**Seven** predicted actions |
+| **94.9 percent** selection aware mean<br>**1,898 of 2,000** successful episodes | **399.084 million** total parameters<br>**135.107 million** trainable parameters |
+| **98.0 percent** Spatial<br>**97.0 percent** Object | **Three** observation frames<br>**Seven** predicted actions |
 | **92.8 percent** Goal<br>**91.8 percent** Long | **Three** actions executed before replanning<br>**Seven** action dimensions |
 
 | Checkpoints | Runtime | Encoders and environment |
@@ -113,23 +112,9 @@ The repository is self contained for WorldDiT inference. LIBERO provides the
 benchmark environments, assets, task definitions, and initial states.
 
 The released runtime and checkpoints were revalidated from a clean installation
-on eight RTX Pro 6000 Blackwell GPUs. The reported aggregate covers five
-hundred simulator episodes per suite. Three hundred episodes per suite informed
-staged checkpoint selection, while two hundred episodes per suite were disjoint
-from selection.
-
-### Parameter count and reported success
-
-Among methods with complete four suite averages, WorldDiT lies on the reported
-Pareto frontier for total model parameters and mean LIBERO success.
-
-<p align="center">
-  <img src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v2/worlddit_parameter_efficiency.png" alt="Reported mean LIBERO success against total instantiated model parameters">
-</p>
-
-<p align="center">
-  <sub>Reported LIBERO success against total model parameters for 24 methods. The line connects methods on the Pareto frontier with complete four suite averages. Because the methods follow different published evaluation protocols, the figure summarizes published results rather than a direct comparison under one common evaluation protocol.</sub>
-</p>
+on eight RTX Pro 6000 Blackwell GPUs. The result is selection aware because
+three hundred episodes per suite informed staged checkpoint selection before
+the final five hundred episode score was assembled.
 
 ## Run a smoke test
 
@@ -166,34 +151,24 @@ python eval.py \
 ```
 
 A successful smoke test confirms that the environment, checkpoint, visual
-encoders, simulator, and rendering path load together. Full benchmark reporting
-uses complete suite evaluations.
+encoders, simulator, and rendering path load together. It is not a benchmark
+result.
 
 ## How WorldDiT works
 
-Each of three recent observation steps contributes primary and wrist images
-together with robot state, while one language instruction conditions the
-sequence. During training, one diffusion transformer backbone learns a seven
-step action chunk together with an auxiliary future normalized RGB patch
-target. At deployment, the encoded history conditions the action velocity field
-directly. RGB patch token construction and RGB prediction head evaluation
-remain outside the inference graph, concentrating computation on action
-generation. The controller executes the first three predicted actions, observes
-again, and replans.
+WorldDiT uses three recent observations, robot state, and language as context.
+During training, one diffusion transformer learns a seven step action chunk and
+an auxiliary future visual target. During deployment, the future visual path is
+absent. The policy executes the first three predicted actions, observes again,
+and replans.
 
-<p align="center">
-  <img src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v2/worlddit_inference_pipeline.png?v=09d6508" alt="WorldDiT inference pipeline">
-</p>
-
-<p align="center">
-  <sub>WorldDiT inference pipeline. The encoded observation history conditions action generation through twenty flow steps. The controller executes the first three actions from each seven action chunk, then updates the window and replans.</sub>
-</p>
+> Future visual prediction is a training signal, not a deployment path.
 
 | Training | Deployment |
 |---|---|
-| Action and future normalized RGB patch targets are learned by one backbone | Encoded history conditions the action velocity field |
+| Action and future visual targets share one backbone | Only the action path remains |
 | Seven action steps are supervised | Seven actions are predicted |
-| Future normalized RGB patch supervision is present | RGB patch tokens and the RGB prediction head remain outside the inference graph |
+| Future visual supervision is present | No future visual output is requested |
 | The complete training objective is active | Three actions execute before replanning |
 
 ## Reference
@@ -221,8 +196,8 @@ CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python eval.py \
 
 Each GPU receives an independent progress bar. After all workers finish, rank 0
 prints per task and overall success rates and writes a structured
-`results.json`. Use a new output directory for each evaluation to preserve
-earlier results.
+`results.json`. Output directories must be new so an earlier evaluation is
+never overwritten.
 
 Supported suites.
 
@@ -255,8 +230,7 @@ libero_10
 ```
 
 `dependencies/` contains the frozen visual and language encoder weights needed
-by the released WorldDiT runtime. The repository contains every model weight
-required for inference.
+by the released policy. No additional model downloads are required.
 
 </details>
 
@@ -287,8 +261,8 @@ Evaluation uses the final temporal slot of the predicted action tensor.
 
 | Component | Specification |
 |---|---|
-| Backbone | WorldDiT diffusion transformer |
-| Observation context | 3 observation steps |
+| Policy | WorldDiT diffusion transformer |
+| Observation context | 3 frames |
 | Action horizon | 7 actions |
 | Action dimension | 7 |
 | Action aggregation | Temporal ensembling |
@@ -303,22 +277,8 @@ Evaluation uses the final temporal slot of the predicted action tensor.
 
 | Intended use | Scope of the release |
 |---|---|
-| Research on world and action modeling for language conditioned robot manipulation. The architecture supports continuous action generation with auxiliary future normalized RGB patch prediction. | The current release evaluates WorldDiT in LIBERO simulation under the released protocol and provides checkpoints for all four suites. |
-| Architecture research, reproduction, and evaluation of multimodal diffusion backbones for robot manipulation. | Real robot reliability, safety, and transfer across embodiments require dedicated future evaluation. The present experiments evaluate the integrated WorldDiT system. Targeted ablations are required to attribute performance to the future normalized RGB patch objective. Total instantiated parameter count characterizes model scale. Training cost, deployment latency, and runtime efficiency require dedicated measurements. |
-
-## Citation
-
-If you use WorldDiT in your research, please cite the paper.
-
-```bibtex
-@article{260723909,
-  title={{WorldDiT: A Unified Diffusion Architecture for World and Action Modeling}},
-  author={Sen Wang and R. Gnana Praveen and Bidhan Roy and Marcos Villagra},
-  year={{2026}},
-  eprint={{2607.23909}},
-  archivePrefix={{arXiv}}
-}
-```
+| Research on language conditioned robot manipulation in the LIBERO simulator. The released checkpoints support reproduction, evaluation, and architecture research across the four released suites. | The results describe LIBERO simulation under the released evaluation protocol. They do not establish real robot reliability, safety, or transfer across embodiments. |
+| The released checkpoints cover all four LIBERO suites. | The release does not isolate the causal contribution of the future visual target. Total parameter count does not measure training cost, deployment latency, or runtime efficiency. |
 
 ## License