bagellabs commited on
Commit
2e2cc58
1 Parent(s): c0548c1

Improve the WorldDiT model card layout (#2)

Browse files

- Improve the WorldDiT model card layout (4b7fb954f1c92b19a043b1eaf4de8bf9b642ca09)

Files changed (1) hide show
  1. README.md +77 -82
README.md CHANGED
@@ -38,91 +38,81 @@ widget:
38
  <img src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/paris2/model-card/v1/bagel_labs_logo.png" alt="Bagel Labs">
39
  </p>
40
 
41
- # WorldDiT
42
 
43
- ## One diffusion backbone learns what to do and what comes next.
44
 
45
  <p align="center">
46
- <a href="https://huggingface.co/bageldotcom/worlddit" target="_blank">
47
- <img src="https://img.shields.io/badge/馃_DOWNLOAD_WORLDDIT_WEIGHTS-FFD21E?style=for-the-badge&logoColor=000000" alt="Download WorldDiT Weights">
48
- </a>
49
- <a href="https://github.com/Lifelong-Robot-Learning/LIBERO" target="_blank">
50
- <img src="https://img.shields.io/badge/馃_LIBERO_BENCHMARK-FF6B6B?style=for-the-badge&logoColor=white" alt="LIBERO Benchmark">
51
- </a>
52
- </p>
53
-
54
  WorldDiT learns continuous robot action chunks and a future visual target
55
  through one shared diffusion transformer. Deployment keeps only the action
56
  path.
 
57
 
 
58
  This release includes four LIBERO checkpoints, a self contained inference
59
  runtime, and an evaluator for reproducing the reported suite results.
 
60
 
61
  ## See WorldDiT act
62
 
63
  The four clips below show successful rollouts from the released checkpoints.
64
  Each clip covers a different LIBERO suite and camera view.
65
 
66
- <Gallery />
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
 
68
- | Suite | View | Video |
69
  |---|---|---|
70
- | LIBERO Spatial | Front view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_spatial_frontview_task05_episode01.mp4) |
71
- | LIBERO Object | Agent view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_object_agentview_task08_episode01.mp4) |
72
- | LIBERO Goal | Side view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_goal_sideview_task10_episode01.mp4) |
73
- | LIBERO Long | Front view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_10_frontview_task06_episode01.mp4) |
74
 
75
- ## What is in this release
76
-
77
- | Release component | Included artifact |
78
- |---|---|
79
- | LIBERO Spatial policy | SafeTensors checkpoint |
80
- | LIBERO Object policy | SafeTensors checkpoint |
81
- | LIBERO Goal policy | SafeTensors checkpoint |
82
- | LIBERO Long policy | SafeTensors checkpoint |
83
- | Model runtime | `inference.py` |
84
- | Evaluation runtime | `eval.py` |
85
- | Frozen encoders | CLIP ViT B 32 and MAE ViT B |
86
- | Configuration | `config.json` |
87
- | Environment | Pinned Python requirements |
88
-
89
- The repository is self contained for WorldDiT inference. LIBERO still provides
90
- the benchmark environments, assets, task definitions, and initial states.
91
-
92
- ## Reported LIBERO results
93
-
94
- Across the four released suite checkpoints, WorldDiT records 1,898 successful
95
- episodes out of 2,000 under the selection aware evaluation protocol.
96
-
97
- | Suite | Successful episodes | Success rate |
98
- |---|---:|---:|
99
- | LIBERO Spatial | 490 of 500 | 98.0 percent |
100
- | LIBERO Object | 485 of 500 | 97.0 percent |
101
- | LIBERO Goal | 464 of 500 | 92.8 percent |
102
- | LIBERO Long | 459 of 500 | 91.8 percent |
103
- | Selection aware mean | 1,898 of 2,000 | 94.9 percent |
104
 
105
  The released runtime and checkpoints were revalidated from a clean installation
106
- on eight RTX Pro 6000 Blackwell GPUs.
107
-
108
- The result is selection aware because three hundred episodes per suite informed
109
- staged checkpoint selection before the final five hundred episode score was
110
- assembled.
111
-
112
- ## Model at a glance
113
-
114
- | Property | Released configuration |
115
- |---|---|
116
- | Total parameters | 399.084 million |
117
- | Trainable parameters | 135.107 million |
118
- | Observation context | Three frames |
119
- | Predicted action horizon | Seven actions |
120
- | Executed before replanning | Three actions |
121
- | Action dimension | Seven |
122
- | Visual encoder | MAE ViT B |
123
- | Language encoder | OpenAI CLIP ViT B 32 |
124
- | Checkpoint format | SafeTensors |
125
- | Evaluation environment | Headless LIBERO with EGL |
126
 
127
  ## Run a smoke test
128
 
@@ -179,7 +169,10 @@ and replans.
179
  | Future visual supervision is present | No future visual output is requested |
180
  | The complete training objective is active | Three actions execute before replanning |
181
 
182
- ## Evaluation
 
 
 
183
 
184
  ### One GPU
185
 
@@ -213,7 +206,10 @@ libero_goal
213
  libero_10
214
  ```
215
 
216
- ## What this repository contains
 
 
 
217
 
218
  ```text
219
  .
@@ -234,7 +230,10 @@ libero_10
234
  `dependencies/` contains the frozen visual and language encoder weights needed
235
  by the released policy. No additional model downloads are required.
236
 
237
- ## Inference API
 
 
 
238
 
239
  ```python
240
  from inference import load_model
@@ -253,7 +252,10 @@ actions = model(primary_images, wrist_images, robot_state, text_tokens)
253
 
254
  Evaluation uses the final temporal slot of the predicted action tensor.
255
 
256
- ## Architecture details
 
 
 
257
 
258
  | Component | Specification |
259
  |---|---|
@@ -267,21 +269,14 @@ Evaluation uses the final temporal slot of the predicted action tensor.
267
  | Evaluation | Headless LIBERO with EGL |
268
  | Checkpoint format | SafeTensors |
269
 
270
- ## Intended use
271
-
272
- WorldDiT is intended for research on language conditioned robot manipulation in
273
- the LIBERO simulator. The released checkpoints support reproduction,
274
- evaluation, and architecture research across the four released suites.
275
 
276
- ## Scope of the release
277
 
278
- The reported results describe LIBERO simulation under the released evaluation
279
- protocol. They do not establish real robot reliability, safety, or transfer
280
- across embodiments.
281
-
282
- The present release does not isolate the causal contribution of the future
283
- visual target. Total parameter count also does not measure training cost,
284
- deployment latency, or runtime efficiency.
285
 
286
  ## Authors and contact
287
 
 
38
  <img src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/paris2/model-card/v1/bagel_labs_logo.png" alt="Bagel Labs">
39
  </p>
40
 
41
+ <h1 align="center">WorldDiT</h1>
42
 
43
+ <h2 align="center">One diffusion backbone learns what to do and what comes next.</h2>
44
 
45
  <p align="center">
 
 
 
 
 
 
 
 
46
  WorldDiT learns continuous robot action chunks and a future visual target
47
  through one shared diffusion transformer. Deployment keeps only the action
48
  path.
49
+ </p>
50
 
51
+ <p align="center">
52
  This release includes four LIBERO checkpoints, a self contained inference
53
  runtime, and an evaluator for reproducing the reported suite results.
54
+ </p>
55
 
56
  ## See WorldDiT act
57
 
58
  The four clips below show successful rollouts from the released checkpoints.
59
  Each clip covers a different LIBERO suite and camera view.
60
 
61
+ <table>
62
+ <tr>
63
+ <td width="50%" valign="top">
64
+ <video width="100%" controls muted loop playsinline preload="metadata"
65
+ src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_spatial_frontview_task05_episode01.mp4"></video>
66
+ <strong>LIBERO Spatial</strong><br>
67
+ Task 5, front view.
68
+ </td>
69
+ <td width="50%" valign="top">
70
+ <video width="100%" controls muted loop playsinline preload="metadata"
71
+ src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_object_agentview_task08_episode01.mp4"></video>
72
+ <strong>LIBERO Object</strong><br>
73
+ Task 8, agent view.
74
+ </td>
75
+ </tr>
76
+ <tr>
77
+ <td width="50%" valign="top">
78
+ <video width="100%" controls muted loop playsinline preload="metadata"
79
+ src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_goal_sideview_task10_episode01.mp4"></video>
80
+ <strong>LIBERO Goal</strong><br>
81
+ Task 10, side view.
82
+ </td>
83
+ <td width="50%" valign="top">
84
+ <video width="100%" controls muted loop playsinline preload="metadata"
85
+ src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_10_frontview_task06_episode01.mp4"></video>
86
+ <strong>LIBERO Long</strong><br>
87
+ Task 6, front view.
88
+ </td>
89
+ </tr>
90
+ </table>
91
+
92
+ [Spatial MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_spatial_frontview_task05_episode01.mp4)
93
+ 路 [Object MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_object_agentview_task08_episode01.mp4)
94
+ 路 [Goal MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_goal_sideview_task10_episode01.mp4)
95
+ 路 [Long MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_10_frontview_task06_episode01.mp4)
96
+
97
+ ## Release snapshot
98
+
99
+ | Reported LIBERO result | Released model |
100
+ |---|---|
101
+ | **94.9 percent** selection aware mean<br>**1,898 of 2,000** successful episodes | **399.084 million** total parameters<br>**135.107 million** trainable parameters |
102
+ | **98.0 percent** Spatial<br>**97.0 percent** Object | **Three** observation frames<br>**Seven** predicted actions |
103
+ | **92.8 percent** Goal<br>**91.8 percent** Long | **Three** actions executed before replanning<br>**Seven** action dimensions |
104
 
105
+ | Checkpoints | Runtime | Encoders and environment |
106
  |---|---|---|
107
+ | Spatial<br>Object<br>Goal<br>Long | `inference.py`<br>`eval.py`<br>`config.json` | MAE ViT B<br>OpenAI CLIP ViT B 32<br>SafeTensors and pinned requirements |
 
 
 
108
 
109
+ The repository is self contained for WorldDiT inference. LIBERO provides the
110
+ benchmark environments, assets, task definitions, and initial states.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
111
 
112
  The released runtime and checkpoints were revalidated from a clean installation
113
+ on eight RTX Pro 6000 Blackwell GPUs. The result is selection aware because
114
+ three hundred episodes per suite informed staged checkpoint selection before
115
+ the final five hundred episode score was assembled.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
116
 
117
  ## Run a smoke test
118
 
 
169
  | Future visual supervision is present | No future visual output is requested |
170
  | The complete training objective is active | Three actions execute before replanning |
171
 
172
+ ## Reference
173
+
174
+ <details>
175
+ <summary><strong>Full evaluation commands</strong></summary>
176
 
177
  ### One GPU
178
 
 
206
  libero_10
207
  ```
208
 
209
+ </details>
210
+
211
+ <details>
212
+ <summary><strong>Repository contents</strong></summary>
213
 
214
  ```text
215
  .
 
230
  `dependencies/` contains the frozen visual and language encoder weights needed
231
  by the released policy. No additional model downloads are required.
232
 
233
+ </details>
234
+
235
+ <details>
236
+ <summary><strong>Inference API and tensor shapes</strong></summary>
237
 
238
  ```python
239
  from inference import load_model
 
252
 
253
  Evaluation uses the final temporal slot of the predicted action tensor.
254
 
255
+ </details>
256
+
257
+ <details>
258
+ <summary><strong>Architecture details</strong></summary>
259
 
260
  | Component | Specification |
261
  |---|---|
 
269
  | Evaluation | Headless LIBERO with EGL |
270
  | Checkpoint format | SafeTensors |
271
 
272
+ </details>
 
 
 
 
273
 
274
+ ## Use and scope
275
 
276
+ | Intended use | Scope of the release |
277
+ |---|---|
278
+ | Research on language conditioned robot manipulation in the LIBERO simulator. The released checkpoints support reproduction, evaluation, and architecture research across the four released suites. | The results describe LIBERO simulation under the released evaluation protocol. They do not establish real robot reliability, safety, or transfer across embodiments. |
279
+ | The released checkpoints cover all four LIBERO suites. | The release does not isolate the causal contribution of the future visual target. Total parameter count does not measure training cost, deployment latency, or runtime efficiency. |
 
 
 
280
 
281
  ## Authors and contact
282