| --- |
| library_name: lerobot |
| pipeline_tag: robotics |
| tags: |
| - lerobot |
| - robotics |
| - imitation-learning |
| - x7s |
| - diffusion-policy |
| - offline-evaluation |
| datasets: |
| - LightwheelAI/Lightwheel-Tasks-X7S |
| --- |
| |
| # LW-Bench Diffusion Policy 5K for X7s |
|
|
| LeRobot Diffusion Policy trained from scratch on the audited X7s strict compositional split in LW-Compositional-Bench. |
|
|
| ## Training contract |
|
|
| - Dataset: `LightwheelAI/Lightwheel-Tasks-X7S` |
| - Train split: 91 clean tasks; tasks 7, 119, and 149 held out |
| - Inputs: three RGB cameras plus 25D robot state |
| - Output: native 21D X7s absolute action |
| - Architecture: ResNet18 plus conditional diffusion |
| - Training horizon: 24 with 2 observations and 16 future action steps |
| - Hardware: 4 GPUs |
| - Seed: 1000 |
| - Effective global batch: 64 |
| - Checkpoint: 5,000 global optimizer updates |
|
|
| ## Offline 5K results |
|
|
| | Metric | Value | |
| | --- | ---: | |
| | H16 MAE | 0.100714 | |
| | H16 RMSE | 0.208515 | |
| | Gripper balanced accuracy | 91.61% | |
| | Boundary-proxy MAE | 0.129451 | |
| | Interior MAE | 0.095209 | |
| | Boundary / interior MAE | 1.360x | |
|
|
| These results cover three strict held-out tasks and are descriptive. Offline action error does not replace closed-loop success evaluation. |
|
|
| ## Load with LeRobot |
|
|
| ```python |
| from lerobot.policies.diffusion.modeling_diffusion import DiffusionPolicy |
| |
| policy = DiffusionPolicy.from_pretrained("Stevenshuqing/LW-Bench-Diffusion") |
| policy.eval() |
| ``` |
|
|
| Use LeRobot 0.4.3 and the included preprocessor/postprocessor files. The repository contains the complete `pretrained_model` directory produced by LeRobot. |
|
|
| ## Provenance |
|
|
| - Code and protocol: https://github.com/stevenqing/LW-Compositional-Bench |
| - Full report: https://github.com/stevenqing/LW-Compositional-Bench/blob/main/artifacts/model-eval/x7s/rotation-v0/budget-5k/report.md |
| - Code revision: `2d0e26dd2fe2e91046f416bd65bc369f8568d7f0` |
| - `model.safetensors` SHA256: `e1540ffe39b85bd762b4ab90c52a64d16e031b048957a0d02cd664663395849b` |
|
|
| ## Limitations |
|
|
| This checkpoint is specific to the X7s observation/action schema. The benchmark is offline because the available A100 GPUs cannot provide Isaac Sim RT camera rendering. Transition boundaries are action-space proxies based on gripper sign changes and robust continuous-action peaks, not human semantic annotations. |
|
|
