| --- |
| tags: |
| - vision-language-action |
| - world-model |
| - robodojo |
| --- |
| |
| # HWM-0-Preview-RoboDojo |
|
|
| HWM-0-Preview combines a vision-language subtask planner, visual and semantic |
| memory, and joint video/action flow matching for dual-arm manipulation. |
| This repository contains the complete RoboDojo inference bundle exported from |
| the **60,000-step EMA checkpoint**. |
|
|
| Run this model through the `HWM_0_preview` adapter in |
| [XPolicyLab](https://github.com/XPolicyLab/XPolicyLab), which provides policy |
| serving and RoboDojo evaluation. This repository supplies the weights and |
| runtime assets used by that adapter. |
|
|
| ## Contents |
|
|
| | File | Size | Description | |
| | --- | ---: | --- | |
| | [model.pt](model.pt) | 31.97 GiB | Complete BF16 model state dictionary | |
| | [config.yaml](config.yaml) | 5.06 KiB | Architecture, conditioning, history and inference settings | |
| | [dataset_statistics.json](dataset_statistics.json) | 5.14 KiB | Required state/action normalization statistics | |
| | [processor/](processor/) | 19.08 MiB | VLM processor, tokenizer and chat template | |
| | [tokenizer/](tokenizer/) | 16.06 MiB | T5 tokenizer assets | |
| | [manifest.json](manifest.json) | 1.11 KiB | Export checksums and configuration fingerprint | |
|
|
| ```text |
| HWM-0-Preview-RoboDojo/ |
| ├── README.md |
| ├── .gitattributes |
| ├── model.pt |
| ├── config.yaml |
| ├── dataset_statistics.json |
| ├── manifest.json |
| ├── processor/ |
| └── tokenizer/ |
| ``` |
|
|
| The checkpoint includes the VLM, T5 encoder, VAE, Video DiT, Action DiT, |
| state projection and learned queries. The matching adapter constructs these |
| components from the bundled configuration without downloading additional |
| backbone weights. Both tokenizers load from local JSON assets. |
|
|
| ## Usage with XPolicyLab |
|
|
| Follow the official [XPolicyLab guide](https://github.com/XPolicyLab/XPolicyLab) |
| for setup, checkpoint conventions and deployment, and the |
| `policy/HWM_0_preview/README.md` in the matching source release for this model's |
| installation requirements. Use a checkout that includes the `HWM_0_preview` |
| adapter. Keep the policy environment separate from the RoboDojo simulator/client |
| environment, as recommended by XPolicyLab. |
|
|
| ### Download the bundle |
|
|
| From the XPolicyLab checkout, enter the adapter directory. With the Hugging Face |
| CLI available, download the complete bundle into `HWM-0-Preview-RoboDojo/`: |
|
|
| ```bash |
| cd policy/HWM_0_preview |
| hf download "HorizonRobotics/HWM-0-Preview-RoboDojo" \ |
| --local-dir HWM-0-Preview-RoboDojo |
| ``` |
|
|
| The adapter also provides `download_checkpoint.sh`, which defaults to |
| current repo: |
|
|
| ```bash |
| bash download_checkpoint.sh |
| ``` |
|
|
| Both commands use `HWM-0-Preview-RoboDojo/` inside the adapter directory. |
| Keep the entire bundle together, including `config.yaml`, normalization |
| statistics, `processor/` and `tokenizer/`. |
|
|
| ### Evaluate through the standard entrypoint |
|
|
| After installing the policy and evaluation environments according to their |
| respective guides, run from the same adapter directory: |
|
|
| ```bash |
| # Replace these with your installed environment names and available GPU IDs. |
| POLICY_ENV=hwm_policy |
| EVAL_ENV=robodojo |
| POLICY_GPU=0 |
| ENV_GPU=1 |
| |
| EVAL_ENV_TYPE=sim bash eval.sh \ |
| RoboDojo stack_bowls ./HWM-0-Preview-RoboDojo arx_x5 joint 0 \ |
| "$POLICY_GPU" "$ENV_GPU" "$POLICY_ENV" "$EVAL_ENV" |
| ``` |
|
|
| `eval.sh` starts the policy server, waits for it to become ready, runs the |
| environment client and cleans up the server. The adapter resolves the checkpoint |
| and loads the model and its assets. Change `stack_bowls` to the desired task. |
| The `ckpt_name` argument is the explicit path `./HWM-0-Preview-RoboDojo`, |
| so different evaluation seeds use the same downloaded bundle. |
|
|
| Use `EVAL_ENV_TYPE=debug` for an offline interface check; it does not measure task |
| success. For simulation, follow the official RoboDojo setup and XPolicyLab |
| workspace layout. For split-machine deployment, follow XPolicyLab's |
| **Deployment Flow** using `setup_eval_policy_server.sh` on the policy machine and |
| `setup_eval_env_client.sh` on the simulator machine. Refer to those official |
| guides for argument details and environment setup. |
|
|
| The model requires approximately 32 GiB for weights alone, plus memory for |
| activations and caches. Select a policy GPU with sufficient headroom; prior |
| integration validation used H20 hardware. Model-specific inference settings are |
| configured through the adapter's `deploy.yml`: `num_inference_steps`, `flow_shift` |
| and `replan_steps` inherit bundle defaults of 10, 5.0 and 30 when set to `null`. |
| `inference_batch_size` defaults to 4; reduce it for lower GPU memory use. |
|
|
| ## Model geometry |
|
|
| | Item | Value | |
| | --- | --- | |
| | Checkpoint | 60,000-step EMA | |
| | Tensors / dtype | 3,078 / BF16 | |
| | Stored tensor elements | 17,162,844,810 | |
| | Robot | ARX X5 dual arm | |
| | State / action dimension | 14 / 14 | |
| | Action horizon | 48 absolute joint-position targets | |
| | Execution horizon | 30 control steps per replan | |
| | Sampling | 10 flow-matching steps; flow shift 5.0 | |
| | Camera input | Head, left-wrist and right-wrist RGB | |
| | Visual history | Up to 20 frames, sampled every 30 control steps | |
| | Normalization | Bundled mean/std statistics, key `new_embodiment` | |
|
|
| ## Integrity |
|
|
| `model.pt` is **34,326,694,240 bytes**. Its SHA256 digest is: |
|
|
| ```text |
| 40d998b67d27824fab5c6b64b4f20c2926d299dac9393dadb07a8c831a226988 |
| ``` |
|
|
| The remaining original bundle checksums are recorded in |
| [manifest.json](manifest.json). Exported weights, configuration, normalization |
| statistics and tokenizer assets are preserved byte for byte. |
|
|
| This release contains inference weights and assets. Training state and |
| benchmark results are not included; no complete benchmark score is claimed here. |
|
|
