Blame for README.md

---
license: other
license_name: rlwrld-model-license-v2.0
license_link: LICENSE.md
library_name: transformers
pipeline_tag: robotics
tags:
  - robotics
  - vla
  - vision-language-action
  - manipulation
  - flow-matching
  - rldx
  - video-input
  - robocasa
base_model: RLWRLD/RLDX-1.1-PT
---

# RLDX-1.1-FT-ROBOCASA

[Paper](https://arxiv.org/abs/2605.03269)  ·  [Project page](https://rlwrld.ai/rldx-1)  ·  [Code](https://github.com/RLWRLD/RLDX-1/tree/v1.1.0)  ·  [Models](https://huggingface.co/collections/RLWRLD/rldx-11)

<p align="center">
<img src="rldx_overview.png" width="100%" alt="RLDX-1.1 overview">
</p>

**RLDX-1.1** is a Vision-Language-Action model (VLA) for human-like dexterous manipulation. Beyond the *versatile intelligence* inherited from pre-trained VLM backbones, RLDX-1.1 adds three **functional capabilities** — motion awareness, long-term memory, and physical sensing — through a unified **Multi-Stream Action Transformer (MSAT)** architecture, a synthetic-augmented training pipeline, and a real-time inference stack.

This repository hosts **`RLDX-1.1-FT-ROBOCASA`**, [`RLWRLD/RLDX-1.1-PT`](https://huggingface.co/RLWRLD/RLDX-1.1-PT) fine-tuned on **RoboCasa Kitchen (24 tasks)** with 4-frame video input.

## What's New in RLDX-1.1

- **Better performance.** A refined RLDX-1 corpus plus a broad collection of open-source robot datasets gives a stronger pre-training foundation. Video-input fine-tunes enable the motion module, which RLDX-1 video checkpoints did not use.
- **Smoother actions in low DoF.** Padded action dimensions are masked from pre-training on, so flow-matching noise no longer reaches unused joints. Keep `mask_padded_action_dims=True` when fine-tuning from this checkpoint.
- **Faster training and inference.** Training is 2.27× faster than RLDX-1. Inference is up to 9.4× faster on H100 and 3.7× on RTX 5090, with fp8 on every supported GPU and nvfp4 on Jetson Thor, GB10 and RTX 5090. See [Optimization](https://github.com/RLWRLD/RLDX-1/blob/v1.1.0/docs/optimization.md).

Details are in the [README](https://github.com/RLWRLD/RLDX-1/tree/v1.1.0#whats-new-in-rldx-11).

<p align="center">
<img src="overview_architecture.png" width="90%" alt="RLDX-1.1 architecture">
</p>

## Model Family

**Video input**

| Checkpoint | Type | Description |
|---|---|---|
| [`RLDX-1.1-PT`](https://huggingface.co/RLWRLD/RLDX-1.1-PT) | Base | Pre-trained |
| **`RLDX-1.1-FT-ROBOCASA`** (this repo) | Fine-tuned | RoboCasa Kitchen |
| [`RLDX-1.1-FT-GR1`](https://huggingface.co/RLWRLD/RLDX-1.1-FT-GR1) | Fine-tuned | GR-1 Tabletop |
| [`RLDX-1.1-FT-LIBERO`](https://huggingface.co/RLWRLD/RLDX-1.1-FT-LIBERO) | Fine-tuned | LIBERO |
| [`RLDX-1.1-FT-RC365`](https://huggingface.co/RLWRLD/RLDX-1.1-FT-RC365) | Fine-tuned | RoboCasa365 |

**Image input**

| Checkpoint | Type | Description |
|---|---|---|
| [`RLDX-1.1-PT-IMG`](https://huggingface.co/RLWRLD/RLDX-1.1-PT-IMG) | Base | Pre-trained |
| [`RLDX-1.1-FT-ROBOCASA-IMG`](https://huggingface.co/RLWRLD/RLDX-1.1-FT-ROBOCASA-IMG) | Fine-tuned | RoboCasa Kitchen |
| [`RLDX-1.1-FT-GR1-IMG`](https://huggingface.co/RLWRLD/RLDX-1.1-FT-GR1-IMG) | Fine-tuned | GR-1 Tabletop |
| [`RLDX-1.1-FT-LIBERO-IMG`](https://huggingface.co/RLWRLD/RLDX-1.1-FT-LIBERO-IMG) | Fine-tuned | LIBERO |
| [`RLDX-1.1-FT-RC365-IMG`](https://huggingface.co/RLWRLD/RLDX-1.1-FT-RC365-IMG) | Fine-tuned | RoboCasa365 |

RLDX-1 checkpoints also load with this codebase; see [Model Checkpoints](https://github.com/RLWRLD/RLDX-1/tree/v1.1.0#model-checkpoints).

## Performance

| Benchmark | Success Rate (%) |
|---|---|
| RoboCasa Kitchen | **72.3** |

Evaluation guide: [RoboCasa Kitchen](https://github.com/RLWRLD/RLDX-1/blob/v1.1.0/run_scripts/eval/robocasa_kitchen/README.md). Training recipe: [`finetune_robocasa.sh`](https://github.com/RLWRLD/RLDX-1/blob/v1.1.0/run_scripts/train/benchmarks/finetune_robocasa.sh) with `VARIANT=video`.

The comparison with other VLAs is in the [README](https://github.com/RLWRLD/RLDX-1/tree/v1.1.0#comparison-with-other-vlas).

## Quick Start

```bash
git clone --branch v1.1.0 https://github.com/RLWRLD/RLDX-1.git
cd RLDX-1
uv sync --python 3.10
```

Load a checkpoint and predict an action chunk in-process:

```python
from rldx.serving.policy import load_policy
from rldx.data.embodiment_tags import EmbodimentTag

policy = load_policy(
    model_path="RLWRLD/RLDX-1.1-FT-ROBOCASA",
    embodiment_tag=EmbodimentTag.GENERAL_EMBODIMENT,
    device="cuda:0",
)

# observation: {"video": {...}, "state": {...}, "language": {...}}
actions, info = policy.get_action(observation)
```

Serve the same policy over ZeroMQ for simulator rollouts or a real robot:

```bash
uv run python rldx/eval/run_rldx_server.py \
    --model-path RLWRLD/RLDX-1.1-FT-ROBOCASA \
    --embodiment-tag GENERAL_EMBODIMENT \
    --host 0.0.0.0 --port 20000
```

Runtime profiles (fp8 / nvfp4, CUDA graphs) and Real-Time Chunking are described in [`docs/inference_server.md`](https://github.com/RLWRLD/RLDX-1/blob/v1.1.0/docs/inference_server.md).

## Model Details

- **Architecture:** Multi-Stream Action Transformer (MSAT) with a Qwen3-VL vision-language backbone and cognition tokens, trained with flow matching.
- **Inputs:** RGB video, 4 frames per camera (stride 2), proprioceptive state, and a language instruction.
- **Outputs:** action chunks of 16 steps.
- **Modules:** motion module on (inserted at VLM layer 9); memory and physics off. Padded action dimensions are masked (`mask_padded_action_dims=True`).
- **Embodiment tag:** `GENERAL_EMBODIMENT`.
- **Base model:** [`RLWRLD/RLDX-1.1-PT`](https://huggingface.co/RLWRLD/RLDX-1.1-PT).
- **Backbone:** initialized from [`Qwen/Qwen3-VL-8B-Instruct`](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) and modified through further training (see [Third-Party Notice](#third-party-notice)).
- **Parameters:** 6.6B.

A walkthrough of the architecture is in [`docs/architecture.md`](https://github.com/RLWRLD/RLDX-1/blob/v1.1.0/docs/architecture.md).

## Intended Use and Limitations

**Intended use.** Research on robotic manipulation, fine-tuning on custom embodiments, simulation benchmarking, and non-commercial real-robot deployment under the RLWRLD Model License v2.0. Commercial use requires prior written permission from RLWRLD (§3.1).

**Out of scope.** Surveillance or monitoring of individuals without their consent, and any use that violates applicable laws or regulations (§3.5 of [`LICENSE.md`](https://huggingface.co/RLWRLD/RLDX-1.1-FT-ROBOCASA/blob/main/LICENSE.md)).

**Limitations.** Results are reported on the RoboCasa Kitchen benchmark. Other scenes, objects, or embodiments are not guaranteed; for a different robot or dataset, fine-tune from [`RLWRLD/RLDX-1.1-PT`](https://huggingface.co/RLWRLD/RLDX-1.1-PT) instead.

## Third-Party Notice

The vision-language backbone of this checkpoint was initialized from [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) by the Qwen Team, Alibaba Cloud, licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0), and was modified by RLWRLD through further training. See [`NOTICE`](https://huggingface.co/RLWRLD/RLDX-1.1-FT-ROBOCASA/blob/main/NOTICE).

## Citation

```bibtex
@article{rldx2026,
  title={RLDX-1 Technical Report},
  author={Dongyoung Kim and Huiwon Jang and Myungkyu Koo and Suhyeok Jang and Taeyoung Kim and others},
  year={2026},
  journal={arXiv preprint arXiv:2605.03269},
  eprint={2605.03269},
  archivePrefix={arXiv}
}
```

Please also cite the Qwen3-VL backbone and RoboCOIN, whose data was used in pre-training.

```bibtex
@misc{qwen3technicalreport,
      title={Qwen3 Technical Report},
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.09388},
}

@article{robocoin,
    title={RoboCOIN: An Open-Sourced Bimanual Robotic Data Collection for Integrated Manipulation},
    author={Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, Zhaoye Long, Yue Wang, Chong Liu, Dihan Wang, Ziqiang Ni, Xiang Yang, You Liu, Ruoxuan Feng, Runtian Xu, Lei Zhang, Denghang Huang, Chenghao Jin, Anlan Yin, Xinlong Wang, Zhenguo Sun, Junkai Zhao, Mengfei Du, Mingyu Cao, Xiansheng Chen, Hongyang Cheng, Xiaojie Zhang, Yankai Fu, Ning Chen, Cheng Chi, Sixiang Chen, Huaihai Lyu, Xiaoshuai Hao, Yequan Wang, Bo Lei, Dong Liu, Xi Yang, Yance Jiao, Tengfei Pan, Yunyan Zhang, Songjing Wang, Ziqian Zhang, Xu Liu, Ji Zhang, Caowei Meng, Zhizheng Zhang, Jiyang Gao, Song Wang, Xiaokun Leng, Zhiqiang Xie, Zhenzhen Zhou, Peng Huang, Wu Yang, Yandong Guo, Yichao Zhu, Suibing Zheng, Hao Cheng, Xinmin Ding, Yang Yue, Huanqian Wang, Chi Chen, Jingrui Pang, YuXi Qian, Haoran Geng, Lianli Gao, Haiyuan Li, Bin Fang, Gao Huang, Yaodong Yang, Hao Dong, He Wang, Hang Zhao, Yadong Mu, Di Hu, Hao Zhao, Tiejun Huang, Shanghang Zhang, Yonghua Lin, Zhongyuan Wang and Guocai Yao},
    journal={arXiv preprint arXiv:2511.17441},
    url = {https://arxiv.org/abs/2511.17441},
    year={2025}
}
```

## License

Released under the **RLWRLD Model License v2.0**, a non-commercial license with attribution and share-alike terms. See [`LICENSE.md`](https://huggingface.co/RLWRLD/RLDX-1.1-FT-ROBOCASA/blob/main/LICENSE.md) for the full text. By using this model you agree to those terms, including the use restrictions in §3.5. The Qwen3-VL component remains subject to the Apache License 2.0 as described in [`NOTICE`](https://huggingface.co/RLWRLD/RLDX-1.1-FT-ROBOCASA/blob/main/NOTICE).