--- license: apache-2.0 library_name: transformers base_model: - Qwen/Qwen3.5-9B - Qwen/Qwen3.8-27B pipeline_tag: video-text-to-text tags: - robotics - atomic-actions - temporal-action-segmentation - video --- # RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning ![RoboAtom overview](https://huggingface.co/datasets/InternRobotics/RoboAtom/resolve/main/project_media/figures/roboatom-overview.png) **RoboAtom-Anno · 9B and 27B video annotation models** ![Project page](https://huggingface.co/datasets/InternRobotics/RoboAtom/resolve/main/project_media/badges/website.svg)![GitHub](https://huggingface.co/datasets/InternRobotics/RoboAtom/resolve/main/project_media/badges/code.svg)![Paper](https://huggingface.co/datasets/InternRobotics/RoboAtom/resolve/main/project_media/badges/paper.svg)![Dataset](https://huggingface.co/datasets/InternRobotics/RoboAtom/resolve/main/project_media/badges/data.svg)![Models](https://huggingface.co/datasets/InternRobotics/RoboAtom/resolve/main/project_media/badges/models.svg) [Models](#models) · [Download](#download) · [Inference](#inference) · [Output and evaluation](#output-and-evaluation) · [License and citation](#license-and-citation) ## Overview RoboAtom-Anno predicts atomic actions from robot videos and task instructions. Each predicted segment contains a **skill label**, an **action description**, and **frame boundaries**. The release provides merged 9B and 27B weights, with tokenizers, video processors, and generation settings. ## Models | Model | Weights | Architecture | Precision | | --------------------- | ------------------------------------------------------------------------------------------------ | --------------------------- | --------- | | **RoboAtom-Anno-9B** | [RoboAtom-Anno-9B/](https://huggingface.co/InternRobotics/RoboAtom/tree/main/RoboAtom-Anno-9B) | Fine-tuned from Qwen3.5-9B | BF16 | | **RoboAtom-Anno-27B** | [RoboAtom-Anno-27B/](https://huggingface.co/InternRobotics/RoboAtom/tree/main/RoboAtom-Anno-27B) | Fine-tuned from Qwen3.8-27B | BF16 | Use Python **3.11+**, `transformers==5.8.0`, and compatible CUDA-enabled PyTorch. The [RoboAtom inference implementation](https://github.com/InternRobotics/RoboAtom#quick-start-benchmark) handles synchronized camera composition, frame sampling, and frame-index prompts. GPU requirements depend on the model and video input. ## Download These examples use `./model` for weights and `./data` for the benchmark: ```bash hf download InternRobotics/RoboAtom --repo-type model \ --include 'RoboAtom-Anno-9B/**' --local-dir ./model hf download InternRobotics/RoboAtom --repo-type dataset \ --include 'RoboAtom-Bench/**' --local-dir ./data ``` Replace `RoboAtom-Anno-9B` with `RoboAtom-Anno-27B` to select the larger model. The Hugging Face CLI (`hf`) is required for these download commands. ## Inference ### 1. Install the code From your [RoboAtom code checkout](https://github.com/InternRobotics/RoboAtom): ```bash python3.11 -m venv .venv source .venv/bin/activate python -m pip install -e '.[inference]' ``` ### 2. Start the model service The following paths match the download layout above; replace them if you chose another location: ```bash roboatom serve \ --model ./model/RoboAtom-Anno-9B \ --media-root ./data/RoboAtom-Bench ``` The service listens on `http://127.0.0.1:3000`. Change the model directory to select the 27B weights. ### 3. Generate predictions In another terminal with the same environment activated: ```bash roboatom infer \ --input ./data/RoboAtom-Bench/benchmark.jsonl \ --output outputs/roboatom_predictions.jsonl \ --base-url http://127.0.0.1:3000 ``` Choose a new output filename for each run; the inference client refuses to overwrite an existing prediction file. | Setting | Released inference behavior | | --------------- | ------------------------------------------------------------------- | | Sampling | Uniform, target 2 FPS, including both endpoints, at most 448 frames | | Precision | BF16 | | Generation | Deterministic, at most 4,096 new tokens | | Time convention | Original-video frame indices, zero-based, both endpoints included | ## Output and evaluation The client writes one prediction record per benchmark input, with an identity key and ordered action segments. A segment covers `start_frame` through `end_frame` inclusively; frames 0–74 contain 75 frames. Keep ground truth separate from model inputs. From the code checkout, score the predictions: ```bash roboatom evaluate \ --gt ./data/RoboAtom-Bench/gt.jsonl \ --benchmark-input ./data/RoboAtom-Bench/benchmark.jsonl \ --prediction outputs/roboatom_predictions.jsonl \ --protocol configs/roboatom_bench.yaml \ --output-dir outputs/roboatom_evaluation ``` The scorer writes `metrics.json`, `summary.tsv`, and `episodes.jsonl`. See the [benchmark card](https://huggingface.co/datasets/InternRobotics/RoboAtom) for data fields and the [code README](https://github.com/InternRobotics/RoboAtom#evaluation) for the evaluation protocol. ## License and citation The model directories contain the [9B Apache-2.0 license](https://huggingface.co/InternRobotics/RoboAtom/blob/main/RoboAtom-Anno-9B/LICENSE) and [27B Apache-2.0 license](https://huggingface.co/InternRobotics/RoboAtom/blob/main/RoboAtom-Anno-27B/LICENSE). See the project's [third-party notices](https://github.com/InternRobotics/RoboAtom/blob/main/THIRD_PARTY_NOTICES.md) for component attribution and [citation file](https://github.com/InternRobotics/RoboAtom/blob/main/CITATION.cff) for publication metadata. If you use RoboAtom in your research, please cite: ```bibtex @misc{wei2026roboatomlargescaleatomicactionsupervision, title={RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning}, author={Xueyuan Wei and Xuanxu Wang and Xudong Li and Yuping He and Wanshui Gan and Jiaqi Peng and Wenqi Guo and Jianjun Zhou and Junting Dong and Chunhua Shen and Haoyu Guo and Mulin Yu and Tao Lu}, year={2026}, eprint={2610.40207}, archivePrefix={arXiv}, primaryClass={cs.RO}, url={https://arxiv.org/abs/2610.40207}, } ```