Update model card: overview, output conventions, vLLM, eval, training

README.md

@@ -27,30 +27,42 @@ base_model:
   🤗 <a href="https://huggingface.co/collections/IffYuan/embodied-r15">Models & Datasets</a>
 </p>
 
-**Embodied-R1.5** is a unified **Embodied Foundation Model (EFM)** that integrates comprehensive embodied reasoning capabilities, spanning embodied cognition, task planning, correction, and pointing, within a single 8B-parameter architecture toward general physical intelligence.
+> *"Reasoning initiates the action; Action fulfills the reasoning."* — Wang Yangming (1509)
 
-Built on the paradigm of our prior work [Embodied-R1](https://embodied-r.github.io/), Embodied-R1.5 leaps from a pointing specialist to a comprehensive EFM that unifies three capability dimensions in one model. Leveraging three automated data construction pipelines, we build a large-scale data system of over 15B tokens and design a multi-task balanced RL recipe to alleviate heterogeneous task conflicts. We further introduce a **Planner-Grounder-Corrector (PGC)** closed-loop framework that enables a single model to autonomously execute and self-correct over long-horizon tasks.
+## Overview
 
-## Highlights
+**Embodied-R1.5** is a unified **Embodied Foundation Model (EFM)**, built on **Qwen3-VL-8B-Instruct**, that integrates comprehensive embodied reasoning within a single architecture. Building on our prior work [Embodied-R1](https://github.com/pickxiguapi/Embodied-R1), it leaps from a pointing specialist to a comprehensive EFM unifying **three core capabilities**:
 
-- **Unified embodied capability system.** A single 8B model unifies three capability dimensions: Cognition & Spatial Reasoning, Planning & Correction, and Pointing & Location.
-- **State-of-the-art performance.** Achieves SOTA on **16 out of 24** embodied VLM benchmarks, with an average score of **70.4%** across 21 main accuracy-based benchmarks, surpassing Gemini-Robotics-ER-1.5 and GPT-5.4 by 17.0% and 21.7% respectively.
-- **Closed-loop autonomy.** The PGC framework lets one model serve as planner, grounder, and corrector simultaneously, completing long-horizon real-world tasks (e.g., making milk tea, sweeping garbage, stacking cups) without human intervention.
-- **Efficient adaptation to action.** Because embodied reasoning is internalized upstream, the model can be fine-tuned into **Embodied-R1.5-VLA** with only a small amount of action data, outperforming strong VLA baselines such as $\pi_{0.5}$ across 4 popular manipulation benchmark suites (e.g., 92.4% on SimplerEnv Google Robot Visual Matching).
-- **Fully open-source.** We release model weights, datasets, training code, and EmbodiedEvalKit, an evaluation framework tailored for embodied tasks.
+- **Spatial cognition & reasoning** — comprehend the semantic and spatial structure of the physical world, including static geometric relations and dynamic interaction possibilities.
+- **Task planning & correction** — cover the full task life cycle: long-horizon decomposition, next-step planning, process detection, error localization, and correction.
+- **Embodied pointing & location** — ground high-level reasoning in coordinates and trajectories, covering referring expression grounding, region-level localization, functional (affordance) grounding, and visual trace generation.
+
+Trained on a 15B-token corpus with a multi-task balanced RL recipe, it further drives a **Planner-Grounder-Corrector (PGC)** closed-loop framework where one model acts as planner, grounder, and corrector to autonomously complete long-horizon real-world tasks. With only 8B parameters, Embodied-R1.5 is best on **16 of 24** embodied VLM benchmarks (avg. **70.4%**), surpassing Gemini-Robotics-ER-1.5 and GPT-5.4; with light action-data fine-tuning it adapts into **Embodied-R1.5-VLA**, outperforming strong baselines like $\pi_{0.5}$ across 4 manipulation benchmark suites; and it generalizes zero-shot to real robots on instruction following, affordance grounding, articulated manipulation, and long-horizon tasks.
 
 ## Model Details
 
 - **Architecture:** Qwen3-VL (`Qwen3VLForConditionalGeneration`)
 - **Parameters:** ~8B
+- **Base model:** Qwen/Qwen3-VL-8B-Instruct
 - **Modality:** Image / Video + Text → Text
-- **Output format:** All outputs are plain-text token sequences. Coordinates are normalized to $[0, 1000]$, trajectories are ordered coordinate sequences, and reasoning is free-form text. The final decision is emitted within an `<answer>...</answer>` tag.
+- **Output format:** All outputs are plain-text token sequences, with the final decision emitted within an `<answer>...</answer>` tag.
+
+## Output Conventions
 
-### Unified Capabilities
+Embodied-R1.5 follows the Qwen3-VL chat format and outputs structured answers inside `<answer>...</answer>` tags. The supported task types and their answer formats are:
 
-1. **Embodied Cognition & Spatial Reasoning** — comprehends the semantic and spatial structure of the physical world, including static geometric relations and dynamic interaction possibilities.
-2. **Embodied Planning & Correction** — covers the full task life cycle: long-horizon task decomposition, next-step planning, process detection, error localization, and error correction.
-3. **Embodied Pointing & Location** — grounds high-level reasoning in coordinates and trajectories, covering referring expression grounding, region-level localization, functional (affordance) grounding, and visual trace generation.
+| Task Type | Answer Format (inside `<answer>`) |
+|-----------|-----------------------------------|
+| `multiple choice` | `A` |
+| `numerical` | `3.14` |
+| `open-ended` | free text |
+| `math` | `$$-\dfrac{3}{2}$$` |
+| `spatial grounding` | `{"boxes": [35, 227, 437, 932]}` |
+| `point` | ` ```json\n[{"point_2d": [230, 138]}]\n``` ` |
+| `trace` | ` ```json\n[{"point_2d": [624, 469]}, ...]\n``` ` |
+| `trace_3d` | ` ```json\n[{"point_2d": [463, 599], "depth": 1.08}, ...]\n``` ` |
+
+> **Coordinate & unit conventions.** All points (`point_2d`) and boxes are normalized to the `[0, 1000]` range, regardless of the original image resolution. For `trace_3d`, the `depth` value is in meters.
 
 ## Quick Start
 
@@ -85,13 +97,57 @@ print(processor.batch_decode(out, skip_special_tokens=True)[0])
 
 The model reasons over the visual observation and emits its final decision within an `<answer>` tag, e.g. `<answer>[{"point_2d": [750, 748]}]</answer>`.
 
+## Inference with vLLM (Recommended)
+
+For higher throughput, serve the model with vLLM:
+
+```bash
+vllm serve IffYuan/Embodied-R1.5 \
+  --served-model-name "Embodied-R1.5" \
+  --tensor-parallel-size 1 \
+  --mm-encoder-tp-mode data \
+  --gpu-memory-utilization 0.7 \
+  --async-scheduling \
+  --media-io-kwargs '{"video": {"num_frames": 32}, "image": {"max_num": 32}}' \
+  --max_model_len 20000 \
+  --limit-mm-per-prompt '{"image": 8, "video": 1}' \
+  --host 0.0.0.0 --port 22002
+```
+
+More runnable examples (vLLM online / offline, HuggingFace, point decoding & visualization) are provided in the [GitHub repository](https://github.com/pickxiguapi/Embodied-R1.5) under `inference/`.
+
+## Evaluation
+
+For benchmark evaluation, see [EmbodiedEvalKit](https://github.com/pickxiguapi/EmbodiedEvalKit), our evaluation framework covering 25+ embodied benchmarks.
+
+## Training & Data
+
+Embodied-R1.5 is trained in two stages: SFT (LLaMA-Factory) followed by RFT (EasyR1). Full training scripts are available in the [GitHub repository](https://github.com/pickxiguapi/Embodied-R1.5). Datasets are released in the [Embodied-R1.5 HuggingFace collection](https://huggingface.co/collections/IffYuan/embodied-r15).
+
 ## Citation
 
+If you find Embodied-R1.5 useful in your research, please cite our work:
+
 ```bibtex
 @article{yuan2026embodiedr15,
-  title   = {Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models},
-  author  = {Yuan, Yifu and others},
-  year    = {2026}
+  title={Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models},
+  author={Yuan, Yifu and Huang, Yaoting and Yao, Xianze and Zhang, Shuoheng and Han, Linqi and Li, Yutong and Li, Pengyi and Sun, Jiangeng and Jia, Wenting and Hu, Yucheng and Liu, Yuhao and Liao, Ruihao and Wu, Qiyu and Li, Yuxiao and Zhang, Zhao and Dong, Zibin and Ni, Fei and Zheng, Yan and Gu, Shuyang and Ma, Yi and Tang, Hongyao and Hu, Han and Hao, Jianye},
+  journal={arXiv preprint},
+  year={2026}
+}
+
+@article{yuan2025embodied,
+  title={Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulation},
+  author={Yuan, Yifu and Cui, Haiqin and Huang, Yaoting and Chen, Yibin and Ni, Fei and Dong, Zibin and Li, Pengyi and Zheng, Yan and Hao, Jianye},
+  journal={ICLR 2026},
+  year={2025}
+}
+
+@article{yuan2025seeing,
+  title={From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation},
+  author={Yuan, Yifu and Cui, Haiqin and Chen, Yibin and Dong, Zibin and Ni, Fei and Kou, Longxin and Liu, Jinyi and Li, Pengyi and Zheng, Yan and Hao, Jianye},
+  journal={ICLR 2026},
+  year={2025}
 }
 ```