dobottrack / README.md
dddsky's picture
Release DobotTrack model weights
70e7cb8
|
Raw History Blame Contribute Delete
3.6 kB
metadata
language:
  - en
base_model: OpenGVLab/InternVL3-1B-hf
tags:
  - robotics
  - vision-language-action
  - embodied-person-tracking
  - internvl3
  - q-identity
  - pytorch
  - arxiv:2611.09041

DobotTrack

Target-Grounded Vision-Language-Action Learning for Embodied Person Tracking

GitHub Code · Project Page · Paper on arXiv

About

DobotTrack is a target-grounded vision-language-action model for embodied person tracking. Given a natural-language description of the target person and a temporal window of RGB observations, it predicts future egocentric waypoints for following the target.

The model combines complementary mask and bounding-box supervision with Q-identity, a target-conditioned history compressor. Cached target-box predictions guide historical target-token selection, while the current frame retains dense visual features.

This release uses InternVL3-1B with Q-identity, box grounding and a SAM2 mask head. Its configuration specifies:

Setting Value
RGB input 448 × 448
Observation window Seven historical frames plus the current frame
Historical stride 2
Historical compression 256 → 16 tokens per historical frame
Current-frame visual tokens 256
Q-identity 3 blocks, 512 hidden channels, 8 attention heads
Task-query order Mask → Box → eight Action queries
Trajectory output Eight future (x, y, yaw) waypoints in the current ego frame

Model files

File Description
model.ckpt Released PyTorch inference checkpoint for the learned DobotTrack policy, approximately 1.05 GB. Load it with the inference code in the GitHub repository.
config.yaml Companion model and observation configuration. Uses mode: q-identity, bbox-guided history selection and Box-before-Action queries. Download it together with the checkpoint.
SHA256SUMS SHA-256 checksums for model.ckpt, config.yaml and this model card.

The external InternVL3-1B-hf base model and SAM2.1 Base+ checkpoint are not bundled in model.ckpt. Prepare them separately and set model.model_path and model.sam2_config.checkpoint in config.yaml to their local paths.

Download

from huggingface_hub import hf_hub_download

for filename in ("model.ckpt", "config.yaml"):
    hf_hub_download(
        repo_id="dddsky/dobottrack",
        filename=filename,
        local_dir="outputs/dobottrack/checkpoints",
    )

Use the downloaded checkpoint and YAML with the repository's Habitat evaluation or deployment workflow.

Citation

@article{deng2026dobottrack,
  title={DobotTrack: Target-Grounded Vision-Language-Action Learning for Embodied Person Tracking},
  author={Shengjun Deng and Yinghan Zhang and Chengrui Wei and Liu Liang and Songyuan Liu and Pengkun Zheng and Yajun Zheng and Ying Zhang and Yuan Tian and Xi Li and Teng Wang},
  journal={arXiv preprint arXiv:2611.09041},
  year={2026},
  url={https://arxiv.org/abs/2611.09041}
}