| --- |
| license: other |
| license_name: component-specific-licenses |
| license_link: https://github.com/pm1255/human-robot-transfer/blob/main/docs/MODEL_RELEASE.md |
| language: |
| - en |
| - zh |
| tags: |
| - robotics |
| - hand-pose-estimation |
| - image-segmentation |
| - depth-estimation |
| - model-collection |
| --- |
| |
| # Embodied Data Loop — Model Assets |
|
|
| **配套代码 / Companion code: [pm1255/human-robot-transfer](https://github.com/pm1255/human-robot-transfer).** |
|
|
| This repository collects third-party pretrained assets used by the Human ↔ Robot |
| Transfer Lab annotation pipelines. These are upstream models, not a newly |
| trained end-to-end model. Each component retains its authorship and license. |
| The current file list and `manifest.json` describe which files have actually |
| been included; a model collection is not loaded through one Transformers call. |
|
|
| ## 从模型找到 GitHub 脚本 |
|
|
| | 模型 / Component | 配套 GitHub 脚本 | 功能 | |
| |---|---|---| |
| | MediaPipe | [tools/run.py](https://github.com/pm1255/human-robot-transfer/blob/main/tools/run.py), [run_mediapipe_hand.py](https://github.com/pm1255/human-robot-transfer/blob/main/human2robot/v2/run_mediapipe_hand.py) | 手/人体关键点与基础检测 | |
| | HaMeR | [hand_mesh/infer.py](https://github.com/pm1255/human-robot-transfer/blob/main/human2robot/hand_mesh/infer.py), [identity.py](https://github.com/pm1255/human-robot-transfer/blob/main/human2robot/hand_mesh/identity.py), [render.py](https://github.com/pm1255/human-robot-transfer/blob/main/human2robot/hand_mesh/render.py) | 三维手型、身份修复与表面渲染 | |
| | SAM 3.1 | [sam3_video.py](https://github.com/pm1255/human-robot-transfer/blob/main/human2robot/object_annotation/sam3_video.py), [unified_multimodal.py](https://github.com/pm1255/human-robot-transfer/blob/main/human2robot/object_annotation/unified_multimodal.py) | 视频分割与掩码派生框 | |
| | MoGe-3 ViT-L | [depth_video.py](https://github.com/pm1255/human-robot-transfer/blob/main/human2robot/object_annotation/depth_video.py) | 单目深度 | |
| | FoundationPose | [foundation_cloth.py](https://github.com/pm1255/human-robot-transfer/blob/main/human2robot/object_annotation/foundation_cloth.py) | 物体位姿 / 衣物刚体代理试验 | |
| | 质量检查（无独立权重） | [confidence.py](https://github.com/pm1255/human-robot-transfer/blob/main/human2robot/object_annotation/confidence.py), [batch30_validate.py](https://github.com/pm1255/human-robot-transfer/blob/main/local_annotation_lab/unified/batch30_validate.py) | 一致性筛查与完整性检查 | |
| | 统一标注网页 | [local_annotation_lab](https://github.com/pm1255/human-robot-transfer/tree/main/local_annotation_lab), [serve_hand_review.py](https://github.com/pm1255/human-robot-transfer/blob/main/tools/serve_hand_review.py) | 查看原视频及各类标注 | |
|
|
| ## Download and verify |
|
|
| ```bash |
| git clone https://github.com/pm1255/human-robot-transfer.git |
| cd human-robot-transfer |
| python -m pip install huggingface_hub |
| python tools/model_assets.py download --component mediapipe --component hamer --component sam31 --component moge3 --component foundationpose |
| python tools/model_assets.py check |
| ``` |
|
|
| Add `--revision <Hub commit>` to pin this repository version. The download tool |
| uses the SHA-256 manifest and installs weights/configuration files into the |
| paths expected by the GitHub scripts. This validates file integrity, not model |
| accuracy. See the [complete setup and usage guide](https://github.com/pm1255/human-robot-transfer/blob/main/docs/MODEL_RELEASE.md) |
| for environment requirements and executable examples. |
|
|
| ## Coverage and required external assets |
|
|
| - The core asset inventory is approximately 7.97 GB, excluding environments, |
| raw datasets, MANO, optional VACE, and robot meshes. |
| - HaMeR needs its checkpoint, adjacent `model_config.yaml`, mean parameters, |
| and **MANO_RIGHT.pkl obtained separately by the user** from |
| [the MANO website](https://mano.is.tue.mpg.de/). MANO is not redistributed here. |
| - FoundationPose requires both networks and their `config.yml` files. |
| - Optional robot-to-human video editing uses the complete official |
| [Wan-AI/Wan2.1-VACE-1.3B](https://huggingface.co/Wan-AI/Wan2.1-VACE-1.3B) |
| snapshot. [setup_vace.sh](https://github.com/pm1255/human-robot-transfer/blob/main/scripts/setup_vace.sh) |
| obtains it from the original repository. |
| - Aloha/G1 retargeting additionally requires robot geometry assets described |
| in [QUICKSTART.md](https://github.com/pm1255/human-robot-transfer/blob/main/docs/QUICKSTART.md). |
| - This project does **not** provide newly trained general semantic segmentation |
| of action timelines, physical contact, affordance, or grasp-prediction weights. |
| Existing timeline labels and 2D contact cues are sample-specific annotations |
| and rules, not ground truth or a separate trained model. |
| |
| ## Licenses and attribution |
| |
| There is no single MIT license for all weights in this collection. Original |
| license texts accompany the collection in `licenses/`. |
| |
| | Component | Original source | License | |
| |---|---|---| |
| | MediaPipe | [Google MediaPipe](https://github.com/google-ai-edge/mediapipe) | Apache-2.0 | |
| | HaMeR | [geopavlakos/hamer](https://github.com/geopavlakos/hamer), checkpoint provenance [Leo-TX/hamer](https://huggingface.co/Leo-TX/hamer) | MIT; MANO separate | |
| | MoGe-3 | [Ruicheng/moge-3-vitl](https://huggingface.co/Ruicheng/moge-3-vitl) | MIT model card; dependencies retain their own licenses | |
| | SAM 3.1 | [facebook/sam3.1](https://huggingface.co/facebook/sam3.1) | SAM License | |
| | FoundationPose | [NVlabs/FoundationPose](https://github.com/NVlabs/FoundationPose) | NVIDIA Source Code License, non-commercial research/evaluation only | |
| |
| The project's own scripts are MIT-licensed. Predictions can contain missed |
| detections, incorrect identity/segmentation, uncalibrated monocular geometry, |
| and uncertain semantic boundaries. Quality scores are screening evidence, not |
| calibrated probabilities or automatic training approval. |
| |
