lastgeyan commited on
Commit
a32bb6d
·
verified ·
1 Parent(s): fdb6695

Model card: simpler inputs wording

Browse files
Files changed (1) hide show
  1. README.md +5 -11
README.md CHANGED
@@ -39,15 +39,9 @@ alongside actions. A single checkpoint runs on any subset of its streams, so the
39
  compute/accuracy operating point is a runtime flag.
40
 
41
  This checkpoint is a large-scale pretrain for bimanual 6-DoF YAM-family robots
42
- with one head camera and two wrist cameras.
43
-
44
- - **It always uses RGB and DINO.** The client sends RGB; the model computes the
45
- DINOv3 features from it.
46
- - **Depth is optional.** With depth and intrinsics the 3D pointmap stream is on as
47
- well; without them the model runs on RGB and DINO alone.
48
- - **Training depth** was DA3 (Depth Anything 3) estimates for ABC-130k, MolmoAct2
49
- and AgiBot World, and the YAM recordings' own sensor depth.
50
-
51
  It is a starting point for fine-tuning; zero-shot use is possible on a matching
52
  rig. The sibling without depth is
53
  [`flex-pi/flexpi-pretrain-rgb-preview`](https://huggingface.co/flex-pi/flexpi-pretrain-rgb-preview).
@@ -58,14 +52,14 @@ rig. The sibling without depth is
58
  |---|---|
59
  | Architecture | Mixture-of-Transformers: Wan2.2-TI2V-5B video expert (5.00 B) + ActionDiT action expert (1.02 B), 30 layers, HBridge 7 / 16 / 7 |
60
  | Cameras | `cam_high` 256×320, `cam_left_wrist` and `cam_right_wrist` 224×224, composited at 384×320 (`robotwin_uniform14`), 33 frames |
61
- | Inputs | three RGB cameras, plus optional metric depth and intrinsics; DINOv3 features are always used, computed by the model from the RGB frames |
62
  | Visual streams | RGB and 3D pointmap through the frozen Wan2.2 VAE; DINOv3 ViT-B/16 (frozen), temporal stride 1 |
63
  | Flex regime | `p_present_* = p_j* = 0.5` for video, DINO and pointmap; cross-modal forcing on |
64
  | Action head | 34-D, shared by 6- and 7-joint arms. Channels 26 and 33 are each arm's 7th joint, which a YAM arm does not have: they are left out of the loss on YAM data and held at 0 when serving a YAM robot, which sends and receives 32-D |
65
  | Wire format | 32-D YAM end-effector state in, 32 × 32 absolute action chunk out |
66
  | Normalization | frame-pooled q01/q99 over 34 channels (`dataset_stats.json`), normalized targets unclamped (`action_norm_clamp: inf`) |
67
  | Training data | ~4,600 hours of robot data: ABC-130k, MolmoAct2, AgiBot World and in-house YAM teleoperation data |
68
- | Training depth | DA3 (Depth Anything 3) estimates for ABC-130k, MolmoAct2 and AgiBot World; the YAM recordings' own sensor depth |
69
  | Checkpoint | `step_190000`, bf16, 12,082,212,409 bytes, sha256 `b168eb68f255b98bf4805ad762dd870e06dd5f0f6f044f3e4280622338799eb2` |
70
 
71
  ## Usage
 
39
  compute/accuracy operating point is a runtime flag.
40
 
41
  This checkpoint is a large-scale pretrain for bimanual 6-DoF YAM-family robots
42
+ with one head camera and two wrist cameras. It takes RGB, with optional depth,
43
+ and always uses DINO, which the model computes from the RGB. Its training depth
44
+ is DA3 for ABC-130k, MolmoAct2 and AgiBot World, and sensor depth for the YAM data.
 
 
 
 
 
 
45
  It is a starting point for fine-tuning; zero-shot use is possible on a matching
46
  rig. The sibling without depth is
47
  [`flex-pi/flexpi-pretrain-rgb-preview`](https://huggingface.co/flex-pi/flexpi-pretrain-rgb-preview).
 
52
  |---|---|
53
  | Architecture | Mixture-of-Transformers: Wan2.2-TI2V-5B video expert (5.00 B) + ActionDiT action expert (1.02 B), 30 layers, HBridge 7 / 16 / 7 |
54
  | Cameras | `cam_high` 256×320, `cam_left_wrist` and `cam_right_wrist` 224×224, composited at 384×320 (`robotwin_uniform14`), 33 frames |
55
+ | Inputs | three RGB cameras, optional depth; DINO is computed from the RGB |
56
  | Visual streams | RGB and 3D pointmap through the frozen Wan2.2 VAE; DINOv3 ViT-B/16 (frozen), temporal stride 1 |
57
  | Flex regime | `p_present_* = p_j* = 0.5` for video, DINO and pointmap; cross-modal forcing on |
58
  | Action head | 34-D, shared by 6- and 7-joint arms. Channels 26 and 33 are each arm's 7th joint, which a YAM arm does not have: they are left out of the loss on YAM data and held at 0 when serving a YAM robot, which sends and receives 32-D |
59
  | Wire format | 32-D YAM end-effector state in, 32 × 32 absolute action chunk out |
60
  | Normalization | frame-pooled q01/q99 over 34 channels (`dataset_stats.json`), normalized targets unclamped (`action_norm_clamp: inf`) |
61
  | Training data | ~4,600 hours of robot data: ABC-130k, MolmoAct2, AgiBot World and in-house YAM teleoperation data |
62
+ | Training depth | DA3 for ABC-130k, MolmoAct2 and AgiBot World; sensor depth for YAM |
63
  | Checkpoint | `step_190000`, bf16, 12,082,212,409 bytes, sha256 `b168eb68f255b98bf4805ad762dd870e06dd5f0f6f044f3e4280622338799eb2` |
64
 
65
  ## Usage