| --- |
| license: apache-2.0 |
| tags: |
| - robotics |
| - imitation-learning |
| - dexflow |
| - residual-mlp |
| - dexterous-manipulation |
| library_name: pytorch |
| --- |
| |
| # DexFlow MLP Stage-2 v2 (M2 / M3 / M4) |
|
|
| Residual MLP Core4 Stage-2 variants on **plastic** and **pill** (30k, seed `20260910`, gate G2). |
|
|
| Org: [Humantwin](https://huggingface.co/Humantwin) |
| Repo: [`Humantwin/dexflow-v2-stage2-tasks`](https://huggingface.co/Humantwin/dexflow-v2-stage2-tasks) |
|
|
| **Not ACT / DiT.** `DexFlowMLPPolicy` (`policy_variant=dexflow_mlp`). |
| Visual backbone: ResNet18 → `Z_int[8×256]` → Residual MLP → action chunk `[30, 41]`. |
|
|
| Stage-1 init: `dexflow_mlp_stage1_core4_current_state_gate_20260910/stage1_step02000.ckpt` |
| Code pack: `dexflow_v2_mlp_20260915.tar.zst` |
|
|
| ## Variants |
|
|
| | Folder | Method | Head | Loss | |
| |---|---|---|---| |
| | `balanced_loss/` | M2 | unified | `0.5 L_body + 0.5 L_hand` | |
| | `split_head/` | M3 | shared trunk + BodyHead/HandHead | same balanced | |
| | `temporal_delta/` | M4 | same as M3 | balanced + `λ_Δ=0.1` hand temporal-delta | |
|
|
| Each variant has `plastic/` and `pill/`: |
|
|
| ```text |
| {variant}/{task}/stage2.ckpt |
| {variant}/{task}/norm_stats.json |
| {variant}/{task}/gate_final.json |
| ``` |
|
|
| ## Open-loop (train-set, G2 PASS @ 30k) |
|
|
| | Path | nRMSE | body RMSE | hand MAE | |
| |---|---:|---:|---:| |
| | `balanced_loss/plastic` | 0.0518 | 0.0053 | 2.48 | |
| | `balanced_loss/pill` | 0.0592 | 0.0065 | 3.74 | |
| | `split_head/plastic` | 0.0465 | 0.0048 | 2.48 | |
| | `split_head/pill` | 0.0501 | 0.0057 | 3.41 | |
| | `temporal_delta/plastic` | 0.0473 | 0.0048 | 2.37 | |
| | `temporal_delta/pill` | 0.0516 | 0.0058 | 3.55 | |
|
|
| ## Load |
|
|
| ```python |
| from deploy.real_robot.load_p0_policy import load_dexflow_policy |
| from dexflow.data.normalize import ActionStateNormalizer |
| |
| model = load_dexflow_policy("split_head/plastic/stage2.ckpt", device="cuda") |
| normalizer = ActionStateNormalizer.load("split_head/plastic/norm_stats.json") |
| # image [B,1,3,480,640] float 0-1; state [B,41] MEAN_STD-normalized |
| chunk_n = model.predict_action_chunk({"image": img, "state": state_n}) |
| action = normalizer.unnormalize_action(chunk_n) # [B,30,41] |
| ``` |
|
|
| `from_checkpoint` restores `mlp_cfg.split_head` and `lambda_delta` from the ckpt. |
|
|
