new FACET-0 post-training model upload
#2
by KevinGuo07 - opened
- README.md +77 -0
- facet0-post-training/config.json +11 -7
README.md
CHANGED
|
@@ -1,3 +1,80 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
pipeline_tag: robotics
|
| 4 |
+
tags:
|
| 5 |
+
- robotics
|
| 6 |
+
- vision-language-action
|
| 7 |
+
- openpi
|
| 8 |
---
|
| 9 |
+
|
| 10 |
+
# Facet-0
|
| 11 |
+
|
| 12 |
+
Model weights for [Facet-0: A Robotic Foundation Model for Contact-Rich Precise Manipulation](https://arxiv.org/abs/2609.01596).
|
| 13 |
+
|
| 14 |
+
## `facet0-post-training`
|
| 15 |
+
|
| 16 |
+
Policy produced by Facet-0 post-training (value-guided policy refinement). A PaliGemma vision-language backbone is paired with a flow-matching action expert. The kinematic state and a causal wrench history enter only the action expert, which generates each 50-step action chunk together with the wrist wrench it is expected to induce. Only the action is executed. During refinement the vision-language backbone is frozen.
|
| 17 |
+
|
| 18 |
+
| File | Content |
|
| 19 |
+
|---|---|
|
| 20 |
+
| `params/` | weights (orbax checkpoint; frozen backbone stored in bfloat16) |
|
| 21 |
+
| `assets/norm_stats.json` | normalization statistics used at inference |
|
| 22 |
+
| `config.json` | input / output description |
|
| 23 |
+
|
| 24 |
+
### Inputs
|
| 25 |
+
|
| 26 |
+
| Key | Shape | Content |
|
| 27 |
+
|---|---|---|
|
| 28 |
+
| `observation/state` | float32 `[13]` | end-effector pose (6: x, y, z, rotation vector), gripper (1), wrist wrench (6) |
|
| 29 |
+
| `observation/wrench_history` | float32 `[10, 6]` | latest 10 measured wrist wrenches, oldest first |
|
| 30 |
+
| `observation/image` | uint8 `[H, W, 3]` | third-person view |
|
| 31 |
+
| `observation/wrist_image` | uint8 `[H, W, 3]` | wrist camera |
|
| 32 |
+
| `observation/extra_view_image` | uint8 `[H, W, 3]` | second third-person view |
|
| 33 |
+
| `prompt` | string | natural-language sub-task instruction |
|
| 34 |
+
|
| 35 |
+
### Outputs
|
| 36 |
+
|
| 37 |
+
| Key | Shape | Content |
|
| 38 |
+
|---|---|---|
|
| 39 |
+
| `actions` | `[50, 7]` | absolute end-effector pose + gripper; the only commanded output |
|
| 40 |
+
| `wrench` | `[50, 6]` | anticipated wrist wrench (N, N m); predictive, never commanded |
|
| 41 |
+
|
| 42 |
+
### Usage
|
| 43 |
+
|
| 44 |
+
The checkpoint loads with the Facet-0 post-training code, which builds on [openpi](https://github.com/Physical-Intelligence/openpi).
|
| 45 |
+
|
| 46 |
+
```python
|
| 47 |
+
import facet0.policy
|
| 48 |
+
|
| 49 |
+
policy = facet0.policy.load("facet0-post-training") # params/ + assets/norm_stats.json
|
| 50 |
+
history = facet0.policy.WrenchHistory() # latest 10 wrenches
|
| 51 |
+
history.push(state[7:13])
|
| 52 |
+
out = policy.infer({
|
| 53 |
+
"observation/state": state,
|
| 54 |
+
"observation/wrench_history": history.window(),
|
| 55 |
+
"observation/image": view1,
|
| 56 |
+
"observation/wrist_image": hand,
|
| 57 |
+
"observation/extra_view_image": view2,
|
| 58 |
+
"prompt": "press down the RAM until it locks into place",
|
| 59 |
+
})
|
| 60 |
+
out["actions"] # [50, 7]: send to the controller
|
| 61 |
+
out["wrench"] # [50, 6]: for logging
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
`guidance_scale` trades between the unconditional branch (0.0) and the label-conditioned branch (1.0, default).
|
| 65 |
+
|
| 66 |
+
## Citation
|
| 67 |
+
|
| 68 |
+
```bibtex
|
| 69 |
+
@article{deng2026facet0,
|
| 70 |
+
title = {Facet-0: A Robotic Foundation Model for Contact-Rich Precise Manipulation},
|
| 71 |
+
author = {Deng, Haoyuan and Liu, Haichao and Guo, Wenkai and Ling, Yuan and Yang, Zaijia and
|
| 72 |
+
Xue, Yuanjiang and Sun, Haosheng and Wang, Liangzi and Wang, Ziwei},
|
| 73 |
+
journal = {arXiv preprint arXiv:2609.01596},
|
| 74 |
+
year = {2026}
|
| 75 |
+
}
|
| 76 |
+
```
|
| 77 |
+
|
| 78 |
+
## License
|
| 79 |
+
|
| 80 |
+
Apache-2.0. The model builds on openpi (Apache-2.0).
|
facet0-post-training/config.json
CHANGED
|
@@ -1,11 +1,19 @@
|
|
| 1 |
{
|
| 2 |
-
"model_name": "Post-training model for FACET-0, which includes wrench-aware value estimation
|
| 3 |
"model_type": "vision-language-action",
|
| 4 |
"license": "apache-2.0",
|
| 5 |
"library": "openpi",
|
| 6 |
"observation": {
|
| 7 |
"state_dim": 13,
|
| 8 |
-
"state_layout": "end-effector pose (
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
"images": [
|
| 10 |
"observation/image",
|
| 11 |
"observation/wrist_image",
|
|
@@ -15,11 +23,7 @@
|
|
| 15 |
},
|
| 16 |
"action": {
|
| 17 |
"horizon": 50,
|
| 18 |
-
"executed_channels": "absolute end-effector pose + gripper"
|
| 19 |
-
"gripper_convention": {
|
| 20 |
-
"0": "closed",
|
| 21 |
-
"1": "open"
|
| 22 |
-
}
|
| 23 |
},
|
| 24 |
"inference": {
|
| 25 |
"guidance_scale_default": 1.0,
|
|
|
|
| 1 |
{
|
| 2 |
+
"model_name": "Post-training model for FACET-0, which includes wrench-aware value estimation, with wrench-conditioned RL goal for evolving on real-world rollouts",
|
| 3 |
"model_type": "vision-language-action",
|
| 4 |
"license": "apache-2.0",
|
| 5 |
"library": "openpi",
|
| 6 |
"observation": {
|
| 7 |
"state_dim": 13,
|
| 8 |
+
"state_layout": "end-effector pose (6: x, y, z, rotation vector) + gripper (1) + six-axis wrist wrench (6: force N, torque N m)",
|
| 9 |
+
"wrench_history": {
|
| 10 |
+
"key": "observation/wrench_history",
|
| 11 |
+
"shape": [
|
| 12 |
+
10,
|
| 13 |
+
6
|
| 14 |
+
],
|
| 15 |
+
"layout": "latest 10 measured wrist wrenches (force N, torque N m), oldest first"
|
| 16 |
+
},
|
| 17 |
"images": [
|
| 18 |
"observation/image",
|
| 19 |
"observation/wrist_image",
|
|
|
|
| 23 |
},
|
| 24 |
"action": {
|
| 25 |
"horizon": 50,
|
| 26 |
+
"executed_channels": "absolute end-effector pose + gripper"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
},
|
| 28 |
"inference": {
|
| 29 |
"guidance_scale_default": 1.0,
|