GR00T-N1.5 β RoboCasa Kitchen baseline policy
GR00T-N1.5-3B finetuned on RoboCasa Kitchen (robocasa_mg_gr00t_300, 24 tasks),
batch size 64, 60k steps. This is the uncompressed baseline in the
quantizability-gate work β the policy that produces the 16-step action chunks
which the gate then decides whether to temporally compress.
Closed-loop, 24 tasks Γ 50 episodes each:
| policy | success | avg steps |
|---|---|---|
| this model, uncompressed | 0.657 | 327 |
| + naive K2 (compress every chunk) | 0.598 | 221 |
| + Aβ² gate (gemma4 teacher), Ο=0.5 | 0.667 | 258 |
Benchmark reproducibility noise is Β±1.5 pp.
Which checkpoint do I need?
- Running the gated benchmark with Aβ² β this model plus the small gate
from
prehj/groot-n15-quantizability-gate-A-robocasa. The gate is a separate 1.3 MB CNN; this policy is unmodified. - Architecture C β do not use this model. C is a single self-contained
policy with the gate token built into its DiT:
prehj/groot-n15-quantizability-gate-C-robocasa. - Uncompressed reference numbers β this model alone.
Usage
from gr00t.model.policy import Gr00tPolicy
policy = Gr00tPolicy(model_path="prehj/groot-n15-robocasa-kitchen-baseline",
embodiment_tag="new_embodiment", denoising_steps=4)
chunk = policy.get_action(obs)["action_pred"] # (16, 12)
Then gate and merge:
from gate_api import QuantizabilityGate # from the A' repo
from merge_k2 import merge_k2
gate = QuantizabilityGate("checkpoints/A_robocasa_gemma4.pt",
"checkpoints/robocasa_task_embeddings.npz")
if gate(views, instruction)["confidence"] >= tau:
chunk = merge_k2(chunk, delta_dims=range(5, 11), state_dims=[11])
Ο is an operating point to search or learn, not a constant β see the Aβ² card.
Action space (12 dims)
0β4 unused (always zero) Β· 5β7 EE delta xyz Β· 8β10 rotation Β· 11 gripper
(0/1). Under K2, dims 5β10 sum across the merged pair and dim 11 takes the last
value.
Running the benchmark
Environment setup, the three-process runtime layout, the full task list, how to sweep tau, and the pitfalls that have actually cost us runs are written up here: RoboCasa evaluation setup guide
Related
Code and evaluation harness: https://github.com/rakybond007/GR00T-action-quantization/tree/action-quantization-gate-v2
- Downloads last month
- 7