Add section 11 (RoboCasa zero-shot transfer); extend section 10 with pretrain_C fine-tune + B-vs-C head-to-head

ablation.md

@@ -8,7 +8,7 @@ All closed-loop numbers below are **paired**: every configuration runs the same
 
 Hardware: 4x Tesla V100-SXM2-32GB (sm_70), fp32, driver 535 / CUDA 12.2,
 torch 2.6.0+cu124, transformers 5.5.4, lerobot 0.6.1. Suite: `libero_spatial`.
-Generated 2026-08-24.
+Generated 2026-08-25.
 
 **The headline dual-rate comparison in the model card is not reproduced.** With the
 clock fixed and 500 paired episodes per configuration, no refresh rate differs
@@ -363,7 +363,7 @@ shared pretrain.
 
 ---
 
-## 10. Fine-tuning pretrain_B on LIBERO
+## 10. Fine-tuning pretrain_B and pretrain_C on LIBERO
 
 Section 9's `pretrain_B` row asks what a cross-embodiment pretrain is worth on
 LIBERO with no fine-tuning at all (nothing - 0%, same as every other pre-fine-tune
@@ -374,7 +374,8 @@ instead of `stage3_distill`?
 
 **Recipe, unmodified from `configs/libero_ft4.yaml` except the resume point:**
 unfreeze vision tower + language stack, backbone at 0.1x lr, 40,000 steps,
-`resume_from: tinyvla2/pretrain_B`. `num_embodiments=16` and pretrain_B's
+`resume_from: tinyvla2/pretrain_B` (and separately `pretrain_C`).
+`num_embodiments=16` and each pretrain's
 `conditioning`/`morph_tokens`/`action_space` fields carried through so the resume
 loads with 0 missing / 0 unexpected keys (verified directly before launching).
 fp16 instead of bf16 (V100, sm_70); effective batch 48 via 4-way data-parallel
@@ -382,13 +383,22 @@ instead of one A6000, matching ft4's effective batch. Final training loss 0.178,
 comparable to ft4's reported 0.066 - 500-episode full-rate closed-loop eval,
 paired against the same episodes as ft4's own full-rate row (section 3).
 
+`pretrain_C` got the identical treatment. One caveat specific to C: its
+`morph_encoder` conditioning has **no forward-pass implementation** in this
+codebase, so a zero-shot evaluation of C would silently measure a *different*
+model than the one the checkpoint describes, and none is reported here. Fine-tuning
+is still legitimate — the backbone unfreezes either way, so `embodiment_emb`
+receives real gradient — and that is what the C row below measures.
+
 | | Success | 95% CI | delta vs. ft4 | paired 95% CI | flips | McNemar p |
 |---|---|---|---|---|---|---|
 | ft4 (from stage3_distill) | 426/500 = 85.2% | — | — | — | — | — |
 | **B fine-tuned (from pretrain_B)** | 345/500 = 69.0% | [64.8,72.9] | -16.2 | [-21.0,-11.4] | +40/-121 | 0.0000*** |
+| **C fine-tuned (from pretrain_C)** | 372/500 = 74.4% | [70.4,78.0] | -10.8 | [-15.2,-6.6] | +36/-90 | 0.0000*** |
 
-**pretrain_B does not make a better starting point than stage3_distill for LIBERO
-specifically - it makes a worse one, by a wide and significant margin.** This isn't
+**Neither cross-embodiment pretrain makes a better starting point than
+stage3_distill for LIBERO specifically - both make a worse one, by wide and
+significant margins.** Head-to-head on the same 500 paired episodes, **C beats B by +5.4 pp** (paired 95% CI [+0.0,+10.8], McNemar p=0.0623, +111/-84 flips) - which does **not** clear significance at 0.05, so the two cross-embodiment pretrains are not separated by this evidence. This isn't
 one catastrophic task dragging an average down: 7/10 tasks score lower for the B-derived fine-tune than for ft4 (the other 3 are within a couple of episodes either way). Worst-hit: task8 at 9/50 vs. ft4's 39/50. Best-preserved: task5 at 49/50 vs. ft4's 42/50. Both checkpoints have identical architecture and parameter/FLOP counts (confirmed via the same profiling harness as section 6) - this is purely a pretraining-lineage effect, not a compute or capacity difference.
 
 Plausible reason: `stage3_distill` was trained specifically as *this* model's
@@ -405,6 +415,80 @@ fine-tuning starts, per section 1's frame-rate check applying identically to
 both - the difference is what the backbone learned before LIBERO ever entered
 the picture).
 
+
+---
+
+## 11. Zero-shot transfer to RoboCasa (a genuinely different domain)
+
+Everything above is LIBERO: one fixed tabletop Panda, one scene layout family.
+This section asks whether `ft4` transfers with no fine-tuning at all to
+[RoboCasa365](https://robocasa.ai/) — PandaOmron (Franka arm on a holonomic
+mobile base) in procedurally-varied kitchens, a different robot, different
+scenes, and a different observation layout.
+
+**Conventions were verified empirically, not assumed** (`verify_robocasa.py`,
+`verify_robocasa_axes.py`), because a silent frame mismatch here would be
+indistinguishable from a policy failure:
+
+* **Gripper**: RoboCasa's -1=open/+1=closed matches LIBERO's polarity directly
+  (measured qpos 0.039 at -1, 0.001 at +1). No sign flip.
+* **Action**: both use the same OSC_POSE delta controller at the same physical
+  scaling (±0.05 m, ±0.5 rad/step), so LIBERO's unnormalized
+  `[dpos(3), drot(3), grip(1)]` maps onto RoboCasa's `ee_pos/ee_rot/gripper`
+  slots. `base_motion(4)` and `control_mode(1)` held at zero — arm-only.
+* **State**: RoboCasa's native 16-dim `agent_pos` is a *different layout*
+  (base pose + relative EE) from the trained convention (absolute EE pose +
+  gripper), converted via `ee_state_from_agent_pos` before normalizing. Feeding
+  the raw vector through would put a base quaternion where the model expects an
+  EE rotvec.
+* **Cameras**: `robot0_agentview_left` + `robot0_eye_in_hand`, used raw. A
+  crosshair projection through the camera's own MuJoCo extrinsics shows +y and
+  +z already match LIBERO's screen convention (+y→left, +z→up), while +x is a
+  *diagonal* blend of both screen axes rather than LIBERO's clean "+x→down" —
+  this camera looks into the kitchen at an angle instead of down a tabletop.
+  **No correction is applied**: a reflection that fixed x would break the y/z
+  agreement. This is a documented residual mismatch, not a fixed one.
+
+Each task runs at RoboCasa's own registered horizon (`get_task_horizon`), not a
+truncated budget: 900 steps for CloseFridge, 750 for OpenDrawer, 450-600 for the
+rest. Semantic refresh 1 Hz, `n_action_steps=1`, 15 episodes/task.
+
+| Task | Horizon | Object registry | Success | 95% Wilson | Note |
+|---|---|---|---|---|---|
+| `CloseFridge` | 900 | lightwheel | 0/15 | 0.0-20.4 | — |
+| `OpenDrawer` | 750 | lightwheel | 0/15 | 0.0-20.4 | — |
+| `NavigateKitchen` | 450 | lightwheel | 0/15 | 0.0-20.4 | requires mobile-base motion; harness holds base_motion at zero |
+
+**Zero-shot transfer is a complete failure: 0/30 across the 2 arm-only tasks (95% Wilson 0.0-11.4%).** Not a partial-credit
+result to be tuned up — nothing succeeds anywhere.
+
+Two failure modes are visible in the saved videos (`results/robocasa_ft4/`), and
+they are *behavioural*, not random flailing: the arm drifts steadily upward until
+it clips the ceiling, and the wrist camera rotates until it faces the floor. Both
+are consistent with the one convention that could **not** be pinned down
+empirically — the correspondence between LIBERO's rotation axes and RoboCasa's
+`ee_rot`. A systematic rotation-frame error produces exactly this: a constant
+spurious delta integrated over hundreds of steps.
+
+Two honest caveats on the table above:
+
+* `NavigateKitchen` is **not a measurement of the policy**. The task requires
+  driving the mobile base across the kitchen, and this harness pins
+  `base_motion` to zero. Its 0/15 is guaranteed by construction, so it is
+  excluded from the aggregate.
+* The object registry differs by row. `TurnOnMicrowave` and
+  `PickPlaceCounterToStove` sample object categories that do not exist in the
+  `lightwheel` registry and **cannot be constructed** without objaverse/aigen
+  assets, so a single registry across all five tasks was not possible without
+  re-running the first two. The fixtures each task acts on (fridge, drawer,
+  microwave, stove) are scene fixtures, not sampled objects; the registry
+  affects distractor content.
+
+This is the motivation for section 12: rather than hand-tuning the one
+unverifiable frame convention, fine-tune on RoboCasa's own recorded data and let
+the model learn whatever axis mapping the dataset actually contains.
+
+
 ---
 
 ## Reproduction