ablation.md: add section 10 - fine-tuning pretrain_B on LIBERO with the ft4 recipe (69.0% vs ft4's 85.2%, significant)

ablation.md

@@ -8,7 +8,7 @@ All closed-loop numbers below are **paired**: every configuration runs the same
 
 Hardware: 4x Tesla V100-SXM2-32GB (sm_70), fp32, driver 535 / CUDA 12.2,
 torch 2.6.0+cu124, transformers 5.5.4, lerobot 0.6.1. Suite: `libero_spatial`.
-Generated 2026-08-22.
+Generated 2026-08-23.
 
 **The headline dual-rate comparison in the model card is not reproduced.** With the
 clock fixed and 500 paired episodes per configuration, no refresh rate differs
@@ -285,11 +285,6 @@ consistent-lag check (section 5, three ages) are both done: 12 grid cells, two
 controls, three consistency cells - 500 + 12x150 + 2x150 + 3x150 = 2,750
 closed-loop episodes.
 
-`checkpoints/stage2_final` (SO-100/101 + Bridge mixture, pre-LIBERO) and
-`checkpoints/stage3_final` (+ spatial distillation) just finished downloading and
-are being evaluated closed-loop on the same suite, to size how much of the 85.2%
-full-rate number comes from the LIBERO fine-tune itself.
-
 ## 9. Other checkpoints in the same repo, same harness
 
 Three checkpoints that predate the LIBERO fine-tune, evaluated closed-loop on
@@ -321,6 +316,50 @@ shared pretrain.
 
 ---
 
+## 10. Fine-tuning pretrain_B on LIBERO
+
+Section 9's `pretrain_B` row asks what a cross-embodiment pretrain is worth on
+LIBERO with no fine-tuning at all (nothing - 0%, same as every other pre-fine-tune
+checkpoint). This section asks the useful version of that question: swap the
+*starting checkpoint* for `ft4`'s recipe and hold everything else fixed - what does
+LIBERO fine-tuning look like starting from the canonical cross-embodiment pretrain
+instead of `stage3_distill`?
+
+**Recipe, unmodified from `configs/libero_ft4.yaml` except the resume point:**
+unfreeze vision tower + language stack, backbone at 0.1x lr, 40,000 steps,
+`resume_from: tinyvla2/pretrain_B`. `num_embodiments=16` and pretrain_B's
+`conditioning`/`morph_tokens`/`action_space` fields carried through so the resume
+loads with 0 missing / 0 unexpected keys (verified directly before launching).
+fp16 instead of bf16 (V100, sm_70); effective batch 48 via 4-way data-parallel
+instead of one A6000, matching ft4's effective batch. Final training loss 0.178,
+comparable to ft4's reported 0.066 - 500-episode full-rate closed-loop eval,
+paired against the same episodes as ft4's own full-rate row (section 3).
+
+| | Success | 95% CI | delta vs. ft4 | paired 95% CI | flips | McNemar p |
+|---|---|---|---|---|---|---|
+| ft4 (from stage3_distill) | 426/500 = 85.2% | — | — | — | — | — |
+| **B fine-tuned (from pretrain_B)** | 345/500 = 69.0% | [64.8,72.9] | -16.2 | [-21.0,-11.4] | +40/-121 | 0.0000*** |
+
+**pretrain_B does not make a better starting point than stage3_distill for LIBERO
+specifically - it makes a worse one, by a wide and significant margin.** This isn't
+one catastrophic task dragging an average down: 7/10 tasks score lower for the B-derived fine-tune than for ft4 (the other 3 are within a couple of episodes either way). Worst-hit: task8 at 9/50 vs. ft4's 39/50. Best-preserved: task5 at 49/50 vs. ft4's 42/50. Both checkpoints have identical architecture and parameter/FLOP counts (confirmed via the same profiling harness as section 6) - this is purely a pretraining-lineage effect, not a compute or capacity difference.
+
+Plausible reason: `stage3_distill` was trained specifically as *this* model's
+LIBERO-adjacent lineage (stage 1-3, spatial distillation against LIBERO-style
+labels per the model card), whereas `pretrain_B` is a from-scratch TinyVLA-2 run
+on a disjoint mixture (SO-100/101 + Bridge + RT-1, no LIBERO exposure anywhere in
+its lineage, canonical-EE-space objective) built for a different question entirely
+(cross-embodiment transfer to `jaco`/`stretch`, not LIBERO manipulation quality).
+40k fine-tuning steps evidently were not enough to close that gap. This is a data
+point against "any reasonable pretrain fine-tunes to a similar place on LIBERO",
+not evidence about canonical vs. native action spaces (both `ft4`'s lineage and
+`pretrain_B` ultimately regress against LIBERO's own action convention once
+fine-tuning starts, per section 1's frame-rate check applying identically to
+both - the difference is what the backbone learned before LIBERO ever entered
+the picture).
+
+---
+
 ## Reproduction
 
 ```bash