Upload tinyvla2/RESULTS.md with huggingface_hub

tinyvla2/RESULTS.md

@@ -796,3 +796,50 @@ Not comparable to our LeKiwi numbers (C-scaled 1.67, +joint 2.10): those are
 integrated endpoint error in canonical base-frame EE space, these are native joint
 space. The only shared, comparable statement is that **both lines fail on LeKiwi**,
 each against its own trivial floor.
+
+## Closed-loop eval was single-draw, and that was costing more than any ablation
+
+`predict_action_chunk` starts from `torch.randn` and nothing seeded it, so every
+closed-loop rollout was a lottery. Caught by accident: the SAME checkpoint with the
+SAME settings scored **4/20 and then 8/20** on consecutive runs. Every previously
+reported closed-loop number was one draw from that distribution.
+
+Fixed by averaging the flow ODE over 4 noise draws per decision and seeding each
+decision deterministically (`--pred-seeds`, default 4). The eval is now reproducible
+— three consecutive runs returned exactly 70.0%.
+
+**How much it was costing depends on how converged the checkpoint is:**
+
+| checkpoint | single draw | 4-seed averaged |
+|---|---|---|
+| new run, 5k steps (n=20) | 20% / 40% (two runs) | **70%** |
+| old run, 5k steps (n=20) | 40% | **60%** |
+| old run, 30k steps (n=50) | 70% | 68% |
+
+At 30k the averaging changes nothing (70 vs 68, within noise) — the flow field is
+sharp enough that one draw is already near the conditional mean. At 5k it is worth
+20-30 points, because an under-trained field is diffuse and the noise compounds over
+a 500-step rollout. **So the published 70% for the converged model stands**, but
+every intermediate-checkpoint number in this document was noise-dominated, and the
+dual-rate comparison (70 vs 68) was never precise enough to support a claim either
+way.
+
+## Gripper command channel — the honest verdict so far
+
+The ceiling gain is real and proven: GT round-trip 8/10 with a decoded command vs
+**9/10 with the carried command**, matching raw GT exactly.
+
+Whether the policy converts that ceiling into success is not yet established:
+
+| comparison (all 4-seed averaged) | result |
+|---|---|
+| new run @5k vs old run @5k, n=20 | 70% vs 60% |
+| within the new checkpoint: decode `canon` vs `threshold`, n=20 | 70% vs 70% |
+
++10 points at a matched step is two episodes out of twenty — not significant. And
+the decode-mode ablation on the *same* checkpoint shows no difference at all, which
+argues the channel is not the binding constraint at this stage: the model's predicted
+openness is already good enough that thresholding it matches reading the command.
+
+The decisive number is the new run at 30k against the old run's 68% at n=50, and that
+needs the training to finish (currently ~10k/30k).