AlexWortega commited on
Commit
d055b5b
·
verified ·
1 Parent(s): 41d966e

Upload tinyvla2/RESULTS.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. tinyvla2/RESULTS.md +37 -10
tinyvla2/RESULTS.md CHANGED
@@ -430,17 +430,44 @@ and commanding the error against the *measured* pose removes the lag and recover
430
  the full ceiling. **Anyone deploying a canonical-action policy on a position-servo
431
  robot needs this; the scaled decode looks reasonable offline and fails on hardware.**
432
 
433
- ## Results
434
- | model | LIBERO-spatial |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
435
  |---|---|
436
- | zero-shot, no LIBERO training (offline ratio to zero-floor) | **0.79** |
437
- | canonical FT, 5k steps, closed loop (n=20) | **40%** |
438
- | v1 native-action, 40k steps (reference) | 86% |
439
-
440
- Zero-shot 0.79 means the cross-embodiment checkpoint predicts LIBERO trajectories
441
- better than the trivial baseline having never seen the robot or the domain. The 5k
442
- closed-loop number is an early checkpoint of a 30k run, reported because it is what
443
- exists — not a converged result.
 
 
 
 
 
 
 
 
 
 
444
 
445
  ## RoboCasa continue-train, re-measured (step_10000, corrected harness)
446
 
 
430
  the full ceiling. **Anyone deploying a canonical-action policy on a position-servo
431
  robot needs this; the scaled decode looks reasonable offline and fails on hardware.**
432
 
433
+ ## Results (30k fine-tune complete)
434
+
435
+ Pipeline ceilings first, so the model number can be read against what the action
436
+ space and decoder actually permit:
437
+
438
+ | what is being replayed | success |
439
+ |---|---|
440
+ | raw GT actions (env ceiling) | 9/10 |
441
+ | canonical encode→decode, GT gripper | **9/10** |
442
+ | canonical encode→decode, decoded gripper | 8/10 |
443
+
444
+ **The canonical motion representation is lossless** — routing GT through
445
+ base-frame EE deltas and back through the servo decoder costs nothing (9/10 either
446
+ way). The whole ~10-point loss comes from the gripper: canonical stores absolute
447
+ openness, and turning that back into LIBERO's binary open/close command is only
448
+ 92.5% accurate per step. That is the one place the canonical space is lossy, and it
449
+ is fixable (predict the command channel directly rather than openness).
450
+
451
+ | model | LIBERO-spatial (n=50) |
452
  |---|---|
453
+ | zero-shot, no LIBERO training (offline ratio to zero-floor) | 0.79 |
454
+ | canonical FT 5k steps | 40% |
455
+ | **canonical FT 30k, full rate** | **70%** |
456
+ | canonical FT 30k, dual-rate (latent @1 Hz) | 68% |
457
+ | decoded-pipeline ceiling | 80% |
458
+ | v1, native actions, 40k steps (reference) | 86% |
459
+
460
+ **70% against an 80% decoder ceiling**: the policy captures ~88% of what the
461
+ pipeline allows. Against v1's 86% in the native action space, canonical actions cost
462
+ 16 points, of which ~10 are the gripper decode and ~6 are the policy.
463
+
464
+ **Dual-rate survives the change of action space** (68% vs 70%, indistinguishable at
465
+ n=50). The v1 claim — a semantic latent refreshed at 1 Hz is enough, the fast path
466
+ carries the rest — holds for a cross-embodiment canonical policy too, which is the
467
+ result that makes the v2 line deployable at v1's compute profile.
468
+
469
+ Zero-shot 0.79 means the cross-embodiment checkpoint beat the trivial baseline on a
470
+ robot and domain it had never seen, before any LIBERO training at all.
471
 
472
  ## RoboCasa continue-train, re-measured (step_10000, corrected harness)
473