Instructions to use AlexWortega/tinyvla with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use AlexWortega/tinyvla with LeRobot:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Upload tinyvla2/RESULTS.md with huggingface_hub
Browse files- tinyvla2/RESULTS.md +47 -5
tinyvla2/RESULTS.md
CHANGED
|
@@ -740,11 +740,53 @@ two trained codes by 0.194 β comparable magnitudes, so there is no special "tr
|
|
| 740 |
subspace" a new robot could be projected into. For an unseen robot every available
|
| 741 |
code is somebody else's, and a random one is no worse.
|
| 742 |
|
| 743 |
-
|
| 744 |
-
|
| 745 |
-
*
|
| 746 |
-
|
| 747 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 748 |
|
| 749 |
That id 15 (g1: a humanoid arm commanded in joint space) is the best of the six is
|
| 750 |
the one sensible signal in the table β it is the closest morphology to an SO-100 arm
|
|
|
|
| 740 |
subspace" a new robot could be projected into. For an unseen robot every available
|
| 741 |
code is somebody else's, and a random one is no worse.
|
| 742 |
|
| 743 |
+
That looks like the argument for the morphology descriptor β a 16-number physical
|
| 744 |
+
descriptor can be *computed* for an unseen robot where a learned id can only be looked
|
| 745 |
+
up. **So it was tested, and the argument does not survive the test.**
|
| 746 |
+
|
| 747 |
+
### Converting the GR00T checkpoint to morphology conditioning does NOT fix LeKiwi
|
| 748 |
+
|
| 749 |
+
The checkpoint has no `morph_encoder` at all (the run discarded C-scaled's, since its
|
| 750 |
+
config is `conditioning: id`). Two steps were tried:
|
| 751 |
+
|
| 752 |
+
1. **Graft only** β set `conditioning: morph`, transplant tv2_C_scaled's trained
|
| 753 |
+
morph encoder. Result 1.30 / 1.11, *worse* than id. Expected: the expert spent 60k
|
| 754 |
+
steps reading a 1-token id and has never seen 2 morph tokens from another model's
|
| 755 |
+
encoder. (All six ids give identical numbers here, which confirms the morph path is
|
| 756 |
+
actually active and the id is ignored.)
|
| 757 |
+
2. **Train it to read the descriptor** β 6k streaming steps with the six GR00T family
|
| 758 |
+
descriptors (`scripts/train_groot_morph.py`), everything else already trained.
|
| 759 |
+
|
| 760 |
+
| conditioning | abs (7 dims) | delta (2 dims) |
|
| 761 |
+
|---|---|---|
|
| 762 |
+
| id, best of six (oracle) | 1.18 | 0.88 |
|
| 763 |
+
| morph, grafted only | 1.30 | 1.11 |
|
| 764 |
+
| morph, 1k steps | **1.13** | 0.83 |
|
| 765 |
+
| morph, 2k steps | 1.16 | 1.00 |
|
| 766 |
+
| morph, 4k steps | **1.12** | 0.87 |
|
| 767 |
+
| morph, 6k steps | 1.17 | 0.93 |
|
| 768 |
+
| oracle repeat-Aβ | 0.91 | 0.54 |
|
| 769 |
+
|
| 770 |
+
**The improvement is 1.18 -> 1.12 at best, and it is not monotonic** (1.13, 1.16,
|
| 771 |
+
1.12, 1.17 across checkpoints) β that spread is noise, not convergence. Both
|
| 772 |
+
conditioning schemes stay above 1.0 on the absolute dims, i.e. worse than holding the
|
| 773 |
+
current pose, and both lose to a dynamics-free predictor that repeats the first
|
| 774 |
+
ground-truth action.
|
| 775 |
+
|
| 776 |
+
So the honest conclusion is narrower than the one above: **being computable is
|
| 777 |
+
necessary but not sufficient.** A descriptor gives you a well-defined input for an
|
| 778 |
+
unseen robot, which an id cannot; it does not by itself give you transfer. This model
|
| 779 |
+
was trained on six morphologies β three panda variants, two humanoids and an R1-Pro β
|
| 780 |
+
none of them a small 5-DOF arm on a wheeled base, and no conditioning scheme extracts
|
| 781 |
+
a LeKiwi policy from that.
|
| 782 |
+
|
| 783 |
+
**Confound, stated rather than buried:** the morph run had to reweight the mixture,
|
| 784 |
+
because the per-episode sampler starved the loader (every draw pulled a ~190 MB
|
| 785 |
+
r1_pro episode and the first batch never arrived). The correction over-shot β realized
|
| 786 |
+
shares became gr1 0.67 / panda_single 0.23 / g1 0.04 / bimanual 0.035 / r1_pro 0.02 β
|
| 787 |
+
so the morph model saw a different mixture than the id model. This is not a matched
|
| 788 |
+
pair, and a matched-pair version would need retraining the id model on the same
|
| 789 |
+
reweighted stream.
|
| 790 |
|
| 791 |
That id 15 (g1: a humanoid arm commanded in joint space) is the best of the six is
|
| 792 |
the one sensible signal in the table β it is the closest morphology to an SO-100 arm
|