| --- |
| license: mit |
| library_name: starvla |
| tags: |
| - robotics |
| - vision-language-action |
| - florence-2 |
| - xtrainer2 |
| - retime |
| - optr |
| --- |
| |
| # ReTime × xTrainer2 — Model Suite |
|
|
| This page indexes the **four ReTime inference bundles** for bimanual xTrainer2 tasks. |
| All repos are **StarVLA policy checkpoints** (consolidated DeepSpeed ZeRO-3 `.pt` files), **not** Hugging Face `transformers` models. |
|
|
| | # | Model | Task | Action mode | Method | Recommended checkpoint | |
| |---|-------|------|-------------|--------|------------------------| |
| | 1 | [`florence-xtrainer2-pick-bottle-uniform`](https://huggingface.co/RainyBot/florence-xtrainer2-pick-bottle-uniform) | pick bottle | **Absolute** EEF | Uniform baseline | `checkpoints/steps_35460.pt` (~60 ep) | |
| | 2 | [`florence-xtrainer2-pick-bottle-rel-aug-uniform`](https://huggingface.co/RainyBot/florence-xtrainer2-pick-bottle-rel-aug-uniform) | pick bottle | **Relative** EEF + image aug | Uniform + rel + aug | `checkpoints/steps_35520.pt` (~60 ep) | |
| | 3 | [`florence-xtrainer2-pack-phone-uniform`](https://huggingface.co/RainyBot/florence-xtrainer2-pack-phone-uniform) | pack phone | **Absolute** EEF | Uniform baseline | `checkpoints/steps_43650.pt` | |
| | 4 | [`florence-xtrainer2-pack-phone-residual`](https://huggingface.co/RainyBot/florence-xtrainer2-pack-phone-residual) | pack phone | **Absolute** EEF | **OPTR residual temporal sampling** (20→60 ep) | `checkpoints/steps_52380.pt` (~60 ep) | |
|
|
| ## Shared setup |
|
|
| | Item | Value | |
| |------|-------| |
| | Backbone VLM | `microsoft/Florence-2-large` | |
| | Policy stack | StarVLA / OPTR (`QwenGR00T` + DiT-B action head) | |
| | Embodiment | xTrainer2, bimanual, 10 Hz | |
| | Cameras | `cam_high`, `cam_left_wrist`, `cam_right_wrist` | |
| | Robot wire | 16-D xyzw: `[xyz(3)+quat(4)+gripper(1)]` × left/right | |
| | Model tensor | 20-D after quaternion → rotation-6D | |
| | Action horizon | 16 | |
| | Image size | 224×224 | |
| | Extra deps | StarVLA tree with xTrainer2 DataConfig; `pytorch3d` | |
|
|
| Each repo layout: |
|
|
| ```text |
| config.yaml |
| dataset_statistics.json |
| checkpoints/steps_*.pt |
| ``` |
|
|
| `read_mode_config` expects `config.yaml` and `dataset_statistics.json` in the **parent of `checkpoints/`**. |
|
|
| ## How to choose a checkpoint |
|
|
| - **pick_bottle absolute baseline** → model 1 |
| - **pick_bottle relative + augmentation** → model 2 |
| - **pack_phone absolute baseline** → model 3 |
| - **pack_phone with ReTime / residual temporal sampling** → model 4 |
|
|
| ## Serve (common) |
|
|
| ```bash |
| export FLORENCE_MODEL_PATH=/path/to/Florence-2-large # optional if using HF id |
| bash examples/Xtrainer2/train_files/run_xtrainer2_policy_server.sh \ |
| /path/to/<repo>/checkpoints/<steps_XXXXX.pt> \ |
| 0 \ |
| 10093 |
| ``` |
|
|
| ## Task instructions |
|
|
| **pick_bottle** (must match training text exactly): |
| |
| - `Pick up the pink primer bottle and place it in the square dish.` |
| - `Pick up the pink dropper and place it in the square dish.` |
| - `Pick up the purple eyebrow gel tube and place it in the square dish.` |
| - `Pick up the silver eyebrow gel tube and place it in the square dish.` |
| |
| **pack_phone**: |
|
|
| - `Pick up the phone, place it in the box, and close the lid.` |
|
|
| ## License |
|
|
| MIT (weights). Florence-2-large follows Microsoft’s license separately. |
|
|
