README.md

metadata

license: mit
library_name: starvla
tags:
  - robotics
  - vision-language-action
  - florence-2
  - xtrainer2
  - retime
  - optr

ReTime × xTrainer2 — Model Suite

This page indexes the four ReTime inference bundles for bimanual xTrainer2 tasks. All repos are StarVLA policy checkpoints (consolidated DeepSpeed ZeRO-3 .pt files), not Hugging Face transformers models.

#	Model	Task	Action mode	Method	Recommended checkpoint
1	florence-xtrainer2-pick-bottle-uniform	pick bottle	Absolute EEF	Uniform baseline	checkpoints/steps_35460.pt (~60 ep)
2	florence-xtrainer2-pick-bottle-rel-aug-uniform	pick bottle	Relative EEF + image aug	Uniform + rel + aug	checkpoints/steps_35520.pt (~60 ep)
3	florence-xtrainer2-pack-phone-uniform	pack phone	Absolute EEF	Uniform baseline	checkpoints/steps_43650.pt
4	florence-xtrainer2-pack-phone-residual	pack phone	Absolute EEF	OPTR residual temporal sampling (20→60 ep)	checkpoints/steps_52380.pt (~60 ep)

Shared setup

Item	Value
Backbone VLM	microsoft/Florence-2-large
Policy stack	StarVLA / OPTR (QwenGR00T + DiT-B action head)
Embodiment	xTrainer2, bimanual, 10 Hz
Cameras	cam_high, cam_left_wrist, cam_right_wrist
Robot wire	16-D xyzw: [xyz(3)+quat(4)+gripper(1)] × left/right
Model tensor	20-D after quaternion → rotation-6D
Action horizon	16
Image size	224×224
Extra deps	StarVLA tree with xTrainer2 DataConfig; pytorch3d

Each repo layout:

config.yaml
dataset_statistics.json
checkpoints/steps_*.pt

read_mode_config expects config.yaml and dataset_statistics.json in the parent of checkpoints/.

How to choose a checkpoint

• pick_bottle absolute baseline → model 1
• pick_bottle relative + augmentation → model 2
• pack_phone absolute baseline → model 3
• pack_phone with ReTime / residual temporal sampling → model 4

Serve (common)

export FLORENCE_MODEL_PATH=/path/to/Florence-2-large   # optional if using HF id
bash examples/Xtrainer2/train_files/run_xtrainer2_policy_server.sh \
  /path/to/<repo>/checkpoints/<steps_XXXXX.pt> \
  0 \
  10093

Task instructions

pick_bottle (must match training text exactly):

• Pick up the pink primer bottle and place it in the square dish.
• Pick up the pink dropper and place it in the square dish.
• Pick up the purple eyebrow gel tube and place it in the square dish.
• Pick up the silver eyebrow gel tube and place it in the square dish.

pack_phone:

• Pick up the phone, place it in the box, and close the lid.

License

MIT (weights). Florence-2-large follows Microsoft’s license separately.