Add Xtrainer2 real robot source and Chinese handoff guide

8101696 verified 6 days ago

10.9 kB

# ReTime for StarVLA

ReTime is an opt-in temporal-sampling extension for the accompanying StarVLA
backend. It preserves the dataset/trajectory distribution and changes the
within-trajectory temporal proposal:

```text
q(t | e) = (1 - alpha) p(t | e) + alpha r(t | e)
training loss = mean((p / q) * per_sample_loss)
```

The weights are ordinary importance weights, not batch-normalized weights.
`K` controls the frozen-policy scoring grid. `shuffle` permutes the residual
masses within each trajectory; `uniform` uses the baseline draw stream.

## Install

Use Linux, Python 3.11, a CUDA driver compatible with PyTorch 2.7.1, and the
dependencies in `requirements.txt`. The core training packages are pinned to
the versions used in the GPU integration test. From this source tree:

```bash
python -m pip install -r requirements.txt
python -m pip install -e .
retime --help
```

The supplied Qwen-GR00T configurations additionally use `AdamW8bit` and
`flash_attention_2`. Install their optional dependencies after the base
environment and select a Flash Attention build compatible with your CUDA and
PyTorch installation:

```bash
python -m pip install -r requirements-qwen.txt
python -m pip install flash-attn==2.7.4.post1 --no-build-isolation
```

Flash Attention may require a local CUDA toolkit and build tools. The version
above was present in the GPU validation environments. Both production Qwen
continuation configurations use Flash Attention 2. The bounded Qwen Bridge
integration run used PyTorch SDPA on the same model/data path.

This release supports the supplied Florence/Qwen GR00T configurations and
LeRobot data layouts. The backend includes the per-example loss and sampler
integration required by ReTime; it is not a claim of compatibility with every
unmodified StarVLA release or arbitrary policy architecture.

## Training

The trained common-start checkpoints used in the paper are not redistributed
with this anonymous artifact. Reviewers should either train their own
common-start policy using `retime-pretrain` or provide a compatible complete
checkpoint. The backbone, dataset, and benchmark assets must also be provided
locally.

Select a configuration matching the policy architecture, action dimensions,
dataset mixture, and training budget. Matching supplied pairs are:

| Setting | Policy configuration | Start → stop | Global batch | Accelerate configuration |
| --- | --- | ---: | ---: | --- |
| Florence × Bridge/RT-1 | `configs/florence_bridge_20k_plus500.yaml` | 20K → 20.5K | 64 | `configs/deepspeed_florence_2gpu.yaml` |
| Qwen × Bridge/RT-1 | `configs/qwen_bridge_29k_plus500.yaml` | 29K → 29.5K | 48 | `configs/qwen_single_1gpu.yaml` |
| Florence × RoboTwin | `configs/florence_robotwin_10k_plus500.yaml` | 10K → 10.5K | 64 | `configs/deepspeed_florence_robotwin_2gpu.yaml` |
| Qwen × RoboTwin | `configs/qwen_robotwin_11k_plus500.yaml` | 11K → 11.5K | 40 | `configs/deepspeed_qwen_robotwin_2gpu.yaml` |

These are the four main simulation continuation contracts. Every arm restores
the complete shared starting state and performs exactly 500 optimizer updates.
Bridge runs additionally require `--statistics-path`.

### Train the base policy locally

Reviewers do not need the authors' original common-start checkpoint.
`retime-pretrain` can train a reviewer-owned common-start policy with the same
architecture, optimizer, scheduler, batch geometry, and dataset-sampling
contract as the selected continuation configuration. For Florence Bridge:

```bash
retime-pretrain \
  --config configs/florence_bridge_20k_plus500.yaml \
  --accelerate-config configs/deepspeed_florence_2gpu.yaml \
  --base-steps 20000 \
  --vlm-path /path/to/Florence-2-large \
  --data-root /path/to/OXE_LEROBOT_DATASET \
  --statistics-path /path/to/dataset_statistics.json \
  --omit-data-identity-files \
  --output /path/to/florence-bridge-base
```

Use `--base-steps 29000`, `10000`, or `11000` with the other three configuration
pairs. The command builds and saves `manifest.json` before training. Two-process
configurations produce `checkpoints/steps_<N>_accelerate`; Qwen Bridge produces
`checkpoints/steps_29000_pytorch_model.pt` with adjacent training-state and
commit files. Both formats contain the model, optimizer, scheduler, RNG state,
sampling position, and completed-update counter required by exact continuation.

`--omit-data-identity-files` stores the computed dataset inventory inside the
manifest. Use the same option for continuation. This avoids requiring separate
provenance and fingerprint files while still checking the dataset inventory
against the frozen manifest.

For the Florence Bridge configuration:

```bash
retime \
  --config configs/florence_bridge_20k_plus500.yaml \
  --accelerate-config configs/deepspeed_florence_2gpu.yaml \
  --base-policy /path/to/steps_20000_accelerate \
  --start resume \
  --manifest /path/to/florence-bridge-base/manifest.json \
  --vlm-path /path/to/Florence-2-large \
  --data-root /path/to/OXE_LEROBOT_DATASET \
  --statistics-path /path/to/dataset_statistics.json \
  --omit-data-identity-files \
  --alpha 0.2 --K 8 \
  --mode retime uniform shuffle \
  --output /path/to/new-run
```

Choose one mode or several. The command prepares valid-anchor support from the
dataset, generates the residual proposal once, constructs Shuffle when selected,
and trains each selected arm sequentially from the same base policy. A
uniform-only run does not score a proposal. Invalid probabilities,
incompatible tensor shapes, incomplete
checkpoint files, and mismatched data support still produce errors.

`--base-policy` denotes the reviewer-provided common-start VLA checkpoint.
It is not bundled with this anonymous artifact and is not downloaded
automatically. `--vlm-path` denotes its Florence/Qwen backbone. They are
distinct inputs. The main configurations use
`--start resume`, which is required for the reported +500 continuation.
Florence Bridge and both RoboTwin settings accept a complete Accelerate
checkpoint directory. Qwen Bridge accepts its committed single-process model
file together with the adjacent training-state and commit files.

Exact resume restores model, optimizer, scheduler, RNG, sampler position/data
order, and completed-update counter. Supply the `manifest.json` produced by
the base-policy run and keep its `config.full.yaml` in the original run layout.
When separate provenance/fingerprint files are available, omit
`--omit-data-identity-files` and set the corresponding environment variables in
both commands. A single-process model file is accepted only when its adjacent
`*_training_state.pt` and `*_checkpoint.json` files form a complete committed
checkpoint.

`--start weights` remains available for a deliberate fresh optimizer/scheduler
run from model weights. It starts at update zero, so it does not reproduce any
of the four +500 experiments even when a continuation configuration is selected.

The launcher never changes microbatch, world size, accumulation, optimizer,
schedule, or random seeds to select an arm. The supplied Bridge pairing remains
`4 x 2 x 8 = 64`. The table above supplies matching Accelerate configurations
for the other three settings. Select a smaller dataset mixture with `--data-mix`
when checking a subset; the default RoboTwin configurations cover all 50 tasks.

Add `--dry-run` to validate inputs and print the command plan without writing or
launching. Successful runs produce `config.yaml`, `plan.json`, a proposal/cache,
and separate `runs/retime`, `runs/uniform`, and `runs/shuffle` directories. Use a
fresh output directory for each experiment. Existing proposal caches are bound
to the checkpoint content and support; incompatible caches cannot be reused.

## Layout

```text
retime/                  method implementation and command-line entrypoint
  pretrain.py            reviewer-owned common-start policy training
  objective.py           ordinary p/q correction
  sampler.py             deterministic hierarchical draw stream
  proposal.py            full-support temporal proposal
  scoring/               frozen-policy scoring and Shuffle control
  commands/              proposal generation
starVLA/                 attributed compatible training/model/data backend
scripts/                 legacy entrypoints and optional analysis utilities
configs/                 model, data, and training settings
licenses/                third-party license texts
```

Legacy `starVLA.*` method imports forward to `retime.*` for compatibility.
Analysis utilities are optional research tools and are not training prerequisites.

## Compatibility

| Interface | Support |
| --- | --- |
| StarVLA training | The original `starVLA/training/train_starvla.py` entrypoint is retained. |
| Legacy Python imports | Objective, sampler, proposal, and scoring modules forward to ReTime implementations. |
| Proposal and manifest files | Existing schemas and probability semantics are preserved. |
| Model initialization | Full model weights initialize a fresh optimizer and scheduler. |
| Full-state continuation | `--start resume` restores a complete Accelerate directory or committed single-process checkpoint. |
| Distributed training | Matching one- and two-GPU configurations are supplied for all four settings. |

Florence-GR00T and Qwen-GR00T were exercised on GPUs with both Bridge/RT-1 and
RoboTwin data. Other policy architectures require a compatible per-sample loss
and dataset adapter.
Historical `OPTR_*` environment variables and `InfoFlow` module names in
example configurations are compatibility identifiers for existing datasets and
checkpoints. The supported command-line entrypoint for this artifact is `retime`.

## Tests and scope

```bash
python -m pip install pytest
python -m pytest tests/test_retime_plugin.py -q
```

These CPU tests check proposal generation, objective correction, deterministic
sampling, Shuffle invariants, legacy imports, CLI dispatch, and error propagation.
A separate GPU integration test completed proposal generation, two optimizer
updates, checkpoint saving, and strict model reload for each of the three modes
in all four settings. See `VALIDATION.md` for details.
Datasets, trained weights, checkpoints, and benchmark repositories are not
redistributed in the working source tree.

## Attribution and anonymous review

Built on [StarVLA](https://github.com/starVLA/starVLA), pinned to
`3422b9f2387b6f682cf02802904a77b23ab13afd`. See `UPSTREAM.md`, `LICENSE`, and
`NOTICE`. Retain the complete license, source copyright notices, and the
separate original upstream commits in the accompanying Git history.

Third-party attribution is intentionally visible. The anonymous additions have
no author-account remote or personal development history. Publish through an
anonymous review endpoint, not an author's personal repository URL. Avoid
uploading local training outputs: they can contain machine paths and metadata.

	# ReTime for StarVLA

	ReTime is an opt-in temporal-sampling extension for the accompanying StarVLA
	backend. It preserves the dataset/trajectory distribution and changes the
	within-trajectory temporal proposal:

	```text
	q(t \| e) = (1 - alpha) p(t \| e) + alpha r(t \| e)
	training loss = mean((p / q) * per_sample_loss)
	```

	The weights are ordinary importance weights, not batch-normalized weights.
	`K` controls the frozen-policy scoring grid. `shuffle` permutes the residual
	masses within each trajectory; `uniform` uses the baseline draw stream.

	## Install

	Use Linux, Python 3.11, a CUDA driver compatible with PyTorch 2.7.1, and the
	dependencies in `requirements.txt`. The core training packages are pinned to
	the versions used in the GPU integration test. From this source tree:

	```bash
	python -m pip install -r requirements.txt
	python -m pip install -e .
	retime --help
	```

	The supplied Qwen-GR00T configurations additionally use `AdamW8bit` and
	`flash_attention_2`. Install their optional dependencies after the base
	environment and select a Flash Attention build compatible with your CUDA and
	PyTorch installation:

	```bash
	python -m pip install -r requirements-qwen.txt
	python -m pip install flash-attn==2.7.4.post1 --no-build-isolation
	```

	Flash Attention may require a local CUDA toolkit and build tools. The version
	above was present in the GPU validation environments. Both production Qwen
	continuation configurations use Flash Attention 2. The bounded Qwen Bridge
	integration run used PyTorch SDPA on the same model/data path.

	This release supports the supplied Florence/Qwen GR00T configurations and
	LeRobot data layouts. The backend includes the per-example loss and sampler
	integration required by ReTime; it is not a claim of compatibility with every
	unmodified StarVLA release or arbitrary policy architecture.

	## Training

	The trained common-start checkpoints used in the paper are not redistributed
	with this anonymous artifact. Reviewers should either train their own
	common-start policy using `retime-pretrain` or provide a compatible complete
	checkpoint. The backbone, dataset, and benchmark assets must also be provided
	locally.

	Select a configuration matching the policy architecture, action dimensions,
	dataset mixture, and training budget. Matching supplied pairs are:

	\| Setting \| Policy configuration \| Start → stop \| Global batch \| Accelerate configuration \|
	\| --- \| --- \| ---: \| ---: \| --- \|
	\| Florence × Bridge/RT-1 \| `configs/florence_bridge_20k_plus500.yaml` \| 20K → 20.5K \| 64 \| `configs/deepspeed_florence_2gpu.yaml` \|
	\| Qwen × Bridge/RT-1 \| `configs/qwen_bridge_29k_plus500.yaml` \| 29K → 29.5K \| 48 \| `configs/qwen_single_1gpu.yaml` \|
	\| Florence × RoboTwin \| `configs/florence_robotwin_10k_plus500.yaml` \| 10K → 10.5K \| 64 \| `configs/deepspeed_florence_robotwin_2gpu.yaml` \|
	\| Qwen × RoboTwin \| `configs/qwen_robotwin_11k_plus500.yaml` \| 11K → 11.5K \| 40 \| `configs/deepspeed_qwen_robotwin_2gpu.yaml` \|

	These are the four main simulation continuation contracts. Every arm restores
	the complete shared starting state and performs exactly 500 optimizer updates.
	Bridge runs additionally require `--statistics-path`.

	### Train the base policy locally

	Reviewers do not need the authors' original common-start checkpoint.
	`retime-pretrain` can train a reviewer-owned common-start policy with the same
	architecture, optimizer, scheduler, batch geometry, and dataset-sampling
	contract as the selected continuation configuration. For Florence Bridge:

	```bash
	retime-pretrain \
	--config configs/florence_bridge_20k_plus500.yaml \
	--accelerate-config configs/deepspeed_florence_2gpu.yaml \
	--base-steps 20000 \
	--vlm-path /path/to/Florence-2-large \
	--data-root /path/to/OXE_LEROBOT_DATASET \
	--statistics-path /path/to/dataset_statistics.json \
	--omit-data-identity-files \
	--output /path/to/florence-bridge-base
	```

	Use `--base-steps 29000`, `10000`, or `11000` with the other three configuration
	pairs. The command builds and saves `manifest.json` before training. Two-process
	configurations produce `checkpoints/steps_<N>_accelerate`; Qwen Bridge produces
	`checkpoints/steps_29000_pytorch_model.pt` with adjacent training-state and
	commit files. Both formats contain the model, optimizer, scheduler, RNG state,
	sampling position, and completed-update counter required by exact continuation.

	`--omit-data-identity-files` stores the computed dataset inventory inside the
	manifest. Use the same option for continuation. This avoids requiring separate
	provenance and fingerprint files while still checking the dataset inventory
	against the frozen manifest.

	For the Florence Bridge configuration:

	```bash
	retime \
	--config configs/florence_bridge_20k_plus500.yaml \
	--accelerate-config configs/deepspeed_florence_2gpu.yaml \
	--base-policy /path/to/steps_20000_accelerate \
	--start resume \
	--manifest /path/to/florence-bridge-base/manifest.json \
	--vlm-path /path/to/Florence-2-large \
	--data-root /path/to/OXE_LEROBOT_DATASET \
	--statistics-path /path/to/dataset_statistics.json \
	--omit-data-identity-files \
	--alpha 0.2 --K 8 \
	--mode retime uniform shuffle \
	--output /path/to/new-run
	```

	Choose one mode or several. The command prepares valid-anchor support from the
	dataset, generates the residual proposal once, constructs Shuffle when selected,
	and trains each selected arm sequentially from the same base policy. A
	uniform-only run does not score a proposal. Invalid probabilities,
	incompatible tensor shapes, incomplete
	checkpoint files, and mismatched data support still produce errors.

	`--base-policy` denotes the reviewer-provided common-start VLA checkpoint.
	It is not bundled with this anonymous artifact and is not downloaded
	automatically. `--vlm-path` denotes its Florence/Qwen backbone. They are
	distinct inputs. The main configurations use
	`--start resume`, which is required for the reported +500 continuation.
	Florence Bridge and both RoboTwin settings accept a complete Accelerate
	checkpoint directory. Qwen Bridge accepts its committed single-process model
	file together with the adjacent training-state and commit files.

	Exact resume restores model, optimizer, scheduler, RNG, sampler position/data
	order, and completed-update counter. Supply the `manifest.json` produced by
	the base-policy run and keep its `config.full.yaml` in the original run layout.
	When separate provenance/fingerprint files are available, omit
	`--omit-data-identity-files` and set the corresponding environment variables in
	both commands. A single-process model file is accepted only when its adjacent
	`_training_state.pt` and `_checkpoint.json` files form a complete committed
	checkpoint.

	`--start weights` remains available for a deliberate fresh optimizer/scheduler
	run from model weights. It starts at update zero, so it does not reproduce any
	of the four +500 experiments even when a continuation configuration is selected.

	The launcher never changes microbatch, world size, accumulation, optimizer,
	schedule, or random seeds to select an arm. The supplied Bridge pairing remains
	`4 x 2 x 8 = 64`. The table above supplies matching Accelerate configurations
	for the other three settings. Select a smaller dataset mixture with `--data-mix`
	when checking a subset; the default RoboTwin configurations cover all 50 tasks.

	Add `--dry-run` to validate inputs and print the command plan without writing or
	launching. Successful runs produce `config.yaml`, `plan.json`, a proposal/cache,
	and separate `runs/retime`, `runs/uniform`, and `runs/shuffle` directories. Use a
	fresh output directory for each experiment. Existing proposal caches are bound
	to the checkpoint content and support; incompatible caches cannot be reused.

	## Layout

	```text
	retime/ method implementation and command-line entrypoint
	pretrain.py reviewer-owned common-start policy training
	objective.py ordinary p/q correction
	sampler.py deterministic hierarchical draw stream
	proposal.py full-support temporal proposal
	scoring/ frozen-policy scoring and Shuffle control
	commands/ proposal generation
	starVLA/ attributed compatible training/model/data backend
	scripts/ legacy entrypoints and optional analysis utilities
	configs/ model, data, and training settings
	licenses/ third-party license texts
	```

	Legacy `starVLA.` method imports forward to `retime.` for compatibility.
	Analysis utilities are optional research tools and are not training prerequisites.

	## Compatibility

	\| Interface \| Support \|
	\| --- \| --- \|
	\| StarVLA training \| The original `starVLA/training/train_starvla.py` entrypoint is retained. \|
	\| Legacy Python imports \| Objective, sampler, proposal, and scoring modules forward to ReTime implementations. \|
	\| Proposal and manifest files \| Existing schemas and probability semantics are preserved. \|
	\| Model initialization \| Full model weights initialize a fresh optimizer and scheduler. \|
	\| Full-state continuation \| `--start resume` restores a complete Accelerate directory or committed single-process checkpoint. \|
	\| Distributed training \| Matching one- and two-GPU configurations are supplied for all four settings. \|

	Florence-GR00T and Qwen-GR00T were exercised on GPUs with both Bridge/RT-1 and
	RoboTwin data. Other policy architectures require a compatible per-sample loss
	and dataset adapter.
	Historical `OPTR_*` environment variables and `InfoFlow` module names in
	example configurations are compatibility identifiers for existing datasets and
	checkpoints. The supported command-line entrypoint for this artifact is `retime`.

	## Tests and scope

	```bash
	python -m pip install pytest
	python -m pytest tests/test_retime_plugin.py -q
	```

	These CPU tests check proposal generation, objective correction, deterministic
	sampling, Shuffle invariants, legacy imports, CLI dispatch, and error propagation.
	A separate GPU integration test completed proposal generation, two optimizer
	updates, checkpoint saving, and strict model reload for each of the three modes
	in all four settings. See `VALIDATION.md` for details.
	Datasets, trained weights, checkpoints, and benchmark repositories are not
	redistributed in the working source tree.

	## Attribution and anonymous review

	Built on [StarVLA](https://github.com/starVLA/starVLA), pinned to
	`3422b9f2387b6f682cf02802904a77b23ab13afd`. See `UPSTREAM.md`, `LICENSE`, and
	`NOTICE`. Retain the complete license, source copyright notices, and the
	separate original upstream commits in the accompanying Git history.

	Third-party attribution is intentionally visible. The anonymous additions have
	no author-account remote or personal development history. Publish through an
	anonymous review endpoint, not an author's personal repository URL. Avoid
	uploading local training outputs: they can contain machine paths and metadata.