Blame for real_robot/source/README.md

# ReTime for StarVLA

ReTime is an opt-in temporal-sampling extension for the accompanying StarVLA
backend. It preserves the dataset/trajectory distribution and changes the
within-trajectory temporal proposal:

```text
q(t | e) = (1 - alpha) p(t | e) + alpha r(t | e)
training loss = mean((p / q) * per_sample_loss)
```

The weights are ordinary importance weights, not batch-normalized weights.
`K` controls the frozen-policy scoring grid. `shuffle` permutes the residual
masses within each trajectory; `uniform` uses the baseline draw stream.

## Install

Use Linux, Python 3.11, a CUDA driver compatible with PyTorch 2.7.1, and the
dependencies in `requirements.txt`. The core training packages are pinned to
the versions used in the GPU integration test. From this source tree:

```bash
python -m pip install -r requirements.txt
python -m pip install -e .
retime --help
```

The supplied Qwen-GR00T configurations additionally use `AdamW8bit` and
`flash_attention_2`. Install their optional dependencies after the base
environment and select a Flash Attention build compatible with your CUDA and
PyTorch installation:

```bash
python -m pip install -r requirements-qwen.txt
python -m pip install flash-attn==2.7.4.post1 --no-build-isolation
```

Flash Attention may require a local CUDA toolkit and build tools. The version
above was present in the GPU validation environments. Both production Qwen
continuation configurations use Flash Attention 2. The bounded Qwen Bridge
integration run used PyTorch SDPA on the same model/data path.

This release supports the supplied Florence/Qwen GR00T configurations and
LeRobot data layouts. The backend includes the per-example loss and sampler
integration required by ReTime; it is not a claim of compatibility with every
unmodified StarVLA release or arbitrary policy architecture.

## Training

The trained common-start checkpoints used in the paper are not redistributed
with this anonymous artifact. Reviewers should either train their own
common-start policy using `retime-pretrain` or provide a compatible complete
checkpoint. The backbone, dataset, and benchmark assets must also be provided
locally.

Select a configuration matching the policy architecture, action dimensions,
dataset mixture, and training budget. Matching supplied pairs are:

| Setting | Policy configuration | Start → stop | Global batch | Accelerate configuration |
| --- | --- | ---: | ---: | --- |
| Florence × Bridge/RT-1 | `configs/florence_bridge_20k_plus500.yaml` | 20K → 20.5K | 64 | `configs/deepspeed_florence_2gpu.yaml` |
| Qwen × Bridge/RT-1 | `configs/qwen_bridge_29k_plus500.yaml` | 29K → 29.5K | 48 | `configs/qwen_single_1gpu.yaml` |
| Florence × RoboTwin | `configs/florence_robotwin_10k_plus500.yaml` | 10K → 10.5K | 64 | `configs/deepspeed_florence_robotwin_2gpu.yaml` |
| Qwen × RoboTwin | `configs/qwen_robotwin_11k_plus500.yaml` | 11K → 11.5K | 40 | `configs/deepspeed_qwen_robotwin_2gpu.yaml` |

These are the four main simulation continuation contracts. Every arm restores
the complete shared starting state and performs exactly 500 optimizer updates.
Bridge runs additionally require `--statistics-path`.

### Train the base policy locally

Reviewers do not need the authors' original common-start checkpoint.
`retime-pretrain` can train a reviewer-owned common-start policy with the same
architecture, optimizer, scheduler, batch geometry, and dataset-sampling
contract as the selected continuation configuration. For Florence Bridge:

```bash
retime-pretrain \
  --config configs/florence_bridge_20k_plus500.yaml \
  --accelerate-config configs/deepspeed_florence_2gpu.yaml \
  --base-steps 20000 \
  --vlm-path /path/to/Florence-2-large \
  --data-root /path/to/OXE_LEROBOT_DATASET \
  --statistics-path /path/to/dataset_statistics.json \
  --omit-data-identity-files \
  --output /path/to/florence-bridge-base
```

Use `--base-steps 29000`, `10000`, or `11000` with the other three configuration
pairs. The command builds and saves `manifest.json` before training. Two-process
configurations produce `checkpoints/steps_<N>_accelerate`; Qwen Bridge produces
`checkpoints/steps_29000_pytorch_model.pt` with adjacent training-state and
commit files. Both formats contain the model, optimizer, scheduler, RNG state,
sampling position, and completed-update counter required by exact continuation.

`--omit-data-identity-files` stores the computed dataset inventory inside the
manifest. Use the same option for continuation. This avoids requiring separate
provenance and fingerprint files while still checking the dataset inventory
against the frozen manifest.

For the Florence Bridge configuration:

```bash
retime \
  --config configs/florence_bridge_20k_plus500.yaml \
  --accelerate-config configs/deepspeed_florence_2gpu.yaml \
  --base-policy /path/to/steps_20000_accelerate \
  --start resume \
  --manifest /path/to/florence-bridge-base/manifest.json \
  --vlm-path /path/to/Florence-2-large \
  --data-root /path/to/OXE_LEROBOT_DATASET \
  --statistics-path /path/to/dataset_statistics.json \
  --omit-data-identity-files \
  --alpha 0.2 --K 8 \
  --mode retime uniform shuffle \
  --output /path/to/new-run
```

Choose one mode or several. The command prepares valid-anchor support from the
dataset, generates the residual proposal once, constructs Shuffle when selected,
and trains each selected arm sequentially from the same base policy. A
uniform-only run does not score a proposal. Invalid probabilities,
incompatible tensor shapes, incomplete
checkpoint files, and mismatched data support still produce errors.

`--base-policy` denotes the reviewer-provided common-start VLA checkpoint.
It is not bundled with this anonymous artifact and is not downloaded
automatically. `--vlm-path` denotes its Florence/Qwen backbone. They are
distinct inputs. The main configurations use
`--start resume`, which is required for the reported +500 continuation.
Florence Bridge and both RoboTwin settings accept a complete Accelerate
checkpoint directory. Qwen Bridge accepts its committed single-process model
file together with the adjacent training-state and commit files.

Exact resume restores model, optimizer, scheduler, RNG, sampler position/data
order, and completed-update counter. Supply the `manifest.json` produced by
the base-policy run and keep its `config.full.yaml` in the original run layout.
When separate provenance/fingerprint files are available, omit
`--omit-data-identity-files` and set the corresponding environment variables in
both commands. A single-process model file is accepted only when its adjacent
`*_training_state.pt` and `*_checkpoint.json` files form a complete committed
checkpoint.

`--start weights` remains available for a deliberate fresh optimizer/scheduler
run from model weights. It starts at update zero, so it does not reproduce any
of the four +500 experiments even when a continuation configuration is selected.

The launcher never changes microbatch, world size, accumulation, optimizer,
schedule, or random seeds to select an arm. The supplied Bridge pairing remains
`4 x 2 x 8 = 64`. The table above supplies matching Accelerate configurations
for the other three settings. Select a smaller dataset mixture with `--data-mix`
when checking a subset; the default RoboTwin configurations cover all 50 tasks.

Add `--dry-run` to validate inputs and print the command plan without writing or
launching. Successful runs produce `config.yaml`, `plan.json`, a proposal/cache,
and separate `runs/retime`, `runs/uniform`, and `runs/shuffle` directories. Use a
fresh output directory for each experiment. Existing proposal caches are bound
to the checkpoint content and support; incompatible caches cannot be reused.

## Layout

```text
retime/                  method implementation and command-line entrypoint
  pretrain.py            reviewer-owned common-start policy training
  objective.py           ordinary p/q correction
  sampler.py             deterministic hierarchical draw stream
  proposal.py            full-support temporal proposal
  scoring/               frozen-policy scoring and Shuffle control
  commands/              proposal generation
starVLA/                 attributed compatible training/model/data backend
scripts/                 legacy entrypoints and optional analysis utilities
configs/                 model, data, and training settings
licenses/                third-party license texts
```

Legacy `starVLA.*` method imports forward to `retime.*` for compatibility.
Analysis utilities are optional research tools and are not training prerequisites.

## Compatibility

| Interface | Support |
| --- | --- |
| StarVLA training | The original `starVLA/training/train_starvla.py` entrypoint is retained. |
| Legacy Python imports | Objective, sampler, proposal, and scoring modules forward to ReTime implementations. |
| Proposal and manifest files | Existing schemas and probability semantics are preserved. |
| Model initialization | Full model weights initialize a fresh optimizer and scheduler. |
| Full-state continuation | `--start resume` restores a complete Accelerate directory or committed single-process checkpoint. |
| Distributed training | Matching one- and two-GPU configurations are supplied for all four settings. |

Florence-GR00T and Qwen-GR00T were exercised on GPUs with both Bridge/RT-1 and
RoboTwin data. Other policy architectures require a compatible per-sample loss
and dataset adapter.
Historical `OPTR_*` environment variables and `InfoFlow` module names in
example configurations are compatibility identifiers for existing datasets and
checkpoints. The supported command-line entrypoint for this artifact is `retime`.

## Tests and scope

```bash
python -m pip install pytest
python -m pytest tests/test_retime_plugin.py -q
```

These CPU tests check proposal generation, objective correction, deterministic
sampling, Shuffle invariants, legacy imports, CLI dispatch, and error propagation.
A separate GPU integration test completed proposal generation, two optimizer
updates, checkpoint saving, and strict model reload for each of the three modes
in all four settings. See `VALIDATION.md` for details.
Datasets, trained weights, checkpoints, and benchmark repositories are not
redistributed in the working source tree.

## Attribution and anonymous review

Built on [StarVLA](https://github.com/starVLA/starVLA), pinned to
`3422b9f2387b6f682cf02802904a77b23ab13afd`. See `UPSTREAM.md`, `LICENSE`, and
`NOTICE`. Retain the complete license, source copyright notices, and the
separate original upstream commits in the accompanying Git history.

Third-party attribution is intentionally visible. The anonymous additions have
no author-account remote or personal development history. Publish through an
anonymous review endpoint, not an author's personal repository URL. Avoid
uploading local training outputs: they can contain machine paths and metadata.