Blame for README.md

---
license: mit
library_name: pytorch
tags:
  - robotics
  - vision-language-action
  - florence-2
  - qwen3-vl
  - bridge
  - importance-sampling
---

# ReTime / OPTR — four-setting handoff

This package contains the server-authoritative Florence–Bridge OPTR code, its complete 20K common-start state and Uniform/Residual-OPTR 20.5K evaluation snapshots. It also contains the complete Qwen3-VL-4B–Bridge 29K common-start state and the environment/seed contract needed for paired multi-seed continuation experiments.

## Unified checkpoint layout

The release is being normalized under [`settings/`](settings/README.md). Each of the four settings has the same entries: `common_start`, `uniform`, `residual`, `keyframe`, `frameskip`, `loss_is`, `configs`, and `dependencies`.

- [`settings/florence_bridge/`](settings/florence_bridge/README.md)
- [`settings/florence_robotwin/`](settings/florence_robotwin/README.md)
- [`settings/qwen_bridge/`](settings/qwen_bridge/README.md)
- [`settings/qwen_robotwin/`](settings/qwen_robotwin/README.md)

The common-start entry is the resumable branch point. Method endpoints are evaluation-ready model snapshots. Shared source code is kept once at repository root and referenced by each setting to avoid four inconsistent copies.

## Rebuttal 训练与测评种子

| 设置 | Common-start 训练 seed | Continuation 训练 seed | Evaluation policy seeds |
|---|---|---|---|
| Florence–Bridge 20K → 20.5K | `42` | 主实验 `117465`，已报告三组 `117465, 533389, 944964` | `518957, 765353, 686765, 516408` |
| Qwen3-VL-4B–Bridge 29K 与 29.5K | `42` | 历史 +500 配置为 `42`，继承 29K RNG。新多种子清单待定 | `518957, 765353, 686765, 516408` |

完整的 Florence RNG 字段、Rebuttal 复用规则和原始记录链接见 [Bridge 训练与测评种子总表](BRIDGE_SEED_PROTOCOL.md)。

## What is included

- [`real_robot/`](real_robot/README.md)：Xtrainer2 真机数据转换、训练、双臂动作适配和 policy server 源码，以及中文交接说明。
- `common_start_20k/checkpoints/steps_20000_accelerate/`: **full resumable state** (model, optimizer, scheduler metadata and per-rank RNG state). Start every new continuation arm here.
- `checkpoints/uniform/steps_20500_pytorch_model.pt`: model-only Uniform +500 snapshot.
- `checkpoints/residual/steps_20500_pytorch_model.pt`: model-only Residual-OPTR +500 snapshot.
- `proposals/residual_temporal_proposal_k8_alpha0.2.json`: frozen K=8, alpha=0.2 Residual proposal.
- `manifests/`: Bridge/RT1 provenance, content fingerprints and valid-anchor identity.
- `configs/`, `scripts/`, `starVLA/`: portable training and evaluation code.
- `EXPERIMENT_CONTRACT.md`: exact experimental contract and results.
- `AGENTS.md`: fast handoff instructions for an AI coding agent.
- `qwen_bridge_common_start_29k/checkpoints/steps_29000_pytorch_model.pt`: Qwen3-VL-4B–Bridge 29K model parameters.
- `qwen_bridge_common_start_29k/checkpoints/steps_29000_training_state.pt`: complete optimizer, scheduler, Python/NumPy/PyTorch/CUDA RNG and data-order state.
- `qwen_bridge_common_start_29k/config.full.yaml`: server-authoritative configuration that produced the 29K common start.
- `qwen_bridge_common_start_29k/ENVIRONMENT.md`: exact Qwen software and training environment.
- `qwen_bridge_common_start_29k/MULTISEED_PROTOCOL.md`: strict separation of common-start, continuation-training and evaluation seeds.
- `qwen_bridge_common_start_29k/AGENTS.md`: Qwen handoff constraints for an AI coding agent.

The 20.5K files are sufficient for evaluation but **not** for exact optimizer/RNG continuation. Any new branch must resume from the complete 20K state.

For Qwen–Bridge, exact continuation requires **both** 29K `.pt` files.

This release does not use implementation, code-path, absolute-path, current-manifest or self-referential SHA gates. Experimental identity is enforced through loadability and semantic model/data/episode/anchor/support/batch/budget checks.

## Florence–Bridge headline result

Fixed no-video SimplerEnv WidowX/Bridge panel: four policy seeds (`518957`, `765353`, `686765`, `516408`) × four tasks × 24 episodes = 384 rollouts.

| Method | Stack | Carrot | Spoon | Eggplant | Overall |
|---|---:|---:|---:|---:|---:|
| Uniform +500 | 13/96 | 31/96 | 50/96 | 68/96 | 162/384 = 42.19% |
| Residual-OPTR +500 | 24/96 | 45/96 | 83/96 | 76/96 | 228/384 = 59.38% |

## Florence–Bridge quick start

1. Download this repository and install `requirements.txt` in the same software environment used for StarVLA.
2. Download Florence-2-large and the OXE LeRobot Bridge/RT1 data.
3. Point the environment variables listed in `AGENTS.md` to those local assets.
4. Keep the contract fixed: two GPUs, BF16, ZeRO-2, micro-batch 4/GPU, accumulation 8, global batch 64, continuation seed 117465.
5. Run one arm:

```bash
bash scripts/run_florence_bridge_arm.sh uniform uniform-reproduction
bash scripts/run_florence_bridge_arm.sh residual residual-reproduction \
  proposals/residual_temporal_proposal_k8_alpha0.2.json
```

The launch path performs only file-presence and semantic data/support checks.

## Important implementation facts

- `framework.name: QwenGR00T` is a historical registry name; the base VLM is Florence-2-large.
- The actual Florence–Bridge run used `cross_attention_dim: 1024` and DeepSpeed ZeRO-2.
- The optimizer groups are the VLM interface and action model. No explicit freeze list was configured; the executed action path updated the Florence vision/text interface and action head, while decoder parameters outside that path did not receive updates.
- Residual sampling uses ordinary `p/q` objective correction. This release intentionally does not upload Matched-Random/shuffle assets.

See `SOURCE_INVENTORY.md` before changing any method or training field.