File size: 4,828 Bytes
977b06e 702bda7 977b06e 702bda7 977b06e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 | # Florence–Bridge experiment contract
## 1. Scope
This handoff covers exactly two published arms from one complete common start:
1. Uniform continuation, 20K → 20.5K.
2. Residual-OPTR continuation, K=8, alpha=0.2, with ordinary p/q correction, 20K → 20.5K.
Matched-Random/shuffle is intentionally excluded from the uploaded checkpoint set.
## 2. Common start and resume semantics
The parent is the Florence–Bridge 20K complete Accelerate checkpoint produced by the seed-42 scratch run. It contains the consolidated model, DeepSpeed ZeRO-2 optimizer shards, model-state shard, per-rank RNG states, `latest`, `checkpoint.json`, and `zero_to_fp32.py`.
The continuation restores the model, optimizer, scheduler position, data-order cursor and update counter from the parent. The external branch initializes its RNG streams with the configured continuation, model and transform seeds. It starts at completed step 20,000 and performs exactly 500 optimizer updates. The active learning rates at resume were `9.407412610098572e-06` for the VLM interface and `9.407412610098571e-05` for the action model. The restored scheduler was `LambdaLR(last_epoch=20000, step_count=20001)`.
The published 20.5K Uniform and Residual files are model-only evaluation snapshots. They are not substitutes for the full parent state.
## 3. Model and optimization
| Field | Value |
|---|---|
| Base VLM | Florence-2-large |
| Registry name | `QwenGR00T` (compatibility name only) |
| VLM hidden dim | 2048 |
| Action head | DiT-B, hidden/action hidden 1024 |
| Cross-attention dim | 1024 |
| Action/state dim | 7 / 7 |
| Horizon | 16 (future window 15) |
| DiT layers/dropout | 16 / 0.2 |
| Diffusion steps | model 8; training repeated steps 4; inference steps 4 |
| Precision/distribution | BF16, 2 GPUs, DeepSpeed ZeRO-2, no CPU offload |
| Batch | 4 per GPU × 2 GPUs × accumulation 8 = global 64 |
| Optimizer | AdamW, betas (0.9, 0.95), eps 1e-8, weight decay 1e-8 |
| Nominal LR | VLM interface 1e-5; action model 1e-4; base field 3e-5 |
| Scheduler | cosine-with-min-LR, warmup 5000, min LR 5e-7 |
| Gradient clipping | 1.0 |
| Seeds | parent 42; continuation/sampler 117465; model 588643; transform 298226 |
| Logging | every 10 steps in the portable handoff; historical main run logged every 50 |
See the [Bridge seed protocol](BRIDGE_SEED_PROTOCOL.md) for the reported multi-seed panel and the seeds used in Rebuttal reproductions.
No explicit `freeze_modules` or `trainable_modules` filter was configured. The optimizer contained the VLM-interface and action-model groups. In the actually executed action path, Florence vision/text interface parameters and the action head changed; decoder parameters outside that path received no update. Do not summarize this as “the entire VLM was frozen” or “every registered parameter was actively trained.”
## 4. Data and proposal
- Data mix: `bridge_rt_infoflow` (Bridge + RT1), OXE LeRobot layout.
- Action type/mode: `delta_ee` / `abs`.
- Image size: 224×224; video backend: `torchvision_av`.
- Source and trajectory allocation remain those of the common-start data contract.
- Residual proposal: schema 1, K=8 prospective grid, linear interpolation within each episode, percentile normalization within episode after interpolation, epsilon=0.1, alpha=0.2.
- Gate42F status: PASS. The proposal changes encounter frequency; ordinary p/q correction preserves the target objective.
- Valid-anchor, episode and proposal support identity are semantic requirements. Code location and absolute paths are not.
## 5. Fixed formal evaluation
Official SimplerEnv WidowX/Bridge setup, BF16 inference, no video:
- policy seeds: 518957, 765353, 686765, 516408;
- tasks: Stack, Carrot, Spoon, Eggplant;
- 24 episodes per policy-seed/task;
- 96 rollouts per task and per seed; 384 per checkpoint.
### Uniform
| Policy seed | Success |
|---|---:|
| 518957 | 41/96 |
| 765353 | 39/96 |
| 686765 | 43/96 |
| 516408 | 39/96 |
Tasks: Stack 13/96, Carrot 31/96, Spoon 50/96, Eggplant 68/96. Overall: **162/384 = 42.1875%**.
### Residual-OPTR
| Policy seed | Success |
|---|---:|
| 518957 | 50/96 |
| 765353 | 60/96 |
| 686765 | 59/96 |
| 516408 | 59/96 |
Tasks: Stack 24/96, Carrot 45/96, Spoon 83/96, Eggplant 76/96. Overall: **228/384 = 59.375%**.
## 6. Reproduction acceptance checks
A reproduction is comparable only if it verifies:
1. the full 20K state loads and the update counter resumes at 20K;
2. world size 2 and global batch 64;
3. dataset/episode/valid-anchor identity and proposal full support;
4. proposal weights are p/q for the Residual arm;
5. step 20500 is saved cleanly;
6. formal evaluation contains exactly 384 raw outcomes with the locked seeds/tasks.
Do not introduce code/path checksums, self-referential manifest checks, or migration-file identity gates.
|