READMEs: step 5500 numbers, fwd step 1500 row

README.md

@@ -1,142 +1,48 @@
----
-pipeline_tag: robotics
-tags:
-- openvla
-- vision-language-action
-- robotics
-- libero
----
+# TERRA — model checkpoints
 
-# TERRA — post-trained policies (real robot + LIBERO)
+OpenVLA-7B + TERRA latent-action phase 2 (`latp2_inv` labels), post-trained with a flow-matching action head and
+4 query tokens. **One directory = one model**, each with its own README (what it is, how to deploy it, what has been
+measured). Every deployable directory holds merged weights (`model-*.safetensors`), the action head
+(`action_decoder.pt`, required), `dataset_statistics.json` (un-normalisation), `ctrl_meta.json` and the processor.
 
-OpenVLA-7B with TERRA latent-action phase 2 (`latp2_inv` labels), post-trained with a flow-matching action head and
-4 query tokens. **One directory per policy**, each self-contained: merged weights (`model-*.safetensors`), the
-action/latent head (`action_decoder.pt` — **required**, it is not inside the merged VLA weights),
-`dataset_statistics.json` (action un-normalisation), `ctrl_meta.json`, processor/tokenizer, and its own `README.md`
-with the measured numbers and the exact deployment loop.
-
-**Code:** https://github.com/DingTianxingjian/Latent_Free — branch `newton`, public, no token needed.
+**Code:** https://github.com/DingTianxingjian/Latent_Free — branch `newton` (deployment wrapper: `acdc_la/realrobot/piper_policy.py`).
 
 ```bash
 git clone -b newton https://github.com/DingTianxingjian/Latent_Free.git latent_free && cd latent_free
 huggingface-cli download Potestates/TERRA --include "<dir>/*" --local-dir .
 ```
 
----
-
-## Which one do I test?
-
-| I want to… | use | why |
-|---|---|---|
-| **press a coloured button** | **`press_button_terra_delta`** | the only policy **confirmed working on hardware** — it presses the requested button |
-| **pick the can, place on plate** | **`piper_pick_can_openvla_delta`** | **3x better open loop** than the arm currently on the robot (1.05 vs 3.24 deg); simplest of the equals |
-| the same, with our full method | `piper_pick_can_terra_delta` | statistically **indistinguishable** from the row above (1.13 vs 1.05, CI contains 0) — either is fine |
-| **one model for both tasks** | `piper_multi_anchored_step5500` | **experimental**, and a retrain will supersede it — see its README |
-| post-train a new task | `p2fft_20k` | the **base**, not a policy |
-
-> ### ⚠ Two things that will break a deployment if you skip them
-> 1. **Action space differs per directory.** `piper_pick_can_10k` emits **absolute** joint targets; everything with
->    `delta` in the name emits **target minus measured joint angle** and must be sent as `q + a[:6]` with `q` re-read
->    **every tick**; the `anchored` ones use `q` read **once per chunk** and a different call. Every `ctrl_meta.json`
->    now records `"action_space"` explicitly and `PiperPolicy` prints it on load — **check the printed line.**
-> 2. **Execute 1 chunk entry, then re-predict** (all per-step `delta` policies). The demos were teleoperated as a
->    continuous 30 Hz command, so later chunk entries are mis-anchored for an arm that settles: drift is
->    **0.855 deg/step on pick-can** and **~2 deg/step on press_button**, and it is opposite-signed on the two joints
->    that carry the motion, so it **bends the direction** instead of just scaling it. This was seen on the robot:
->    chunk 1 presses the button, chunk 2 "the direction is a bit off". Use an `anchored` checkpoint if you need
->    multi-step chunks.
-
----
+## Which model? (快速选择)
 
-## Real-robot policies — AgileX Piper
-
-Open-loop error = mean |error| per joint in **degrees** on held-out episodes never trained on, chunk sampled from
-noise through the deployment image path. Reproduce any row with
-`envs/univla_vla/bin/python -m acdc_la.realrobot.piper_policy --checkpoint <dir> --replay <rlds root>` — it is
-deterministic, and reproducing it is the check that your preprocessing matches ours.
-
-### `press_button` — "press the {blue, orange, pink, red} button", 100 episodes, layout reshuffled per episode
-
-| dir | base | action | `chunk[0]` | reads the instruction? | on the robot |
+| dir | role | task | action (dims 0-5) | how to run it on the Piper | status |
 |---|---|---|---|---|---|
-| **`press_button_terra_delta`** | our phase 2 | delta | **0.918** | **yes** — output moves 0.42 deg on a colour swap; true word wins 4/5 episodes | ✅ **presses the button** (chunk 1; slightly off centre) |
-| `press_button_openvla_delta` | stock openvla-7b | delta | 1.920 | **no** — output moves **2.4e-7 deg**; true word wins 1/5 = chance | ❌ not for deployment |
-| *null: "stay where you are"* | | | *1.691* | | |
-
-The control is *worse than not moving*, and its invariance to language is **exact, not approximate**: its instruction
-signal is 0.28% of the query-token state, which is below bfloat16 machine epsilon (7.8e-3), so it cannot survive its
-own forward pass. With 4 colours and per-episode layouts a policy that cannot read the colour can only learn an
-average press — which is what its j1/j2 error looks like. Caveats: **5 held-out episodes, 3 of 4 colours, one seed
-per arm, offline**; and the pair changes *both* the base and the latent branch, so the effect cannot be attributed to
-the latent branch alone.
-
-### `piper_pick_can` — "pick up the can and place it on the plate", 150 episodes, **one** instruction throughout
-
-| dir | base | action | `chunk[0]` | ensembled | note |
-|---|---|---|---|---|---|
-| **`piper_pick_can_openvla_delta`** | stock openvla-7b | delta | **1.054** | 0.914 | the matched control — and here a perfectly good policy |
-| **`piper_pick_can_terra_delta`** | our phase 2 | delta | **1.126** | 0.950 | ours; **not separated** from the row above |
-| `piper_pick_can_10k` | our phase 2 | **absolute** | 3.24 | 3.69 | the arm tested on the robot first; **superseded** by the two above |
-| *null: "stay where you are"* | | | *1.20* | | |
-| *null: repeat the last command* | | | *2.06* | | |
-| *between-demonstration floor* | | | *0.61* | | **nothing can beat this** |
-
-**Delta vs absolute is the real win here: 3.24 → ~1.05 deg, 3x**, and it moves the policy from 2.7x *worse* than
-doing nothing to better than it. **TERRA vs control is not a win for either side**: paired cluster bootstrap over the
-8 held-out episodes (the independent unit — 353 steps is not the sample size) gives a difference of +0.072 deg,
-95% CI **[-0.038, +0.189]**, sign test p = 0.727.
-
-A null here is what the mechanism's own story predicts: **all 150 episodes carry the identical instruction**, so
-there is nothing for a language-grounding advantage to do. The same pair separates 2.1x on `press_button`.
-
-### `piper_multi_anchored_step2500`, `piper_multi_anchored_step5500` — ⚠ experimental
-
-One model for **both** tasks (press_button x2 + pick-can mixture), using **chunk-ANCHORED** deltas
-(`target[s+k] - q[s]`, `q` read once per chunk) so that executing several entries open loop is valid. Better than the
-per-step parameterisation from k>=2, slightly worse at k=0. Trained with the old backward sample window (magnitudes
-under-predicted ~0.8x); **a forward-window retrain is in progress and will supersede these.** Different deployment
-call (`chunk_targets`, no ensembling) — read their README.
-
----
-
-## LIBERO policies
-
-| dir | suite | demos | success, standard 500-episode eval |
-|---|---|---|---|
-| `libero_10_full` | `libero_10_no_noops` | full | **89.0** |
-| `libero_goal_full` | `libero_goal_no_noops` | full | **94.4** |
-
-```bash
-python -m acdc_la.downstream.vla.run_libero_eval_cont --pretrained_checkpoint <dir>
-```
-
----
-
-## `p2fft_20k` — the phase-2 base (not a deployable policy)
-
-The **base** every TERRA policy above is post-trained from: OpenVLA-7B fully fine-tuned for 20,000 steps on OXE to
-predict our continuous latent action (`latp2_inv`). `vla_full.pt` (15,081,838,026 B) is the VLA state dict, and
-**`action_decoder.pt` (105,268,830 B) is required too** — `finetune_libero_ctrl.py:273` initialises the post-training
-heads from it, so a run without it starts from a random head and will not reproduce these policies.
-
-```bash
-huggingface-cli download Potestates/TERRA --include "p2fft_20k/*" --local-dir .
-INIT=./p2fft_20k  ...  bash acdc_la/scripts/run_vla_post2_portable.sh
-```
-
-The matched control is `INIT=none LATENT=none`, which post-trains the official `openvla/openvla-7b` with everything
-else identical — that is how "what did phase 2 buy" is measured, and it is what the two `*_openvla_delta`
-directories are.
-
----
-
-## One metric trap, worth repeating
-
-`action_l1_1step` in every `train_log.jsonl` is **oracle-conditioned**: `FlowHeads.loss` scores a one-step
-correction from a point that already contains the ground truth. On the pick-can pair it ranks the two arms
-**backwards** (TERRA 0.0563 vs control 0.0648, i.e. TERRA "13% better", while both free-running read-outs favour the
-control). Use it to track a single run or pick a checkpoint within one run — **never to compare two arms.** Rank with
-`--replay`.
-
-`HANDOFF_press_button.md` at the root of this repo is the brief for training the next real-robot task on a fresh
-host. The full experiment ledger, including everything retracted or superseded, is `LEDGER.md` in the code repo.
+| `press_button_terra_delta` | **policy** | press the {blue, orange, pink, red} button | per-step **delta** `target − q` | `to_joint_targets(a, q)`, re-read `q` EVERY tick, **chunk = 1** | **pressed the button on the robot** (chunk=1; slightly off-centre). chunk ≥ 2 bends the direction — don't |
+| `press_button_anchored_fwd_step1500` | **policy, newest fix** | press the {blue, orange, pink, red} button | **chunk-anchored delta**, fixed training window | read `q` ONCE per chunk, `chunk_targets(chunk, q)`, N = 1 / 2 / 4 | early snapshot (1500/3000) of the window-fixed run; best offline at N = 4 (2.28 deg); not yet on the robot |
+| `piper_multi_anchored_step5500` | **policy, one model for both tasks** | press button (4 colours) + pick up the can and place it on the plate | **chunk-anchored delta** `target[s+k] − q[s]` | read `q` ONCE per chunk, `chunk_targets(chunk, q)`, execute N = 1 / 2 / 4 | experimental, latest snapshot (step 5500); not yet run on the robot |
+| `piper_multi_anchored_step2500` | same run, earlier snapshot | same | same | same | superseded by step 5500; kept for comparison |
+| `piper_pick_can_10k` | policy | `pick up the can and place it on the plate` | **absolute** joint target | send `chunk[k]` directly | offline only (3.24 deg) |
+| `press_button_openvla_delta` | **control (baseline)** | press_button | per-step delta | as `press_button_terra_delta` | official OpenVLA-7B, same recipe: 1.92 vs 0.92 deg, **ignores the colour word** — for the comparison, not for use |
+| `libero_10_full` | policy (sim) | LIBERO-10 | LIBERO EEF delta | `run_libero_eval_cont` | 89.0 % (500 episodes) |
+| `libero_goal_full` | policy (sim) | LIBERO-Goal | LIBERO EEF delta | `run_libero_eval_cont` | 94.4 % (500 episodes) |
+| `p2fft_20k` | **pretrained base**, not a policy | — | — | `INIT=./p2fft_20k` for post-training | phase 2, 20k steps on OXE |
+
+### The three action spaces on the Piper — getting this wrong drives the arm somewhere else
+
+`ctrl_meta.json → action_space` says which one a directory uses, and `PiperPolicy` reads it.
+
+* **absolute** (no `action_space` key): `chunk[k]` IS the joint target.
+* **delta** (`"delta"`): `chunk[k]` = target − measured joints *at step k*. Send `q_now + chunk[k]`, `q` re-read every
+  tick. Use **chunk = 1**: the demos were teleoperated at 30 Hz with a continuous command, so the demonstrated arm
+  runs ~0.86–2 deg/step *ahead* of where an arm that settles at each 10 Hz command would be; with chunk = N that lead
+  adds up linearly and with OPPOSITE signs on j1/j2 (−0.74 / +0.75 deg per step on pick-can) — the direction bends.
+* **delta_anchored** (`"delta_anchored"`): `chunk[k]` = target[s+k] − joints read ONCE at the chunk's start. Send
+  `q_s + chunk[k]`; the targets do not depend on how the arm tracks inside the chunk, so the tracking drift above
+  does not exist — what remains is the model's own error, which grows with k (see the directory's README).
+
+The method ledger is `LEDGER.md` in the code repo. `HANDOFF_press_button.md` at the root of this repo is the brief
+for training a new real-robot task.
+
+## `p2fft_20k` — the phase-2 pretrained base
+
+See `p2fft_20k/README.md`. Post-training from it: `INIT=./p2fft_20k ... bash acdc_la/scripts/run_vla_post2_portable.sh`;
+the matched control is `INIT=none` (official `openvla/openvla-7b`, everything else identical).

piper_multi_anchored_step5500/README.md

@@ -38,9 +38,10 @@ Held-out press_button (5 episodes), open-loop target error by chunk position k (
 
 | k | 0 | 1 | 2 | 3 | 7 | 11 |
 |---|---|---|---|---|---|---|
+| **step 5500 (this, anchored, exact)** | 0.89 | 1.98 | 3.22 | 4.50 | 9.21 | 13.43 |
 | step 2500 (anchored, exact) | 1.12 | 2.43 | 3.74 | 4.87 | 10.04 | 15.91 |
 | `press_button_terra_delta`, ideal tracking | 0.87 | 2.45 | 4.33 | 6.36 | 14.39 | 22.43 |
 
-Better from k>=2, worse at k=0. Trained with the old BACKWARD sample window (input frame misaligned with the action
+Step 5500: on par at k=0 (0.89 vs 0.87), better from k=1. Executing 2 / 4 steps: 1.44 / 2.65 deg vs 1.66 / 3.50 for the per-step model. Trained with the old BACKWARD sample window (input frame misaligned with the action
 chunk at episode starts, magnitudes under-predicted ~0.8x); a forward-window retrain is in progress and will
 supersede this. Its value is a robot check: anchored targets do not depend on how the arm tracks within a chunk.