pepijn223 HF Staff commited on
Commit
6419f04
·
unverified ·
1 Parent(s): 7f41624

Restructure blog from paper format to chronological narrative

Browse files

Reorganize Train/Eval/Experiments sections so the reader follows
the iterative journey rather than seeing all settings then all results.

- Simplify 05 to Data Overview (filtering/SARM details move to journey)
- Merge 06 (Training) + 07 (Evaluation) into Model and Evaluation Setup
- Remove 07 import from article.mdx (content now in 06)
- Rewrite 08 as chronological narrative: baseline → data curation →
algorithmic improvements (SARM, RABC, relative actions, DAgger) →
breakthrough with high-quality data → what didn't work → full comparison
- Slim 09 Learnings (DAgger now covered in journey chapter)

Made-with: Cursor

app/src/content/article.mdx CHANGED
@@ -56,7 +56,6 @@ import Hardware from "./chapters/folding/03-hardware.mdx";
56
  import DataCollection from "./chapters/folding/04-data-collection.mdx";
57
  import DataDiversity from "./chapters/folding/05-data-diversity.mdx";
58
  import Training from "./chapters/folding/06-training.mdx";
59
- import Evaluation from "./chapters/folding/07-evaluation.mdx";
60
  import Experiments from "./chapters/folding/08-ablations.mdx";
61
  import Learnings from "./chapters/folding/09-learnings.mdx";
62
  import References from "./chapters/folding/12-references.mdx";
@@ -82,8 +81,6 @@ import VibeCodingCharts from "./chapters/demo/vibe-coding-charts.mdx";
82
 
83
  <Training />
84
 
85
- <Evaluation />
86
-
87
  <Experiments />
88
 
89
  <Learnings />
 
56
  import DataCollection from "./chapters/folding/04-data-collection.mdx";
57
  import DataDiversity from "./chapters/folding/05-data-diversity.mdx";
58
  import Training from "./chapters/folding/06-training.mdx";
 
59
  import Experiments from "./chapters/folding/08-ablations.mdx";
60
  import Learnings from "./chapters/folding/09-learnings.mdx";
61
  import References from "./chapters/folding/12-references.mdx";
 
81
 
82
  <Training />
83
 
 
 
84
  <Experiments />
85
 
86
  <Learnings />
app/src/content/chapters/folding/05-data-diversity.mdx CHANGED
@@ -1,18 +1,13 @@
1
- import Note from "../../../components/Note.astro";
2
- import Sidenote from "../../../components/Sidenote.astro";
3
  import Wide from "../../../components/Wide.astro";
4
- import Accordion from "../../../components/Accordion.astro";
5
  import diversityGridImg from "../../assets/image/lerobot-data-collection_level12_rac_2_2026-02-08_1_grid_15x10.jpg";
6
 
7
- ## Data and Diversity
8
 
9
- Raw episodes are only the beginning. What you do with them before training determines whether your model learns to fold or learns to fumble.
10
-
11
- We collected two datasets: a **full dataset** containing every episode, and a **curated dataset** built by selecting the best episodes from the full set and supplementing them with additional high-quality recordings.
12
 
13
  ### Dataset Statistics
14
 
15
- | Metric | Large dataset | High-quality dataset |
16
  |:---|:---:|:---:|
17
  | Total episodes | **5,688** | **1,200** |
18
  | Total frames | **14.1M** | **3.2M** |
@@ -29,23 +24,10 @@ The grid below shows one frame from each of 100 different episodes. Notice the v
29
  <img src={diversityGridImg.src} alt="Trajectory diversity grid showing variation in t-shirt color, background, camera viewpoint, and robot height" style="width:100%; border-radius: 8px;" />
30
  </Wide>
31
 
32
- ### Data Augmentation and Curation
33
-
34
- #### Filtering
35
-
36
- We filtered episodes in two ways:
37
-
38
- 1. **End-state image filtering** to discard episodes where the final frame doesn't show a properly folded shirt. If the end result isn't good, the demonstration isn't useful.
39
- 2. **Length-based filtering** using the LeRobot data visualizer to remove outliers. Episodes that are suspiciously short tend to be low quality.
40
-
41
- The [LeRobot Data Visualizer](https://huggingface.co/spaces/lerobot/visualize_dataset) was invaluable for inspecting the dataset, spotting outliers, and understanding distributions. Try it right here with our dataset:
42
 
43
  <Wide>
44
  <div className="card" style="overflow: hidden; border-radius: 10px;">
45
  <iframe src="https://lerobot-visualize-dataset.hf.space/?path=%2Flerobot%2Fhigh_quality_folding%2Fepisode_0" width="100%" height="800" frameBorder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" style="border: none;"></iframe>
46
  </div>
47
  </Wide>
48
-
49
- #### SARM Annotation
50
-
51
- We also annotated every episode using our trained **[Stage-Aware Reward Modeling (SARM)](https://huggingface.co/docs/lerobot/sarm)** reward model. This gave us continuous scores we could use as weights at training time. More details in [SARM: Our Reward Model](#sarm-our-reward-model).
 
 
 
1
  import Wide from "../../../components/Wide.astro";
 
2
  import diversityGridImg from "../../assets/image/lerobot-data-collection_level12_rac_2_2026-02-08_1_grid_15x10.jpg";
3
 
4
+ ## Data Overview
5
 
6
+ After weeks of collection we had **5,688 episodes** across 8 setups. Not all of them are equally useful — later we curated a smaller, high-quality subset. Here are both datasets side by side:
 
 
7
 
8
  ### Dataset Statistics
9
 
10
+ | Metric | Full dataset | High-quality dataset |
11
  |:---|:---:|:---:|
12
  | Total episodes | **5,688** | **1,200** |
13
  | Total frames | **14.1M** | **3.2M** |
 
24
  <img src={diversityGridImg.src} alt="Trajectory diversity grid showing variation in t-shirt color, background, camera viewpoint, and robot height" style="width:100%; border-radius: 8px;" />
25
  </Wide>
26
 
27
+ Explore the dataset yourself with the [LeRobot Data Visualizer](https://huggingface.co/spaces/lerobot/visualize_dataset):
 
 
 
 
 
 
 
 
 
28
 
29
  <Wide>
30
  <div className="card" style="overflow: hidden; border-radius: 10px;">
31
  <iframe src="https://lerobot-visualize-dataset.hf.space/?path=%2Flerobot%2Fhigh_quality_folding%2Fepisode_0" width="100%" height="800" frameBorder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" style="border: none;"></iframe>
32
  </div>
33
  </Wide>
 
 
 
 
app/src/content/chapters/folding/06-training.mdx CHANGED
@@ -2,16 +2,11 @@ import Note from "../../../components/Note.astro";
2
  import Sidenote from "../../../components/Sidenote.astro";
3
  import Accordion from "../../../components/Accordion.astro";
4
  import Wide from "../../../components/Wide.astro";
5
- import Stack from "../../../components/Stack.astro";
6
  import HtmlEmbed from "../../../components/HtmlEmbed.astro";
7
 
8
- import sarmEp300 from "../../assets/image/lerobot-data-collection_level2_final_quality3_ep300_progress.gif";
9
- import sarmEp2500 from "../../assets/image/lerobot-data-collection_level12_rac_2_2026-02-08_1_ep2500_progress.gif";
10
- import sarmEp2200 from "../../assets/image/lerobot-data-collection_level12_rac_2_2026-02-08_1_ep2200_progress.gif";
11
 
12
- ## Training
13
-
14
- Before talking about hyperparameters, one needs to understand what the trained model actually *is*: what it takes in, what it produces, and why those choices matter for cloth folding.
15
 
16
  ### Model Architecture
17
 
@@ -33,8 +28,6 @@ The model generates actions through **flow matching**, a generative approach tha
33
  Flow matching is closely related to diffusion models but uses a simpler, more direct interpolation path between noise and data.
34
  </Sidenote>
35
 
36
- Before diving further into the model architecture, let's introduce a few key ingredients: [RTC](https://huggingface.co/docs/lerobot/rtc), [SARM](https://huggingface.co/docs/lerobot/sarm) and [relative actions](https://huggingface.co/docs/lerobot/action_representations).
37
-
38
  #### [Real-Time Chunking (RTC)](https://huggingface.co/docs/lerobot/rtc)
39
 
40
  RTC is crucial for real-world deployment. Instead of waiting for the predicted action chunk to finish before generating the next, RTC generates the next chunk while executing the current one. It "freezes" actions that are already committed and "inpaints" the remaining ones, producing smooth asynchronous motion. In practice, this sped up our rollouts by at least a factor of 2.
@@ -50,38 +43,11 @@ sequenceDiagram
50
  end
51
  ```
52
 
53
- #### [Stage-Aware Reward Modeling (SARM)](https://huggingface.co/docs/lerobot/sarm)
54
-
55
- SARM is a trained reward model that scores trajectories based on how well the robot is progressing toward task completion, it acts as a learned "critic" that predicts whether things are going well or badly.
56
-
57
- SARM is trained on our demonstration data to predict 0-1 task progression. The takeaway: it correctly identifies **mistakes** (drops in value) and **progress** (increases) in real time.
58
-
59
- <Wide>
60
- <Stack layout="3-column" gap="small">
61
- <img src={sarmEp300.src} alt="SARM annotation on episode 300" style="width:100%; border-radius: 8px;" />
62
- <img src={sarmEp2500.src} alt="SARM annotation on episode 2500" style="width:100%; border-radius: 8px;" />
63
- <img src={sarmEp2200.src} alt="SARM annotation on episode 2200" style="width:100%; border-radius: 8px;" />
64
- </Stack>
65
- </Wide>
66
-
67
- We use SARM exclusively for **RABC** (Reward-Advantage-Based Conditioning): every episode is scored with a per-timestep quality signal, and during training, actions are weighted by their contribution to progress. High-reward actions contribute more to the loss, low-reward ones contribute less. Negative progress is clipped to 0. Unlike binary success/fail labels, SARM provides a continuous signal at every timestep.
68
-
69
- #### [Relative actions representation](https://huggingface.co/docs/lerobot/action_representations)
70
-
71
- Following [UMI](https://arxiv.org/abs/2402.10329) approach, actions are expressed as **relative trajectories**: each action in the chunk is an offset from the robot's current state at prediction time, not from the previous action. This avoids error accumulation (unlike true delta) and doesn't require a global coordinate frame (unlike absolute).
72
-
73
- <HtmlEmbed
74
- id="action-representations"
75
- src="folding/action-representations.html"
76
- title="Action Representations"
77
- desc="Relative trajectory (blue) references all actions to the current state. Delta (yellow) chains each action to the previous one, accumulating error. Absolute (red) requires a global coordinate frame. Diagram adapted from UMI (Chi et al., 2024)."
78
- />
79
-
80
- See the [LeRobot action representations docs](https://huggingface.co/docs/lerobot/action_representations) for a full guide on how to use relative actions in your own training.
81
 
82
  ### Models
83
 
84
- We initially trained multiple architectures supported in LeRobot, but we ended up focusing on two VLA architectures for our cloth folding data:
85
 
86
  - **π0** the base flow-matching VLA, trained with standard imitation learning
87
  - **[π0.5](https://huggingface.co/docs/lerobot/pi05)** an improved variant with more pretraining and several improvements to the flow matching denoising process
@@ -97,22 +63,67 @@ Both are finetuned from pretrained checkpoints. Starting from this pretrained fo
97
  | Action chunk size | **30** |
98
  | Optimizer | AdamW |
99
  | Learning rate | **1e-4** (with warmup + cosine decay) |
100
- | Training steps | **200k** (Series 1) / **100k** (Series 2 fine-tune) |
101
 
102
  <Sidenote>
103
  Multi-GPU training with 8xH100 and gradient accumulation was necessary to fit the large batch sizes needed for stable VLA training.
104
  </Sidenote>
105
 
106
- ### Loss Curves
107
 
108
- <Wide>
109
- <HtmlEmbed
110
- id="loss-curves"
111
- src="folding/loss-curves.html"
112
- title="Training Loss"
113
- desc="Click legend entries to toggle runs. <strong>Series 1</strong>: trained from pretrained checkpoints on full data (200k steps). <strong>Series 2</strong>: fine-tuned on curated high-quality data (100k steps)."
114
- frameless
115
- />
116
- </Wide>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
117
 
118
- Our training followed two phases: **Series 1** trained from pretrained base checkpoints on the full dataset for 200k steps. **Series 2** fine-tuned the best Series 1 checkpoint on curated high-quality data for 100k steps.
 
2
  import Sidenote from "../../../components/Sidenote.astro";
3
  import Accordion from "../../../components/Accordion.astro";
4
  import Wide from "../../../components/Wide.astro";
 
5
  import HtmlEmbed from "../../../components/HtmlEmbed.astro";
6
 
7
+ ## Model and Evaluation Setup
 
 
8
 
9
+ Before diving into experiments, let's establish the model architecture we're training and how we measure performance. These stay constant across every iteration.
 
 
10
 
11
  ### Model Architecture
12
 
 
28
  Flow matching is closely related to diffusion models but uses a simpler, more direct interpolation path between noise and data.
29
  </Sidenote>
30
 
 
 
31
  #### [Real-Time Chunking (RTC)](https://huggingface.co/docs/lerobot/rtc)
32
 
33
  RTC is crucial for real-world deployment. Instead of waiting for the predicted action chunk to finish before generating the next, RTC generates the next chunk while executing the current one. It "freezes" actions that are already committed and "inpaints" the remaining ones, producing smooth asynchronous motion. In practice, this sped up our rollouts by at least a factor of 2.
 
43
  end
44
  ```
45
 
46
+ All experiments use **RTC** with an action queue size of 30 and a maximum action horizon of 20, along with **action interpolation** (upsampling from 30 Hz to 90 Hz) which makes the robot much quieter and smoother.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
 
48
  ### Models
49
 
50
+ We focused on two VLA architectures for our cloth folding data:
51
 
52
  - **π0** the base flow-matching VLA, trained with standard imitation learning
53
  - **[π0.5](https://huggingface.co/docs/lerobot/pi05)** an improved variant with more pretraining and several improvements to the flow matching denoising process
 
63
  | Action chunk size | **30** |
64
  | Optimizer | AdamW |
65
  | Learning rate | **1e-4** (with warmup + cosine decay) |
 
66
 
67
  <Sidenote>
68
  Multi-GPU training with 8xH100 and gradient accumulation was necessary to fit the large batch sizes needed for stable VLA training.
69
  </Sidenote>
70
 
71
+ ---
72
 
73
+ ### Evaluation Protocol
74
+
75
+ **Evaluation is as hard as training.** In robotics and with real hardware, no standardized benchmarks exist. If your evaluation protocol is inconsistent, every downstream decision will be wrong.
76
+
77
+ For every experiment, we evaluate the model on:
78
+
79
+ - **5 different t-shirts for Level 1** (laid-out to fold)
80
+ - **5 different t-shirts for Level 2** (messy to spread to fold, then place aside)
81
+
82
+ Each t-shirt fold is attempted **twice consecutively**, giving **10 rollouts per level** and **20 rollouts total per experiment**. Every evaluation is filmed and scored from video afterward, so judgment is decoupled from execution.
83
+
84
+ <Note>
85
+ The evaluation protocol t-shirts, attempts count, scoring rubric, and filming setup is identical across every experiment.
86
+ </Note>
87
+
88
+ ### Metrics
89
+
90
+ We report four complementary metrics:
91
+
92
+ 1. **Success Rate** Binary pass/fail per rollout.
93
+ 2. **Score** Partial credit based on subtasks completed. This distinguishes a model that consistently reaches Fold 3 from one that fails at Unfold, even if neither achieves full success.
94
+
95
+ <Accordion title="Scoring rubric Level 1 and Level 2">
96
+
97
+ **Level 1** Laid-out shirt to fold (shirt starts flat):
98
+
99
+ | Subtask | Points |
100
+ |:---|:---:|
101
+ | Do first horizontal fold | +10 |
102
+ | Do second horizontal fold | +10 |
103
+ | Do third vertical fold | +10 |
104
+ | Do final fold | +10 |
105
+ | Rotate | +10 |
106
+ | **Maximum per rollout** | **50** |
107
+
108
+ **Level 2** Messy shirt to spread, fold, and place aside:
109
+
110
+ | Subtask | Points |
111
+ |:---|:---:|
112
+ | Unfold (spread the shirt) | +50 |
113
+ | Fold 1 | +10 |
114
+ | Fold 2 | +10 |
115
+ | Fold 3 | +10 |
116
+ | Fold 4 | +10 |
117
+ | Rotation + Place aside | +10 |
118
+ | **Maximum per rollout** | **100** |
119
+
120
+ Scores are summed across all the rollouts in an experiment. With 10 Level 1 rollouts (max 50 points each) and 10 Level 2 rollouts (max 100 points each), the **maximum total score per experiment is 1,500 points**.
121
+
122
+ </Accordion>
123
+
124
+ 3. **Fold quality** A 1–5 rating of the final fold appearance, averaged across successful rollouts.
125
+ 4. **Completion time** Seconds to complete Level 1/Level 2, averaged across successful rollouts.
126
+
127
+ ### Statistical uncertainty
128
 
129
+ With 20 rollouts per experiment, even large apparent differences can be statistically indistinguishable. We report **Clopper-Pearson 90% confidence intervals** on all success rates and run formal pairwise significance tests using [STEP](https://tri-ml.github.io/step/) (Sequential Testing for Efficient Policy comparison). Running 50-100 rollouts per experiment would give tighter estimates but was not feasible for us across 11 experiments on real hardware.
app/src/content/chapters/folding/08-ablations.mdx CHANGED
@@ -1,12 +1,160 @@
1
  import Note from "../../../components/Note.astro";
2
  import Sidenote from "../../../components/Sidenote.astro";
3
  import Wide from "../../../components/Wide.astro";
 
4
  import Accordion from "../../../components/Accordion.astro";
5
  import HtmlEmbed from "../../../components/HtmlEmbed.astro";
6
 
7
- ## Experiments
 
 
8
 
9
- We ran 11 experiments to understand what *actually* matters. **Series 1** trains from pretrained base checkpoints on the full dataset. **Series 2** finetunes Series 1 checkpoints on curated high-quality data (2.1–2.4 from 1.3, 2.5 from 1.7). One early lesson: **undertraining makes the policy shaky** - make sure your model has converged before drawing conclusions.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
 
11
  <Wide>
12
 
@@ -26,16 +174,6 @@ We ran 11 experiments to understand what *actually* matters. **Series 1** trains
26
 
27
  </Wide>
28
 
29
- All experiments use **[RTC](https://huggingface.co/docs/lerobot/rtc)** (Real-Time Chunking) with an action queue size of 30 and a maximum action horizon of 20, along with **action interpolation** (upsampling from 30 Hz to 90 Hz).
30
-
31
- RTC gave us a ~2x speedup (sometimes even 2.5x) and action interpolation made the robot much quieter and smoother. Both are now available on [LeRobot repository](https://github.com/huggingface/lerobot).
32
-
33
- ---
34
-
35
- ### Results Overview
36
-
37
- Now let's look at how each experiment actually performed. The charts below show success rates, scores, completion times, and failure modes across all 11 experiments. The pattern is consistent: **Series 2 dominates Series 1**, and within each series, RABC combined with relative actions produces the best results. We break down the key findings below.
38
-
39
  <HtmlEmbed
40
  id="success-rates"
41
  src="folding/success-rates.html"
@@ -43,8 +181,6 @@ Now let's look at how each experiment actually performed. The charts below show
43
  desc="Success rates (Total, Level 1, Level 2) across all experiments. Series 1 trains from scratch on full data; Series 2 finetunes the best Series 1 checkpoint on curated high-quality data."
44
  />
45
 
46
- The gap between Series 1 and Series 2 is immediately visible. Experiment 2.5 reaches 90% total success rate (100% Level 1, 80% Level 2), while the best Series 1 result tops out at 40%. No Series 1 experiment achieves a single Level 2 success.
47
-
48
  <HtmlEmbed
49
  id="total-score"
50
  src="folding/total-score.html"
@@ -52,7 +188,7 @@ The gap between Series 1 and Series 2 is immediately visible. Experiment 2.5 rea
52
  desc="Overall score (% of maximum 1500) per experiment. The dashed line marks 50% as a reference point."
53
  />
54
 
55
- Total score captures partial progress that binary success rate misses. Even failed rollouts earn credit for completed subtasks, revealing that some Series 1 experiments make meaningful progress despite 0% Level 2 success. Only two experiments break the 50% threshold, all from Series 2.
56
 
57
  <HtmlEmbed
58
  id="l1-time-quality"
@@ -61,7 +197,7 @@ Total score captures partial progress that binary success rate misses. Even fail
61
  desc="Each bubble is one experiment. X-axis = completion time (faster is left), Y-axis = fold quality (higher is better), bubble size = total success rate. The best experiments cluster in the top-left corner."
62
  />
63
 
64
- Speed and quality correlate strongly with data quality. Series 2 experiments fold 2-3x faster than Series 1 (40s vs 100s+), and fold quality only breaks past 3.0 with high-quality training data. Faster isn't a separate goal from better; it's a consequence of the policy learning a clear, unambiguous strategy.
65
 
66
  <HtmlEmbed
67
  id="subtask-heatmap"
@@ -70,11 +206,9 @@ Speed and quality correlate strongly with data quality. Series 2 experiments fol
70
  desc="Average time (seconds) per subtask across all experiments. Green is fast, red is slow. Dashes indicate the subtask was never completed."
71
  />
72
 
73
- The heatmap shows where time is spent. Series 1 experiments are slow across the board, especially on Unfold. Series 2 compresses most subtasks to under 10 seconds, with the remaining time concentrated on Unfold for Level 2, which is inherently the most complex step.
74
-
75
  ### Where the policies fail
76
 
77
- Before interpreting success rates, understanding *how* each experiment fails — not just whether it fails — is essential.
78
 
79
  <HtmlEmbed
80
  id="failure-analysis"
@@ -95,63 +229,3 @@ With 20 rollouts per experiment, not every visible gap is real. We run **[STEP](
95
  />
96
 
97
  The CLD tells a clear story: **Series 2 separates from Series 1**. Experiments 2.5 and 2.2 form group 'a' — statistically distinct from the bulk of Series 1 experiments which cluster in groups 'c' and 'd'. Within Series 1, no experiment separates from another, meaning none of the algorithmic changes (RABC, relative actions, normalization) produced a statistically detectable improvement on their own when trained on the full dataset. The jump only comes from switching to curated high-quality data in Series 2. Toggle to Level 2 and the separation is even starker: every Series 1 experiment sits in group 'b' with 0% success, while 2.5 stands alone in group 'a'.
98
-
99
- ---
100
-
101
- ### Key Findings
102
-
103
- #### 1. Data quality matters most
104
-
105
- This is the finding we're most confident in it held regardless of which confidence level or correction method we used. The best Series 1 result (1.7) achieves 40% total success rate. The best Series 2 result (2.5) achieves 90% using the *same architecture*. The pairwise tests cleanly separate these two groups, and no amount of algorithmic tuning within Series 1 came close to closing the gap.
106
-
107
- We hypothesise that the root cause is the difference in **multi-modality** between the high-quality and full dataset. The full dataset contains demonstrations with some inconsistent strategies: different grips, unfolding sequences, and timing, while the high-quality dataset enforces a more unified, consistent protocol.
108
-
109
- <Note variant="info" emoji="💡">
110
- Define the exact task protocol before collecting data. Speed is secondary to consistency and clarity of intent at every step.
111
- </Note>
112
-
113
- #### 2. Relative actions improve performance consistently
114
-
115
- Comparing π0.5 without relative actions (1.2: 20% total SR, 40% Level 1) to π0.5 with relative actions and quantile normalization (1.3: 35% total SR, 70% Level 1), and then to the full combination in 1.7 (40% total SR, 80% Level 1), shows that training with relative actions consistently improves performance. The trend is clear and shows up in every comparison we made.
116
-
117
- With only 20 rollouts, the exact gap between experiments is hard to pin down, but the improvement is consistent across every comparison. **Caveat:** π0.5 is likely pretrained with relative actions, so 1.3 and 1.7 fine-tune in a regime consistent with pretraining, while 1.2 fine-tunes against it.
118
-
119
- #### 3. RABC helps especially on long tasks like level 2
120
-
121
- RABC on high-quality data yields the two best results overall: experiments 2.2 and 2.5 clearly outperform every experiment that lacks it. The effect is strongest on **Level 2**, the longer and harder task — 2.2 reaches 50% Level 2 SR and 2.5 reaches 80%, while every experiment without RABC on clean data stays at 0%.
122
-
123
- #### 4. Fine-tuning from a strong checkpoint is the winning recipe
124
-
125
- The best results share the same recipe: fine-tune a Series 1 checkpoint on curated high-quality data with RABC and relative actions.
126
-
127
- | Experiment | Total SR | Level 1 SR | Level 2 SR | Recipe |
128
- |:---:|:---:|:---:|:---:|:---|
129
- | 2.5 | **90%** | **100%** | **80%** | 1.7 → HQ + RABC, 100k steps |
130
- | 2.2 | 75% | 100% | 50% | 1.3 → HQ + RABC, 100k steps |
131
- | 1.7 | 40% | 80% | 0% | All data, Relative Actions + RABC + QUANTILES |
132
-
133
- The jump from Series 1 to Series 2 is unambiguous in the statistical analysis — 2.5 and 2.2 clearly stand out from the Series 1 group. The Series 1 checkpoint already knows how to fold shirts in general, the high-quality data teaches the correct protocol, and RABC emphasizes the best demonstrations within an already clean dataset.
134
-
135
- Both 2.2 and 2.5 were trained for 100k steps. 2.2 fine-tunes from 1.3 while 2.5 fine-tunes from 1.7 (the stronger base). The difference (75% → 90%) likely reflects this stronger starting point. They don't separate from each other in the pairwise tests, suggesting the recipe itself (HQ + RABC + Relative Actions) is the key ingredient, with the base checkpoint providing an additional boost.
136
-
137
- #### 5. Level 2 requires everything to be right simultaneously
138
-
139
- Every Series 1 experiment achieves exactly **0% Level 2 success**. Level 2 only becomes tractable in Series 2, and only with RABC on high-quality data (2.2: 50% L2, 2.5: 80% L2). The 0% → 50–80% jump is as clean a signal as you'll find in a 20-rollout experiment. Level 2 is genuinely harder. It requires the policy to have seen consistent, high-quality demonstrations of the full task, because without a reliable starting state after unfolding, the subsequent folds can't succeed.
140
-
141
- #### 6. Speed and fold quality both track data quality
142
-
143
- Series 1 completes Level 1 in **78–122s**; Series 2 does it in **41–73s**. Fold quality (1–5 scale) hits a ceiling around 2.8 in Series 1, breaking past 3.0 only with high-quality data.
144
-
145
- | Experiment | Level 1 Time | Level 1 SR | Quality |
146
- |:---:|:---:|:---:|:---:|
147
- | 1.1 (π0, all data) | 121.5s | 80% | 2.70 |
148
- | 1.7 (best S1) | 99.5s | 80% | 2.30 |
149
- | 2.1 (HQ finetune)* | 57.6s | 70% | 2.80 |
150
- | 2.2 (HQ + RABC) | 43.2s | 100% | 3.30 |
151
- | 2.5 (best overall) | **40.8s** | **100%** | **4.10** |
152
-
153
- Policies trained on the full dataset learned hesitant motions; the high-quality dataset enforces deliberate and progress oriented actions. Faster completion isn't a separate goal from quality it's a consequence of a clear, unambiguous strategy.
154
-
155
- #### 8. What did not work
156
-
157
- **Mirroring augmentation** (2.3): only 5% total SR. Mirroring doubles multimodality, making convergence much harder even at 100k steps.
 
1
  import Note from "../../../components/Note.astro";
2
  import Sidenote from "../../../components/Sidenote.astro";
3
  import Wide from "../../../components/Wide.astro";
4
+ import Stack from "../../../components/Stack.astro";
5
  import Accordion from "../../../components/Accordion.astro";
6
  import HtmlEmbed from "../../../components/HtmlEmbed.astro";
7
 
8
+ import sarmEp300 from "../../assets/image/lerobot-data-collection_level2_final_quality3_ep300_progress.gif";
9
+ import sarmEp2500 from "../../assets/image/lerobot-data-collection_level12_rac_2_2026-02-08_1_ep2500_progress.gif";
10
+ import sarmEp2200 from "../../assets/image/lerobot-data-collection_level12_rac_2_2026-02-08_1_ep2200_progress.gif";
11
 
12
+ ## From Data to Deployment
13
+
14
+ With the data collected, hardware running, and evaluation protocol fixed, we began training. What follows is the chronological story of how we iterated from a barely-working baseline to a 90% success rate, and what we learned at each step.
15
+
16
+ ### First attempt: training on all data
17
+
18
+ We started with the simplest possible recipe: train π0 and π0.5 on the full dataset of 5,688 episodes for 200k steps each.
19
+
20
+ | Experiment | Model | Data | Steps | Key change |
21
+ |:---:|:---:|:---|:---:|:---|
22
+ | 1.1 | π0 | All data (5,688 episodes) | 200k | Baseline |
23
+ | 1.2 | π0.5 | All data (5,688 episodes) | 200k | Baseline |
24
+
25
+ The loss curves show both models converging, but convergence of loss for robotics doesn't necessarily mean good performance:
26
+
27
+ <Wide>
28
+ <HtmlEmbed
29
+ id="loss-curves"
30
+ src="folding/loss-curves.html"
31
+ title="Training Loss"
32
+ desc="Click legend entries to toggle runs. <strong>Series 1</strong>: trained from pretrained checkpoints on full data (200k steps). <strong>Series 2</strong>: fine-tuned on curated high-quality data (100k steps)."
33
+ frameless
34
+ />
35
+ </Wide>
36
+
37
+ When we evaluated, the results were disappointing. Experiment 1.1 (π0) achieved 40% Level 1 success but 0% Level 2. Experiment 1.2 (π0.5) managed only 20% total success rate, also with 0% Level 2. The models could sometimes fold a laid-out shirt, but they were slow, and produced poor-quality folds. Level 2, spreading a messy shirt before folding was completely out of reach.
38
+
39
+ Watching the rollouts, the problem was clear: the policy had learned the *average* of many different folding strategies in the dataset. Different operators used different grip points, different fold sequences, and different speeds. The model averaged over all of them, producing indecisive, hesitant motions.
40
+
41
+ One early lesson: **undertraining makes the policy shaky**. Make sure your model has converged before drawing conclusions about what works.
42
+
43
+ ---
44
+
45
+ ### Improving the data
46
+
47
+ Before adding algorithmic complexity, we addressed the data quality problem directly.
48
+
49
+ #### Filtering
50
+
51
+ We filtered episodes in two ways:
52
+
53
+ 1. **End-state image filtering** to discard episodes where the final frame doesn't show a properly folded shirt. If the end result isn't good, the demonstration isn't useful.
54
+ 2. **Length-based filtering** using the LeRobot data visualizer to remove outliers. Episodes that are suspiciously short tend to be low quality.
55
+
56
+ #### Stage-Aware Reward Modeling (SARM)
57
+
58
+ To go beyond binary keep/discard filtering, we trained a reward model: **[SARM](https://huggingface.co/docs/lerobot/sarm)** (Stage-Aware Reward Modeling). SARM is a learned "critic" that scores trajectories based on how well the robot is progressing toward task completion. It predicts a 0–1 progression signal at every timestep, correctly identifying **mistakes** (drops in value) and **progress** (increases) in real time:
59
+
60
+ <Wide>
61
+ <Stack layout="3-column" gap="small">
62
+ <img src={sarmEp300.src} alt="SARM annotation on episode 300" style="width:100%; border-radius: 8px;" />
63
+ <img src={sarmEp2500.src} alt="SARM annotation on episode 2500" style="width:100%; border-radius: 8px;" />
64
+ <img src={sarmEp2200.src} alt="SARM annotation on episode 2200" style="width:100%; border-radius: 8px;" />
65
+ </Stack>
66
+ </Wide>
67
+
68
+ We annotated every episode in both datasets using SARM. This gave us continuous per-timestep quality scores that we could use in two ways: for **data curation** (scoring and ranking episodes) and for **reward-weighted training** (which we introduce next).
69
+
70
+ ---
71
+
72
+ ### Algorithmic improvements
73
+
74
+ With cleaner data and SARM scores in hand, we systematically tested three algorithmic improvements.
75
+
76
+ #### Relative actions
77
+
78
+ Following the [UMI](https://arxiv.org/abs/2402.10329) approach, we switched from absolute actions to **relative trajectories**: each action in the chunk is an offset from the robot's current state at prediction time, not from the previous action. The state observation remains absolute — only the action targets are relative. This avoids error accumulation (unlike true delta) and doesn't require a global coordinate frame (unlike absolute).
79
+
80
+ <HtmlEmbed
81
+ id="action-representations"
82
+ src="folding/action-representations.html"
83
+ title="Action Representations"
84
+ desc="Relative trajectory (blue) references all actions to the current state. Delta (yellow) chains each action to the previous one, accumulating error. Absolute (red) requires a global coordinate frame. Diagram adapted from UMI (Chi et al., 2024)."
85
+ />
86
+
87
+ See the [LeRobot action representations docs](https://huggingface.co/docs/lerobot/action_representations) for a full guide.
88
+
89
+ The result: experiment 1.3 (π0.5 + relative actions + quantile normalization) jumped to **35% total success rate and 70% Level 1**, up from 1.2's 20% total and 40% Level 1. The improvement was consistent across every comparison we made.
90
+
91
+ <Sidenote>
92
+ π0.5 is likely pretrained with relative actions, so 1.3 fine-tunes in a regime consistent with pretraining, while 1.2 fine-tunes against it.
93
+ </Sidenote>
94
+
95
+ #### RABC (Reward-Advantage-Based Conditioning)
96
+
97
+ Next, we put SARM's per-timestep scores to work during training. With **RABC**, every episode is weighted by its contribution to progress: high-reward actions contribute more to the loss, low-reward ones contribute less. Negative progress is clipped to 0. Unlike binary success/fail labels, this provides a continuous signal at every timestep.
98
+
99
+ Experiments 1.4 and 1.5 tested RABC with different κ values (0.01 and 0.0215). Both showed improvement, with 1.5's higher κ performing slightly better.
100
+
101
+ #### Combining everything
102
+
103
+ Experiment 1.7 combined all three improvements: **relative actions + RABC + quantile normalization**. The result: **40% total success rate, 80% Level 1** — the best of Series 1.
104
+
105
+ | Experiment | Key change | Total SR | Level 1 SR | Level 2 SR |
106
+ |:---:|:---|:---:|:---:|:---:|
107
+ | 1.1 | π0 baseline | 20% | 40% | 0% |
108
+ | 1.2 | π0.5 baseline | 20% | 40% | 0% |
109
+ | 1.3 | + Relative actions | 35% | 70% | 0% |
110
+ | 1.4 | + RABC (κ=0.01) | 25% | 50% | 0% |
111
+ | 1.5 | + RABC (κ=0.0215) | 30% | 60% | 0% |
112
+ | 1.7 | Relative + RABC + QUANTILES | **40%** | **80%** | **0%** |
113
+
114
+ The trend was encouraging, but one number was impossible to ignore: **every Series 1 experiment achieved exactly 0% Level 2 success**. Algorithmic improvements alone couldn't compensate for the inconsistency in the full dataset.
115
+
116
+ #### DAgger: targeted corrections
117
+
118
+ We also explored **DAgger** (Dataset Aggregation) as a way to fix specific failure modes. The idea: the operator watches the policy run, pauses when it fails, teleoperates a recovery, and hands control back. This generates correction data precisely where the model struggles.
119
+
120
+ DAgger felt promising — corrections targeting the model's actual failure modes should be highly effective at pushing success rates higher. The full [Human-in-the-Loop infrastructure](https://github.com/huggingface/lerobot/tree/main/examples/hil) is now merged into LeRobot for anyone to use.
121
+
122
+ ---
123
+
124
+ ### The breakthrough: high-quality data
125
+
126
+ No Series 1 experiment cracked Level 2. We needed a fundamentally better dataset.
127
+
128
+ We went back to data collection with a clear plan: collect new demonstrations specifically for Level 1 and Level 2, with a **unified strategy** across operators. Every episode followed the same grip points, the same fold sequence, the same timing. We also manually curated the best episodes from the large dataset, selecting only those with consistent technique and clear progress.
129
+
130
+ The result was the **high-quality dataset**: 1,200 episodes combining the best of the original data with new, strategically collected demonstrations. Less data, but dramatically better data.
131
+
132
+ **Series 2** fine-tuned the best Series 1 checkpoints on this curated dataset for 100k steps:
133
+
134
+ | Experiment | Base | Key change | Total SR | Level 1 SR | Level 2 SR |
135
+ |:---:|:---:|:---|:---:|:---:|:---:|
136
+ | 2.1 | 1.3 | HQ data only | 35% | 70% | 0% |
137
+ | 2.2 | 1.3 | HQ + RABC + Relative | **75%** | **100%** | **50%** |
138
+ | 2.5 | 1.7 | HQ + RABC + Relative | **90%** | **100%** | **80%** |
139
+
140
+ The jump was dramatic. Experiment 2.5 reached **90% total success rate** — 100% Level 1, 80% Level 2 — up from 40% in Series 1. The same architecture, the same training recipe, just better data.
141
+
142
+ Both 2.2 and 2.5 used the same recipe (HQ + RABC + Relative Actions), but 2.5 fine-tuned from 1.7 (the stronger base with relative actions + RABC already baked in) while 2.2 fine-tuned from 1.3. The difference (75% → 90%) likely reflects this stronger starting point. Data quality was the single biggest lever, and RABC's effect was strongest on **Level 2** — the longer, harder task where emphasizing the best demonstrations mattered most.
143
+
144
+ ---
145
+
146
+ ### What didn't work
147
+
148
+ Not every idea improved results:
149
+
150
+ - **Mirroring augmentation** (experiment 2.3): only 5% total success rate. Mirroring the data doubles the number of valid strategies the model sees, which is the opposite of what we needed — convergence on a single consistent approach.
151
+ - **Larger chunk size** (experiment 2.4, chunk=45): no improvement over the default chunk size of 30.
152
+
153
+ ---
154
+
155
+ ### Full comparison
156
+
157
+ Now that the full journey is clear, here is the complete picture across all 11 experiments.
158
 
159
  <Wide>
160
 
 
174
 
175
  </Wide>
176
 
 
 
 
 
 
 
 
 
 
 
177
  <HtmlEmbed
178
  id="success-rates"
179
  src="folding/success-rates.html"
 
181
  desc="Success rates (Total, Level 1, Level 2) across all experiments. Series 1 trains from scratch on full data; Series 2 finetunes the best Series 1 checkpoint on curated high-quality data."
182
  />
183
 
 
 
184
  <HtmlEmbed
185
  id="total-score"
186
  src="folding/total-score.html"
 
188
  desc="Overall score (% of maximum 1500) per experiment. The dashed line marks 50% as a reference point."
189
  />
190
 
191
+ Total score captures partial progress that binary success rate misses. Even failed rollouts earn credit for completed subtasks, revealing that some Series 1 experiments make meaningful progress despite 0% Level 2 success.
192
 
193
  <HtmlEmbed
194
  id="l1-time-quality"
 
197
  desc="Each bubble is one experiment. X-axis = completion time (faster is left), Y-axis = fold quality (higher is better), bubble size = total success rate. The best experiments cluster in the top-left corner."
198
  />
199
 
200
+ Speed and quality correlate strongly with data quality. Series 2 experiments fold 2–3x faster than Series 1 (40s vs 100s+), and fold quality only breaks past 3.0 with high-quality training data. Faster isn't a separate goal from better — it's a consequence of the policy learning a clear, unambiguous strategy.
201
 
202
  <HtmlEmbed
203
  id="subtask-heatmap"
 
206
  desc="Average time (seconds) per subtask across all experiments. Green is fast, red is slow. Dashes indicate the subtask was never completed."
207
  />
208
 
 
 
209
  ### Where the policies fail
210
 
211
+ Understanding *how* each experiment fails — not just whether it fails — is essential for knowing what to try next.
212
 
213
  <HtmlEmbed
214
  id="failure-analysis"
 
229
  />
230
 
231
  The CLD tells a clear story: **Series 2 separates from Series 1**. Experiments 2.5 and 2.2 form group 'a' — statistically distinct from the bulk of Series 1 experiments which cluster in groups 'c' and 'd'. Within Series 1, no experiment separates from another, meaning none of the algorithmic changes (RABC, relative actions, normalization) produced a statistically detectable improvement on their own when trained on the full dataset. The jump only comes from switching to curated high-quality data in Series 2. Toggle to Level 2 and the separation is even starker: every Series 1 experiment sits in group 'b' with 0% success, while 2.5 stands alone in group 'a'.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
app/src/content/chapters/folding/09-learnings.mdx CHANGED
@@ -1,5 +1,4 @@
1
  import Note from "../../../components/Note.astro";
2
- import Sidenote from "../../../components/Sidenote.astro";
3
 
4
  ## Learnings
5
 
@@ -12,7 +11,6 @@ Beyond the experiment findings above, several practical insights stood out:
12
  - **Train a reward model.** [SARM](https://huggingface.co/docs/lerobot/sarm) gave us data scoring, advantage conditioning, and curation in one package. We recommend it even for tasks where you think manual filtering would suffice.
13
  - **Invest in recording quality early.** More time upfront on clean and consistent recordings pays off more than extra volume.
14
  - **Record at higher frequency.** We'd record at 50 fps if we had to do it again. Folding is dynamic and higher record rates capture transitions better.
15
- - **DAgger is promising.** Corrections targeting the model's actual failure modes should be highly effective at pushing success rates higher. This infrastructure is ready and now also merged into LeRobot.
16
 
17
  ### For the community: the order of operations
18
 
@@ -50,17 +48,7 @@ lerobot-train \
50
  --policy.use_relative_actions=true
51
  ```
52
 
53
- 7. **Use DAgger for targeted improvement.** Once you have a model that mostly works, collect correction data for its specific failure modes. LeRobot's [HIL scripts](https://github.com/huggingface/lerobot/tree/main/examples/hil) handle the full loop, the operator watches the policy run, pauses on failure, teleoperates a recovery, and hands control back:
54
-
55
- ```bash
56
- python examples/hil/hil_data_collection.py \
57
- --robot.type=bi_openarm_follower \
58
- --teleop.type=openarm_mini \
59
- --policy.path=outputs/pretrain/checkpoints/last/pretrained_model \
60
- --dataset.repo_id=your-username/hil-dataset \
61
- --rtc.enabled=true \
62
- --rtc.execution_horizon=20
63
- ```
64
 
65
  8. **Film every evaluation.** Metrics alone won't tell the full story. Video reveals subtle failure modes that success rate misses, and lets you score quality.
66
 
 
1
  import Note from "../../../components/Note.astro";
 
2
 
3
  ## Learnings
4
 
 
11
  - **Train a reward model.** [SARM](https://huggingface.co/docs/lerobot/sarm) gave us data scoring, advantage conditioning, and curation in one package. We recommend it even for tasks where you think manual filtering would suffice.
12
  - **Invest in recording quality early.** More time upfront on clean and consistent recordings pays off more than extra volume.
13
  - **Record at higher frequency.** We'd record at 50 fps if we had to do it again. Folding is dynamic and higher record rates capture transitions better.
 
14
 
15
  ### For the community: the order of operations
16
 
 
48
  --policy.use_relative_actions=true
49
  ```
50
 
51
+ 7. **Use [DAgger](https://github.com/huggingface/lerobot/tree/main/examples/hil) for targeted improvement.** Once you have a model that mostly works, collect correction data for its specific failure modes using LeRobot's Human-in-the-Loop scripts.
 
 
 
 
 
 
 
 
 
 
52
 
53
  8. **Film every evaluation.** Metrics alone won't tell the full story. Video reveals subtle failure modes that success rate misses, and lets you score quality.
54