cupstack_bspline_act_v2

An ACT policy that predicts B-spline trajectory segments instead of discrete action chunks, trained on SO-101 cup stacking. This is the "Reg.+BSP" variant from B-spline Policy: Accelerating Manipulation Policies via B-spline Action Representations.

The practical consequence: you can run this checkpoint faster without retraining it. The policy outputs a curve, so executing a(n·t) replays the same trajectory geometry n times faster. Speed-up is an inference flag, and it is exact — verified to 0.000e+00 deviation at 2× and 4×.

This supersedes cupstack_bspline_act. Two things changed: idle frames were trimmed from the demonstrations, and 5 episodes were held out for validation, so for the first time there are held-out numbers for this task.

Read the Evaluation section before trusting any number here. Held-out error is ~19°, against 2.1° on training data. The policy is clearly reading the scene — it beats the inter-demonstration spread by 42% — but it has never been run on hardware, and task success rate is unknown. Keep the e-stop within reach.

Download

hf download Dimios45/cupstack_bspline_act_v2 --local-dir ./cupstack_bspline_act_v2

The repository root holds the recommended checkpoint, so --policy.path also accepts the repo id directly. Individual checkpoints live under checkpoints/<step>/.

Run it

lerobot-rollout \
    --strategy.type=base \
    --policy.path=./cupstack_bspline_act_v2/checkpoints/040000/pretrained_model \
    --policy.speed_up=1.0 \
    --device=cuda \
    --robot.type=so101_follower \
    --robot.port=/dev/ttyACM0 --robot.id=FOLLOWER \
    --robot.cameras='{front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, gripper: {type: opencv, index_or_path: 4, width: 640, height: 480, fps: 30}}' \
    --task="Cup stacking" \
    --fps=30 --duration=60

--policy.path takes either a Hub repo id (namespace/name) or a local directory containing config.json — which is the pretrained_model/ subdirectory, not the step directory above it. There is no --policy.subfolder flag.

To try the speed-up, raise --policy.speed_up to 2.0 or 4.0. Start at 1.0 and step up only once the task succeeds. Retiming does not make the arm stronger — past the low-level controller's tracking limit it overshoots rather than stopping. The paper reaches 4× on cube picking but only 2× on speed stacking, and cup stacking is the same precise-placement regime.

If the control loop can't keep up, drop --display_data (Rerun can saturate its channel at two 640×480 streams) rather than assuming the policy is too slow — a B-spline policy runs the network only when a segment is exhausted, roughly every 0.76 s, so most control ticks are just spline evaluation.

Camera naming is not optional

The --robot.cameras dict keys must be front and gripper, lowercase:

Key	Shape
observation.images.front	(3, 480, 640)
observation.images.gripper	(3, 480, 640)
observation.state	(6,)

Joint order is shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper, in degrees. Swapping the two cameras produces confident but wrong behaviour rather than an error.

Requirements

This policy type is not in upstream LeRobot. You need a checkout containing the bspline_act policy, installed with the scipy extra:

uv pip install -e ".[feetech,bspline]"

scipy>=1.15 is required for interpolate.generate_knots.

Unnormalization statistics are stored as buffers inside model.safetensors, so each checkpoint is self-contained — you do not need the training dataset on the robot machine.

Checkpoints

Step	Epoch	Train error	Held-out error	Eval loss
10000	17	6.56°	19.20°	0.3074
20000	33	4.95°	19.36°	0.3072
30000	50	3.89°	19.33°	0.3060
40000	67	2.63°	19.06°	0.3084	recommended
45000	75	2.49°	18.72°	0.3005	best, within noise of 40k
50000	83	2.28°	19.05°	0.3088
60000	100	2.08°	19.22°	0.3121	final

All twelve checkpoints of the run span 18.72–19.36° held-out (mean 19.03, sd 0.22), so every checkpoint here performs the same on unseen data. 45000 is nominally best and is included for that reason, but its margin over 40000 is a quarter of the run's own noise. Pick 40000 unless you have a reason not to.

Error is mean absolute joint error in degrees, worst joint, over each predicted segment's horizon (~0.76 s), measured by decoding the spline and comparing against ground-truth actions.

Training

Steps	60,000
Batch size	32
Epochs	100
Dataset	aryankakad/CUPSTACKING → idle-trimmed → B-spline converted
Episodes	45 train / 5 held out
Frames	20,956 (from 24,044; 12.9% boundary idle removed)
Optimizer	AdamW, lr 2e-5 (backbone 1e-5), wd 1e-4
Precision	bf16 autocast + channels_last
Hardware	1× RTX 4090, 3 h 16 min
Seed	1000
Augmentation	none

Preprocessing

Leading and trailing motionless frames were removed from every episode — 77 s of lead-in and 26 s of trailing idle across 50 episodes. Interior pauses were kept (only 0.3% of frames, and they are real behaviour).

Motion is measured as peak-to-peak joint range over a ±0.5 s window rather than frame-to-frame delta. A single jittery sample inside a dead stretch defeats a delta test, which silently leaves the idle in.

Leading idle matters more than it looks: it maps the opening observation to "don't move", and since executing a stay-still segment doesn't change the scene, the policy can re-predict it indefinitely and never start.

Architecture

Standard ACT with one change: the decoder emits a B-spline parameter matrix rather than an action chunk.

(n_knots, 1 + action_dim) = (16, 7) = 112 values per prediction
   column 0   knot vector, in source-frame units, 0 = "now"
   columns 1: control points, one per joint

Params	51.6 M
Vision backbone	ResNet18 (ImageNet init, fine-tuned)
dim_model / chunk_size	512 / 16
VAE	enabled, kl_weight 10.0
B-spline degree	3 (cubic, C² continuous)
bspline_chunk_size	10
Fitting tolerance ε	0.2 (degrees)
Max knot span	6 frames

Knot spacing is fitted adaptively, so a fixed 16 rows covers a variable time horizon — 0.50 s to 1.03 s per segment, 0.76 s mean. The network runs only when a segment is exhausted, which decouples policy rate from control rate.

On the fitting tolerance

ε = 0.2 is not the paper's value. The paper uses 0.002 for metre-scale end-effector actions; SO-101 stores joint targets in degrees (~±120), roughly 100× larger. At ε = 0.002 compression is 1.06× — one knot per frame, which defeats the representation. At 0.2 the fit compresses 2.41× and reconstructs to 0.187° p99, well below servo resolution.

max_knot_span caps how wide a single knot interval may be, bounding open-loop exposure at chunk_size × 6 frames. On this dataset it is nearly a no-op (uncapped p95 span is already 1.17 s); it matters on recordings with long pauses, where uncapped fitting produced 18-second segments.

Evaluation

Offline only. No hardware evaluation has been performed.

	Train (45 eps)	Held-out (5 eps)
Best checkpoint	2.49°	18.72°
Recommended (40k)	2.63°	19.06°

Two reference points for the held-out number:

• Inter-demonstration spread: 32.27° — how far apart two different human demonstrations of this task are, phase-aligned. At 18.7° the policy predicts an unseen episode's trajectory substantially better than another demonstration would. It is reading the scene, not replaying an average.
• Hold-still baseline: 17.59° — emitting the current action for the next 0.76 s. The policy scores near this, but the comparison flatters a do-nothing policy: over a short horizon on a slow task, "don't move" is a decent predictor while being a useless policy. Do not read this as "no better than nothing".

Held-out performance was fixed after step ~3,000. Across twelve checkpoints and 55,000 steps it moved 0.6°, while training error fell from 8.00° to 2.08°. Everything after the first few thousand steps went into fitting the training episodes harder.

The predecessor model card reported "2.10° open-loop error at 100k" without a validation split. This run reproduces that figure (2.08° at 60k) and shows the held-out number underneath it is ~19°. That was a training-data measurement, not a generalization result.

Temporal rescaling is exact: on a real predicted segment, a(2t) and a(4t) reproduce the 1× samples to 0.000e+00, consuming the segment in 24 / 12 / 6 control ticks.

Limitations

• No hardware evaluation. Task success rate is unknown.
• Held-out split is the last 5 episodes, not a random sample. Episodes were largely cut from a continuous teleoperation stream (40 of 49 boundaries are contiguous to under 1°), so adjacent episodes are correlated and an interleaved split would leak. The tail split avoids leakage at the cost of possible session drift.
• Single task, single scene. No robustness to lighting, camera placement, or cup position changes should be assumed.
• Speed-up has a hardware ceiling. Expect degradation near 2× on this task.
• Camera assignment is silent when wrong. Swapped views degrade behaviour without raising an error.

Citation

@article{han2026b,
  title={B-spline Policy: Accelerating Manipulation Policies via B-spline Action Representations},
  author={Han, Xiaoshen and Xiong, Haoyu and Chen, Haonan and Liu, Chaoqi and
          Torralba, Antonio and Zhu, Yuke and Du, Yilun},
  journal={arXiv preprint arXiv:2607.09648},
  year={2026}
}