BEHAVIOR Challenge 2026, task 57 sweeping_garage: fine-tuned 2025 winner (expert, step 7499)

This is our submitted policy for task 57 sweeping_garage (R1Pro robot, OmniGibson / Isaac Sim 5.1). It is the 2025 BEHAVIOR Challenge 1st-place model, fine-tuned on the 2026 sweeping_garage demos.

Summary

Base checkpoint IliaLarchenko/behavior_submission, checkpoint_3 (HF revision 89545bc1b7aa7f2e687bc0032d091f132d715d4e)
Code IliaLarchenko/behavior-1k-solution @ ca556f7, plus our changes (see below)
Data all 200 sweeping_garage demos (episodes 0–199, 894,606 frames at 30 fps) from behavior-1k/2026-challenge-demos (LeRobot v3)
Cameras / inputs head + left/right wrist RGB, proprioception. Depth is not used.
Total fine-tuning 27,500 steps in 3 stages, batch size 32
Hardware 1Γ— RTX 5090, about 3.4 s/step
This checkpoint pi_behavior_sweep_expert/sweeping_garage_expert/7499

Training stages

Every stage uses batch 32, a cosine LR schedule with warmup, no EMA, and the base checkpoint's own norm stats and FAST tokenizer. Each stage starts from the previous stage's final params. The SigLIP vision tower and the PaliGemma (Gemma-2B) backbone stay frozen in every stage.

# Config name Steps Warmup Peak LR β†’ final LR Trainable Final train loss
1 pi_behavior_sweep_lora 12,000 500 5e-5 β†’ 5e-6 LoRA on both Gemma towers + small heads ~0.14
2 pi_behavior_sweep_lora_cont 8,000 200 2e-5 β†’ 2e-6 same as stage 1 ~0.12
3 pi_behavior_sweep_expert (this model) 7,500 300 2e-5 β†’ 2e-6 stage 2 + the full Gemma-300M action expert (~531M params) ~0.11

"Small heads" means the task/stage embeddings, fusion, kv_transform, and action/time projections. Model settings come from the 2025 solution: action horizon 30, action dim 32, correlated flow noise (Ξ² 0.5), FAST auxiliary loss (weight 0.05), subtask loss (weight 0.1), and per-timestamp normalization.

Changes from the 2025 code

  • It reads the 2026 LeRobot v3 data and uses the 2026 61-dim proprio layout (same field names as 2025).
  • The 2025 model has no slot for task 57, so task 57 is trained and served in model slot 36 (clean_a_patio). This keeps every parameter shape identical to the checkpoint.
  • base_qvel in the state is zeroed. It was about 0 in the 2025 data, but in 2026 it holds the real velocity, which falls far outside the model's training distribution.
  • The FAST tokenizer was re-implemented with tokenizers for transformers 5 (it gives identical ids). Weights are restored with openpi's restore_params (newer orbax requires this).

Exact commands

The code is src/ and scripts/ from the training repo (also included as policy_code/ in our submission zip). All three configs are in src/b1k/training/config.py. Each run was launched like this, with the openpi venv:

cd b1k_winner_ft
# stage 1
PYTHONPATH=src XLA_PYTHON_CLIENT_MEM_FRACTION=0.92 python scripts/train.py \
    pi_behavior_sweep_lora --exp-name sweeping_garage_from_winner_ckpt3 --overwrite
# stage 2
PYTHONPATH=src XLA_PYTHON_CLIENT_MEM_FRACTION=0.92 python scripts/train.py \
    pi_behavior_sweep_lora_cont --exp-name sweeping_garage_cont8k --overwrite
# stage 3 (this checkpoint)
PYTHONPATH=src XLA_PYTHON_CLIENT_MEM_FRACTION=0.92 python scripts/train.py \
    pi_behavior_sweep_expert --exp-name sweeping_garage_expert --overwrite

No run crashed or was resumed. The configs contain absolute pod paths for the base checkpoint (_WINNER_CKPT), the data root, and each stage's init params. Edit those paths before reusing them.

Serving

PYTHONPATH=src python scripts/serve_b1k_2026.py \
    --policy-dir <this repo> --config pi_behavior_sweep_expert --port 8000

The server maps evaluator task id 57 to model slot 36. The 2025 per-task "eval tricks" are off, because slot 36's rules would force both grippers open. A torso cap, on by default, limits trunk joint commands to the range seen in the demos (--no-torso-cap disables it). Without the cap, the model often tilts the torso back and looks at the ceiling.

Evaluation

We followed the official protocol: public instances 0–9 (scene instances 301–310), 1 rollout each, the default timeout (6709 steps), RGBDFullResWrapper, and the torso cap on.

Run Success Mean q-score Mean dirt left
Official run (submitted) 1/10 (instance 310, 5552 steps) 0.10 0.73
Repeat run, identical settings 0/10 0.00 0.86

The simulator is not deterministic from run to run, so treat the 1/10 as fragile. "Dirt left" is the fraction of the ~25 dirt particles still on the floor at the end. It comes from our logging hook and doesn't affect the score.

Files

  • params/: orbax inference weights
  • assets/: norm stats used at inference time
  • _CHECKPOINT_METADATA

The optimizer state (train_state/) is not included.

License

The base model is released under Apache-2.0. It is derived from Ο€0.5 / PaliGemma, so the Gemma Terms of Use also apply.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading

Model tree for Luke-Xing/task57b1k

Finetuned
(5)
this model

Dataset used to train Luke-Xing/task57b1k