BEHAVIOR Challenge 2026 β€” task 12 (preparing_lunch_box), fine-tune ft1, step 9999

Selected checkpoint for the BEHAVIOR-1K 2026 submission.

Model

  • Architecture: the 2025 BEHAVIOR Challenge winner's pi0.5-based policy (IliaLarchenko/behavior-1k-solution, ~3B params, task-ID embeddings, no language prompt).
  • Init: the winner's specialist checkpoint_2, which already covered task 12 (2025 q 0.517 public / 0.367 hidden; 50 task slots, so its task-12 embedding is kept).
  • Training config: pi_behavior_b1k_2026_task12 (in the team pipeline repo's behavior-1k-solution/src/b1k/training/config.py).
  • Data: 2026 challenge demos, task 12 episodes 2400-2579 (last 20 held out), RGB cameras only.
  • Recipe: 10,000 steps (final step is 9999), batch 16, peak lr 5e-5, 1x A100 80GB.
  • Checkpoint is params-only (no optimizer state): it can be served or used as an init, not --resumed.

Layout

  • params/ β€” orbax/ocdbt parameters
  • assets/b1k_2026_task12/norm_stats.json β€” normalization stats
  • _CHECKPOINT_METADATA

Serve with scripts/serve_b1k.py --task-id 12 policy:checkpoint --policy.config pi_behavior_b1k_2026_task12 --policy.dir <this dir>.

Evaluation (local; q = fraction of goal predicates newly satisfied, 1.0 on success)

Scored the 2026 leaderboard way: public instances 0-9 (ids 301-310), one rollout each, default 1.5x timeouts, task score = sum / 10.

Task 12 (6 goal predicates: apple halves, sandwich, cookie and tea bottle into the lunch box; fridge closed)

Leaderboard score: q = 0.3667 (sum 3.67 over 301-310; no full completions)

instance 301 302 303 304 305 306 307 308 309 310
q .333 0 .667 .5 .667 .667 .333 0 .5 0

Partial credit on 7 of 10 instances; the fridge-closed predicate is already true at reset, so it earns nothing.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading