SmolVLA_sim_sample_loading_visionft3500-lr1e6 (checkpoint 003500)

SmolVLA policy for the RoboSynChallenge simulation task sample_loading. The repository root is a complete LeRobot pretrained_model directory.

Files

File	Size (bytes)	sha256
config.json	2864	1f55ea3217697f4a9eeab7174bd4bbf784f1850135cbb2c75cdd41c8590940ad
model.safetensors	906712520	c10608e6e18d2b470b5df39b0d9283a75ba979a5af58743bc207c12ab7341811
train_config.json	7927	6982fe1cde630cba9a3816a189c0200e14ff4799addfd9679336229417b3076d
policy_preprocessor.json	2166	3d27479887406087c60e253f754ce3b2d82ef227b5421af7ea7107a29b80a958
policy_preprocessor_step_5_normalizer_processor.safetensors	13664	c9036facbb00e9e2ead3b3c2d96c63346f95400f3d00d121ea10fafaeb472786
policy_postprocessor.json	661	c2ec9f9760e19b11648f361a04e1bd7661f2cc015d2a7b9769d43209cfe90b61
policy_postprocessor_step_0_unnormalizer_processor.safetensors	13728	9ed9a6ba46bf0cc2cceb9dbd3faf26f1325190bed77ad6f99e0b67d526d38108

Model

• Architecture: SmolVLA (SmolVLM2-500M-Video-Instruct VLM + flow-matching action expert); 16 VLM layers, cross-attention expert (expert_width_multiplier 0.75), 10 flow-matching steps at inference
• Initialization: official released checkpoint RoboSynChallenge/SmolVLA_sim_sample_loading
• Training: full-parameter fine-tune of the vision encoder + action expert + action/state projections (the VLM text backbone stays frozen). Constant lr 1e-6, batch size 8, 3500 steps total (first run 0→2000, then resumed 2000→3500), seed 1000.
• Training patch note: the SigLIP vision tower was unfrozen during training via a local sitecustomize wrapper around SmolVLMWithExpertModel.set_requires_grad / train (the checkpoint config.json still records the stock freeze_vision_encoder: true flag). I.e. vision-encoder tensors differ from the official checkpoint (128 of 197 vision tensors updated by step 3500).
• Data: cobotmagic_Sim_sample_loading, LeRobot v3.0, 1000 episodes, 372000 frames, 25 fps. The data is not included.
• W&B run: https://wandb.ai/cyanrobosyn/robosynchallenge/runs/guxbrj5g
• Code: team stack smolvla-latest (policy/smolvla_latest/finetune.sh training path) with the vision-unfreeze patch above (training only)
• Environment: training Python 3.13.5, lerobot 0.6.1, torch 2.11.0, RTX 5090; evaluation Python 3.10.12, torch 2.7.0+cu128, RTX 5090

Input and output

• State inputs: observation.state, observation.qvel, observation.qf [14] each (14-dim CobotMagic qpos, grippers at index 6 and 13)
• Image inputs: observation.images.cam_high, observation.images.cam_left_wrist, observation.images.cam_right_wrist [3, 480, 640] (resized with padding to 512x512 internally)
• Action: action [14] absolute joint positions (CobotMagic)
• Normalization: STATE MEAN_STD, ACTION MEAN_STD, VISUAL IDENTITY
• chunk_size 50, n_action_steps 50, num_steps 10

Procedure: download and evaluate

1. hf download mair-robosyn/SmolVLA_sim_sample_loading_visionft3500-lr1e6 --repo-type model --revision 96c9a3680e61653987f859e0be2f1b89b38b0667 --local-dir checkpoints/SmolVLA_sim_sample_loading_visionft3500-lr1e6 (model files verified at this revision; later commits update this model card only)
2. bash policy/smolvla_latest/eval.sh sample_loading random checkpoints/SmolVLA_sim_sample_loading_visionft3500-lr1e6 0 --pytorch_device cuda --headless True

Results (team-internal, not organizer results)

• Protocol: setting random, 20 episodes, seed 240926, max_steps 500, standard policy/smolvla_latest/eval.sh, host robosyn5090
• Success rate: 3/20 = 15.0% (2026-10-11)
• Reference: official released SmolVLA_sim_sample_loading, same protocol: 1/10 = 10%. Official ACT: 0/10 and 0/30; official DP: 2/30 = 6.7%.
• Warning: this task shows large run-to-run variance under the standard eval (the policy sampling RNG is not seeded per episode; the same checkpoint re-scored 3/10 and 1/10 on identical scenes). Treat small success-rate differences as noise.

Status

• UPLOADED: yes, revision 96c9a3680e61653987f859e0be2f1b89b38b0667 (token-free re-download sha256 match for all 7 files, 2026-10-11)
• LOAD VERIFIED: yes, same revision (2026-10-11). Minimal CPU load + dummy batch: action (1, 14) torch.float32 finite True.
• EVAL VERIFIED (team-internal): yes, same revision, 5 episodes (seed 240926, max_steps 500): 0/5, 2026-10-11. The official policy/smolvla_latest/eval.sh ran on the re-downloaded directory and wrote evaluation_metrics.json. (Small-N screen; see the 20-episode reference above.)

Warning

These results come from our local runs. The organizers did not verify them.