HITBench SO-101 SmolVLA EEF Aug

SmolVLA fine-tuned from lerobot/smolvla_base on the front-view HITBench SO-101 interaction dataset. The root model is the final 30,000-step checkpoint.

Checkpoints

Training step Hugging Face path
15,000 checkpoints/015000
20,000 checkpoints/020000
25,000 checkpoints/025000
30,000 repository root (also recorded as checkpoints/030000.json)

Every loadable checkpoint directory contains config.json, model.safetensors, train_config.json, and action_contract.json.

Input/output contract

  • Image: observation.images.rgb, RGB (3, 256, 256), front view.
  • State: observation.state, six measured SO-101 joints.
  • Language: the trajectory's causal task/interaction prompt.
  • Output: 50 actions, each a normalized 7D EEF command (dx, dy, dz, dRx, dRy, dRz, gripper).
  • Action normalization: identity. Decode with action_contract.json.
  • State normalization: dataset mean/std stored in the policy artifacts.

Training

  • Dataset: nhatcm/hitbench-so101-eef-interactions.
  • 477 source trajectories, 1,565 causal segments, 349,611 frames.
  • Eight GPUs, batch size 7 per rank, global batch size 56.
  • 30,000 optimizer steps; checkpoints every 5,000 steps.
  • 80% uniformly sampled frame anchors across the complete dataset.
  • 20% balanced anchors around H2R release and R2H grasp boundaries.
  • The action-padding mask excludes invalid tail actions from the loss.

Visual augmentation

Semantic wall masks protect the robot, person, table, tray, and task objects. Wall appearance is replaced with probability 0.85: 75% of replacements use the ten curated wall backgrounds and 25% use procedural color/texture. Training also applies strong brightness, contrast, saturation, hue, sharpness, and sensor noise augmentation. It does not use dropout masks, crops, or geometric warps.

The exact reproducibility records are included at the repository root: sampling_contract.json, wall_augmentation_contract.json, and strong_visual_augmentation.json.

Status

The final 30k training loss was approximately 0.003-0.006. This is a training metric, not a physical task-success measurement; hardware evaluation is still required before deployment.

Downloads last month
5
Safetensors
Model size
0.5B params
Tensor type
F32
·
BF16
·
Video Preview
loading