Instructions to use nhatcm/hitbench-so101-smolvla-eef-aug with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use nhatcm/hitbench-so101-smolvla-eef-aug with LeRobot:
# See https://github.com/huggingface/lerobot?tab=readme-ov-file#installation for more details git clone https://github.com/huggingface/lerobot.git cd lerobot pip install -e .[smolvla]
# Launch finetuning on your dataset python lerobot/scripts/train.py \ --policy.path=nhatcm/hitbench-so101-smolvla-eef-aug \ --dataset.repo_id=lerobot/svla_so101_pickplace \ --batch_size=64 \ --steps=20000 \ --output_dir=outputs/train/my_smolvla \ --job_name=my_smolvla_training \ --policy.device=cuda \ --wandb.enable=true
# Run the policy using the record function python -m lerobot.record \ --robot.type=so101_follower \ --robot.port=/dev/ttyACM0 \ # <- Use your port --robot.id=my_blue_follower_arm \ # <- Use your robot id --robot.cameras="{ front: {type: opencv, index_or_path: 8, width: 640, height: 480, fps: 30}}" \ # <- Use your cameras --dataset.single_task="Grasp a lego block and put it in the bin." \ # <- Use the same task description you used in your dataset recording --dataset.repo_id=HF_USER/dataset_name \ # <- This will be the dataset name on HF Hub --dataset.episode_time_s=50 \ --dataset.num_episodes=10 \ --policy.path=nhatcm/hitbench-so101-smolvla-eef-aug - Notebooks
- Google Colab
- Kaggle
HITBench SO-101 SmolVLA EEF Aug
SmolVLA fine-tuned from lerobot/smolvla_base on the front-view HITBench
SO-101 interaction dataset. The root model is the final 30,000-step checkpoint.
Checkpoints
| Training step | Hugging Face path |
|---|---|
| 15,000 | checkpoints/015000 |
| 20,000 | checkpoints/020000 |
| 25,000 | checkpoints/025000 |
| 30,000 | repository root (also recorded as checkpoints/030000.json) |
Every loadable checkpoint directory contains config.json,
model.safetensors, train_config.json, and action_contract.json.
Input/output contract
- Image:
observation.images.rgb, RGB(3, 256, 256), front view. - State:
observation.state, six measured SO-101 joints. - Language: the trajectory's causal task/interaction prompt.
- Output: 50 actions, each a normalized 7D EEF command
(dx, dy, dz, dRx, dRy, dRz, gripper). - Action normalization: identity. Decode with
action_contract.json. - State normalization: dataset mean/std stored in the policy artifacts.
Training
- Dataset:
nhatcm/hitbench-so101-eef-interactions. - 477 source trajectories, 1,565 causal segments, 349,611 frames.
- Eight GPUs, batch size 7 per rank, global batch size 56.
- 30,000 optimizer steps; checkpoints every 5,000 steps.
- 80% uniformly sampled frame anchors across the complete dataset.
- 20% balanced anchors around H2R release and R2H grasp boundaries.
- The action-padding mask excludes invalid tail actions from the loss.
Visual augmentation
Semantic wall masks protect the robot, person, table, tray, and task objects. Wall appearance is replaced with probability 0.85: 75% of replacements use the ten curated wall backgrounds and 25% use procedural color/texture. Training also applies strong brightness, contrast, saturation, hue, sharpness, and sensor noise augmentation. It does not use dropout masks, crops, or geometric warps.
The exact reproducibility records are included at the repository root:
sampling_contract.json, wall_augmentation_contract.json, and
strong_visual_augmentation.json.
Status
The final 30k training loss was approximately 0.003-0.006. This is a training metric, not a physical task-success measurement; hardware evaluation is still required before deployment.
- Downloads last month
- 5