Add TransportBeaker model provenance and safety draft card

bcb65e6 verified 1 day ago

4.02 kB

---
base_model: zjunlp/LabVLA-5B-Base
datasets:
- zjunlp/LabUtopia-Level3-TransportBeaker
tags:
- robotics
- vision-language-action
- labvla
- transportbeaker
- franka
- fine-tuned
---
# LabVLA-5B-LabUtopia-TransportBeaker

Task-specific **LabVLA-5B** fine-tuned model for the **LabUtopia Level3 TransportBeaker** robotic laboratory task.

**Draft release:** Private pending review of redistribution rights of the historical warm-start checkpoint and its contributing RoboInter and Open X-Embodiment datasets. No unverified model license is claimed.

## Model

- Architecture: Qwen3-VL vision-language backbone with Diffusion Transformer (DiT) flow-matching action expert.
- Task: TransportBeaker, single-arm Franka manipulation, three RGB cameras.
- Actual initialization: mixed RoboInter + Open X-Embodiment Knowledge Isolation checkpoint at 20,000 steps. This is **not** direct fine-tuning from the published LabVLA-5B-Base weights.
- Task fine-tuning: **40,000 optimizer steps**, effective global batch 192 (4 GPUs × per-GPU batch 48 × Gradient Accumulation 1), Learning Rate 5e-5, Warmup 2,000 steps.
- Trainable Vision Encoder, Language Model Gradient Checkpointing.
- Precision: bfloat16; Action Chunk Size: 50; inference denoising steps: 10.
- Raw action and state: 8 dimensions. The first seven arm dimensions are Delta Action; the last dimension is absolute gripper width.
- Output: self-contained inference-ready Safetensors checkpoint, not a fully resumable optimizer checkpoint.

The base_model metadata links the upstream LabVLA architecture and family; the actual mixed Knowledge Isolation initialization differs and is documented above.
## Evaluation
Full 120-episode runs in the official LabUtopia Level3 TransportBeaker simulation with the Franka asset:
| Material Split | Success | Success Rate |
| --- | ---: | ---: |
| Test Material (Out-of-Distribution) | 114 / 120 | **95.00%** |
| Training Material (In-Distribution) | 118 / 120 | **98.33%** |
These are simulation success rates, not physical-robot success rates.
## Files and Loading
Keep this exact structure:
- pretrained_model/model.safetensors — 997 tensors, approximately 9.44 GB
- pretrained_model/config.json and vlm_config.json
- pretrained_model/tokenizer.json, tokenizer_config.json, merges.txt and vocab.json
- pretrained_model/chat_template.json and preprocessor files
- labvla_schema.json — three-camera mapping and Action Schema
- norm_stats.json — original training-time normalization

Download with the Hugging Face CLI and use the repository root as the custom LabVLA inference server's pretrained_path. Use the associated pretrained_model directory for local tokenizer/processor assets. A generic Transformers AutoModel loader is not sufficient to load this LabVLA checkpoint.

The bundled norm_stats.json matches the associated TransportBeaker dataset's meta/stats_canonical_grip.json byte-for-byte (SHA256 9e9da9599949fe24ad7c241cc63af930962e02faa4cd02d8a4c8096b645a4230). Do not replace it with generic dataset statistics.
## Limitations and Licensing
Evaluated in simulation for this task only. The original LabUtopia data assets are CC BY-NC 4.0; source-code licensing is separate. The additional RoboInter/Open X-Embodiment initialization sources require a redistribution-license review before public release.
## References
- LabVLA code and deployment: https://github.com/zjunlp/LabVLA
- Upstream model: https://huggingface.co/zjunlp/LabVLA-5B-Base
- LabUtopia: https://github.com/Rui-li023/LabUtopia
- LabVLA paper: https://arxiv.org/abs/2606.13578
Please cite the original LabVLA and LabUtopia papers when using this checkpoint.
## Deployment Reproducibility
See [DEPLOYMENT.md](DEPLOYMENT.md) for the complete tested Policy Server command, Action Normalization settings, three-camera message protocol, and a real-observation inference smoke test. The checkpoint requires a compatible LabVLA loader supporting self-contained Qwen3-VL assets; an older model loader was found to be incompatible.

	---
	base_model: zjunlp/LabVLA-5B-Base
	datasets:
	- zjunlp/LabUtopia-Level3-TransportBeaker
	tags:
	- robotics
	- vision-language-action
	- labvla
	- transportbeaker
	- franka
	- fine-tuned
	---

	# LabVLA-5B-LabUtopia-TransportBeaker

	Task-specific LabVLA-5B fine-tuned model for the LabUtopia Level3 TransportBeaker robotic laboratory task.

	Draft release: Private pending review of redistribution rights of the historical warm-start checkpoint and its contributing RoboInter and Open X-Embodiment datasets. No unverified model license is claimed.

	## Model

	- Architecture: Qwen3-VL vision-language backbone with Diffusion Transformer (DiT) flow-matching action expert.
	- Task: TransportBeaker, single-arm Franka manipulation, three RGB cameras.
	- Actual initialization: mixed RoboInter + Open X-Embodiment Knowledge Isolation checkpoint at 20,000 steps. This is not direct fine-tuning from the published LabVLA-5B-Base weights.
	- Task fine-tuning: 40,000 optimizer steps, effective global batch 192 (4 GPUs × per-GPU batch 48 × Gradient Accumulation 1), Learning Rate 5e-5, Warmup 2,000 steps.
	- Trainable Vision Encoder, Language Model Gradient Checkpointing.
	- Precision: bfloat16; Action Chunk Size: 50; inference denoising steps: 10.
	- Raw action and state: 8 dimensions. The first seven arm dimensions are Delta Action; the last dimension is absolute gripper width.
	- Output: self-contained inference-ready Safetensors checkpoint, not a fully resumable optimizer checkpoint.

	The base_model metadata links the upstream LabVLA architecture and family; the actual mixed Knowledge Isolation initialization differs and is documented above.

	## Evaluation

	Full 120-episode runs in the official LabUtopia Level3 TransportBeaker simulation with the Franka asset:

	\| Material Split \| Success \| Success Rate \|
	\| --- \| ---: \| ---: \|
	\| Test Material (Out-of-Distribution) \| 114 / 120 \| 95.00% \|
	\| Training Material (In-Distribution) \| 118 / 120 \| 98.33% \|

	These are simulation success rates, not physical-robot success rates.

	## Files and Loading

	Keep this exact structure:

	- pretrained_model/model.safetensors — 997 tensors, approximately 9.44 GB
	- pretrained_model/config.json and vlm_config.json
	- pretrained_model/tokenizer.json, tokenizer_config.json, merges.txt and vocab.json
	- pretrained_model/chat_template.json and preprocessor files
	- labvla_schema.json — three-camera mapping and Action Schema
	- norm_stats.json — original training-time normalization

	Download with the Hugging Face CLI and use the repository root as the custom LabVLA inference server's pretrained_path. Use the associated pretrained_model directory for local tokenizer/processor assets. A generic Transformers AutoModel loader is not sufficient to load this LabVLA checkpoint.

	The bundled norm_stats.json matches the associated TransportBeaker dataset's meta/stats_canonical_grip.json byte-for-byte (SHA256 9e9da9599949fe24ad7c241cc63af930962e02faa4cd02d8a4c8096b645a4230). Do not replace it with generic dataset statistics.

	## Limitations and Licensing

	Evaluated in simulation for this task only. The original LabUtopia data assets are CC BY-NC 4.0; source-code licensing is separate. The additional RoboInter/Open X-Embodiment initialization sources require a redistribution-license review before public release.

	## References

	- LabVLA code and deployment: https://github.com/zjunlp/LabVLA
	- Upstream model: https://huggingface.co/zjunlp/LabVLA-5B-Base
	- LabUtopia: https://github.com/Rui-li023/LabUtopia
	- LabVLA paper: https://arxiv.org/abs/2606.13578

	Please cite the original LabVLA and LabUtopia papers when using this checkpoint.

	## Deployment Reproducibility

	See [DEPLOYMENT.md](DEPLOYMENT.md) for the complete tested Policy Server command, Action Normalization settings, three-camera message protocol, and a real-observation inference smoke test. The checkpoint requires a compatible LabVLA loader supporting self-contained Qwen3-VL assets; an older model loader was found to be incompatible.