SuNya558/lingbot-vla-80k

LingBot-VLA 2.0 — dual-arm, 80k

从原始 LingBot-VLA 2.0 基座进行全参数微调（包含视觉编码器）的双臂模型。 已完成 80,000 小步 / 5,000 次参数更新。本仓库包含推理权重、原始保存的处理器、训练日志包、历史阶段报告和完整1000局最终评测报告。

Training

• Base: lerobot/lingbot_vla_v2_base, converted from robbyant/lingbot-vla-v2-6b.
• 50 tasks, clean demonstrations only: 2,500 source episodes / 549,787 frames; 2,400 training episodes and 100 held out (last two episodes per task). Saved normalization statistics used all clean source frames, so the holdout is not statistically independent of normalization.
• Native three-camera RGB at 320×240, 50 fps; 14-dimensional state and next-frame absolute action: left six joints, left gripper, right six joints, right gripper. The saved policy image processor performs its original resizing and Qwen processing.
• Two ranks × batch 8 × accumulation 16 = effective batch 256. Full fine-tuning, BF16 model computation and saved weights, FP32 optimizer master weights; Muon matrices and AdamW other parameters. This is native explicit BF16, not Accelerator AMP.
• Learning rate 1e-4 cosine decay to 5e-5 over 80,000 microsteps, no warmup; original auxiliary losses and QUANTILES normalization. Four complete checkpoints were saved at 20k/40k/60k/80k; this inference release contains only 80k.
• 50 actions per chunk; 10 denoising steps. Training completed normally; model and optimizer finite-value checks passed.

Final evaluation: 1,000 episodes complete

The original BF16 / batch-1 evaluation completed normally on 2026-10-26 at 11:22:45 UTC; full verification at 12:31:50 UTC confirmed all 100 task-setting cells and 1,300 evidence-file SHA-256 values.

Setting	Successes / episodes	Success rate
clean	331 / 500	66.2%
randomized	176 / 500	35.2%
pooled	507 / 1,000	50.7%

All 50 tasks have 10 real completed episodes in each setting. This is a 10-per-cell evaluation, not 100 per cell. It retains native expert-solvability filtering, same-seed reconstruction, unseen instructions, native task-specific limits/success conditions and 50-action chunks. Infrastructure interruptions are preserved and excluded from policy failures; completed episodes are retained without replay. CuRobo used use_cuda_kernel=False for runtime compatibility. Expert filtering and simulator nondeterminism limit strict comparison with differently configured community results.

Read the final evaluation report, final JSON, per-task CSV, episode records and source-log SHA index. The earlier seven-page report and report JSON remain dated historical snapshots; their incomplete evaluation fields are superseded by the final report. Training curves, training logs, weights and processors are unchanged.

A separate FP32 diagnostic remains incomplete: 31/40 references processed, 26 matched, 5 unmatched, 9 pending; FP32 12/26 versus the same BF16 references 13/26. It is excluded from the formal 1,000 and does not establish a general precision benefit. Earlier engineering integration/concurrency trials are also excluded.

Load with the matching runtime

This uses the custom lingbot_vla_v2 policy, not a generic Transformers AutoModel. Use the pinned Xbotics LeRobot revision 2e41022f5f8c2274f5ce99f00e9f5fe0d493182b:

git clone https://github.com/Xbotics-Embodied-AI-club/lerobot.git
cd lerobot
git checkout 2e41022f5f8c2274f5ce99f00e9f5fe0d493182b
# In a suitable isolated Python 3.12 environment:
pip install -e '.[lingbot_vla_v2,av-dep]'

The original evaluation used PyTorch 2.11.0+cu128, Transformers 5.5.4, tokenizers 0.22.2 and safetensors 0.8.0. Choose a PyTorch build suitable for your hardware.

from huggingface_hub import snapshot_download
import importlib.util
root = snapshot_download("SuNya558/lingbot-vla-80k")
spec = importlib.util.spec_from_file_location("release_loader", root + "/load_model.py")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
policy, preprocess, postprocess = module.load_model(root, device="cuda")

At each new episode call policy.reset(). Observation keys are observation.state (14 values), observation.images.cam_high, observation.images.cam_left_wrist, observation.images.cam_right_wrist, and task. Supply RGB tensors and the original simulator's state/action units. Each step can use postprocess(policy.select_action(preprocess(observation))). Saved processors map the input keys to the three canonical camera slots and map the 14 actions back from the internal 55-slot representation.

Export and attribution

The BF16 weight file and saved processor/normalizer assets are unchanged; no quantization or extra policy training was performed. The export clears machine-specific model and unused teacher paths and adds a helper resolving bundled architecture configuration. Offline CPU checks match all model tensor names/shapes and load saved three-camera/14D processors; this is a metadata-model and processor check, not a new full policy forward or robot rollout after export. release.json records the original weight export; SHA256SUMS includes this final report update.

Optimizer states, datasets and distillation teacher weights are not included; original checkpoints remain retained separately. Upstream: LingBot-VLA 2.0, LeRobot, Xbotics LeRobot, Qwen3-VL-4B-Instruct. See LICENSE and NOTICE.