npow/nanovla-bimanual-yam-3cam

🤖 nanoVLA: The Multi-Variant 3-Camera Bimanual YAM Policy Suite

Welcome to the official model repository for nanoVLA, a high-frequency, lightweight Vision-Language-Action (VLA) suite optimized for bimanual manipulation on the I2RT Bimanual YAM (Dual 7-DoF Arm) platform with 3 cameras (Overhead, Left Wrist, Right Wrist).

---

🏆 Comprehensive Architecture Leaderboard

All checkpoints below are packaged and downloadable directly from this repository:

Model Checkpoint	Architecture & Key Innovation	14-DoF Action MSE	Frequency / Latency	Primary Strength
spatial_chunked_flagship.pt	588 Spatial Patches + Motion Cross-Attn + H=16 Chunking	0.19648	30.6 Hz	🥇 Flagship Multi-Camera Policy
arena_champion.pt	Continuous Smooth L1 Huber + 14-DoF Proprioception	0.00689	64.0 Hz	🥇 Lowest Single-Step Action MSE
foveated_champion.pt	Foveated Dynamic Spatial Token Pruning (66% Pruned)	0.14443	73.1 Hz ⚡	2.4x Speedup + Visual Salience
dual_speed_champion.pt	Dual-Speed Fast/Slow Hierarchy (5 Hz / 1000 Hz)	0.03567	907.7 Hz 🚀	1.1 ms High-Frequency Impedance
action_chunking_champion.pt	H=16 Action Chunking + Temporal Ensembling	0.22064	49.1 Hz	0.027 rad/s² Silky Smooth Jitter
so400m_champion.pt	Scaled SigLIP-SO400M-384 Backbone (900.1M Params)	0.06013	4.3 Hz	Rich Zero-Shot Spatial Grounding
critic_recovery_champion.pt	Verification Critic & Anomaly Recovery Detector	0.12926	>10,000 Hz	Real-time Slip & Grasp Monitoring
bimanual_coupled_champion.pt	Cross-Arm Attention & Relative Distance Constraint	0.16939	250.9 Hz	Prevents Dropping Grasped Objects
fk_consistent_champion.pt	Differentiable Forward Kinematics 3D Metric Loss	0.13258	50.0 Hz	3D Millimeter Cartesian Precision
rvq_hybrid_champion.pt	Residual Vector Quantization (K=64 Codebook)	0.11779	50.3 Hz	Multi-Modal + Sub-mm Precision
generalization_champion.pt	Robust Domain Randomization (5-Strata Invariant)	0.03944	64.0 Hz	Extreme Lighting / Sensor Noise

---

🦾 Multi-Camera Coordinate Setup

                                  Prompt: "bimanual pick and place red cube"
                                                   │
    [Top Camera]         [Left Wrist Cam]    [Right Wrist Cam]     [SigLIP Text Tokens]
   (3, 224, 224)          (3, 224, 224)       (3, 224, 224)             (1, 16)
         │                      │                   │                      │
         ▼                      ▼                   ▼                      ▼
  ┌──────────────┐       ┌──────────────┐    ┌──────────────┐       ┌──────────────┐
  │ SigLIP-Base  │       │ SigLIP-Base  │    │ SigLIP-Base  │       │ Language     │
  │ Vision (86M) │       │ Vision (86M) │    │ Vision (86M) │       │ Embedding    │
  └──────┬───────┘       └──────┬───────┘    └──────┬───────┘       └──────┬───────┘
         │ 196 tokens           │ 196 tokens        │ 196 tokens           │ 16 tokens
         └──────────────────────┴──────────┬────────┴──────────────────────┘
                                           │  588 + 16 = 604 tokens
                                           ▼
                            ┌──────────────────────────────┐
                            │  nanoVLA Transformer Trunk   │
                            │  (6 Layers, 384 Dim, 6 Heads)│
                            └──────────────┬───────────────┘
                                           │
                                           ▼
                                 ┌────────────────────┐
                                 │ 14-DoF Action Head │
                                 │  Left (7) + Right(7)
                                 └────────────────────┘

---

🚀 Quickstart Inference

import torch
from model import NanoVLA, NanoVLAConfig

# Load Flagship Champion Checkpoint
ckpt = torch.load("spatial_chunked_flagship.pt", map_location="cuda")
model = NanoVLA(ckpt['config']).cuda()
model.load_state_dict(ckpt['model_state_dict'])
model.eval()

# Predict 14-DoF action for 3 cameras
imgs = torch.rand(1, 3, 3, 224, 224, device="cuda")
txt = torch.zeros(1, 16, dtype=torch.long, device="cuda")
proprio = torch.zeros(1, 14, device="cuda")

with torch.no_grad():
    actions = model.predict_action(imgs, txt, proprio_state=proprio)
print("Predicted 14-DoF Actions (7 Left + 7 Right):", actions.cpu().numpy())

---

📄 Citation & License

MIT License (c) 2026 npow. Developed for fast, accessible, and robust robot learning.