README.md

metadata

license: apache-2.0
tags:
  - vision
  - 3d-object-detection
  - autonomous-driving
  - nuscenes
  - transformers
  - bevformer
datasets:
  - nuscenes
pipeline_tag: object-detection

BEVFormer-tiny (nuScenes 10-Class 3D Object Detection)

Community conversion of BEVFormer-tiny model weights converted to Hugging Face Transformers format with safetensors.

Model Details

• Architecture: BEVFormer (v1)
• Backbone: ResNet-50 (single FPN feature level, stage 4 / C5)
• Cameras: 6 surround-view cameras (front, front right, front left, back, back left, back right)
• BEV Grid: 50 × 50 (spatial range [-51.2m, 51.2m] × [-51.2m, 51.2m])
• Transformer Encoder Layers: 3 (Spatial Cross Attention + Temporal Self Attention)
• Transformer Decoder Layers: 6 (900 object queries)
• Classes: 10 nuScenes classes (car, truck, construction vehicle, bus, trailer, barrier, motorcycle, bicycle, pedestrian, traffic cone)
• Parameters: ~33.5M

Quickstart

import torch
from transformers import BevformerFor3DObjectDetection, BevformerImageProcessor

# Load model and preprocessor
model = BevformerFor3DObjectDetection.from_pretrained("hasanfaesal/bevformer-tiny")
processor = BevformerImageProcessor.from_pretrained("hasanfaesal/bevformer-tiny")

# Multi-camera surround view images (batch_size=1, num_cameras=6, height=900, width=1600, channels=3)
# lidar2img: Camera calibration projection matrices (batch_size=1, num_cameras=6, 4, 4)
# can_bus: CAN bus ego motion array (batch_size=1, 18)

# Preprocessing: handles 0.5x scaling, ImageNet normalization, and lidar2img adjustment
inputs = processor(images=images, lidar2img=lidar2img, return_tensors="pt")

model.eval()
with torch.no_grad():
    outputs = model(**inputs, can_bus=can_bus)

logits = outputs.logits  # (batch_size, num_decoder_layers, num_queries, num_classes)
pred_boxes = outputs.pred_boxes  # (batch_size, num_decoder_layers, num_queries, 10)
bev_embed = outputs.bev_embed  # (batch_size, bev_h * bev_w, d_model) -> pass as prev_bev for next frame

Reference

@inproceedings{li2022bevformer,
  title={BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers},
  author={Li, Zhiqi and Wang, Wenhai and Li, Hongyang and Xie, Enze and Sima, Chonghao and Lu, Tong and Qiao, Yu and Dai, Jifeng},
  booktitle={ECCV},
  year={2022}
}