| --- |
| license: apache-2.0 |
| tags: |
| - vision |
| - 3d-object-detection |
| - autonomous-driving |
| - nuscenes |
| - transformers |
| - bevformer |
| datasets: |
| - nuscenes |
| pipeline_tag: object-detection |
| --- |
| |
| # BEVFormer-tiny (nuScenes 10-Class 3D Object Detection) |
|
|
| Community conversion of BEVFormer-tiny model weights converted to Hugging Face Transformers format with `safetensors`. |
|
|
| ## Model Details |
| - **Architecture:** BEVFormer (v1) |
| - **Backbone:** ResNet-50 (single FPN feature level, stage 4 / C5) |
| - **Cameras:** 6 surround-view cameras (front, front right, front left, back, back left, back right) |
| - **BEV Grid:** 50 × 50 (spatial range [-51.2m, 51.2m] × [-51.2m, 51.2m]) |
| - **Transformer Encoder Layers:** 3 (Spatial Cross Attention + Temporal Self Attention) |
| - **Transformer Decoder Layers:** 6 (900 object queries) |
| - **Classes:** 10 nuScenes classes (car, truck, construction vehicle, bus, trailer, barrier, motorcycle, bicycle, pedestrian, traffic cone) |
| - **Parameters:** ~33.5M |
|
|
| ## Quickstart |
|
|
| ```python |
| import torch |
| from transformers import BevformerFor3DObjectDetection, BevformerImageProcessor |
| |
| # Load model and preprocessor |
| model = BevformerFor3DObjectDetection.from_pretrained("hasanfaesal/bevformer-tiny") |
| processor = BevformerImageProcessor.from_pretrained("hasanfaesal/bevformer-tiny") |
| |
| # Multi-camera surround view images (batch_size=1, num_cameras=6, height=900, width=1600, channels=3) |
| # lidar2img: Camera calibration projection matrices (batch_size=1, num_cameras=6, 4, 4) |
| # can_bus: CAN bus ego motion array (batch_size=1, 18) |
| |
| # Preprocessing: handles 0.5x scaling, ImageNet normalization, and lidar2img adjustment |
| inputs = processor(images=images, lidar2img=lidar2img, return_tensors="pt") |
| |
| model.eval() |
| with torch.no_grad(): |
| outputs = model(**inputs, can_bus=can_bus) |
| |
| logits = outputs.logits # (batch_size, num_decoder_layers, num_queries, num_classes) |
| pred_boxes = outputs.pred_boxes # (batch_size, num_decoder_layers, num_queries, 10) |
| bev_embed = outputs.bev_embed # (batch_size, bev_h * bev_w, d_model) -> pass as prev_bev for next frame |
| ``` |
|
|
| ## Reference |
| ```bibtex |
| @inproceedings{li2022bevformer, |
| title={BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers}, |
| author={Li, Zhiqi and Wang, Wenhai and Li, Hongyang and Xie, Enze and Sima, Chonghao and Lu, Tong and Qiao, Yu and Dai, Jifeng}, |
| booktitle={ECCV}, |
| year={2022} |
| } |
| ``` |
|
|
