PQR3D

Code and five released checkpoints for PQR3D: Progressive Query Refinement over Reference-Conditioned Temporal Windows for Multi-View 3D Object Detection. This Hub repository mirrors the original PQR3D project at commit 5b105ba. The paper, implementation, and checkpoints are credited to the original authors.

• Paper: arXiv:2609.32163
• Code: GitHub
• Original checkpoint links and reported results: Model Zoo

Checkpoint contents

The five .pth files in checkpoints/ are trained PQR3D model checkpoints. Download the checkpoint matching the config you will use:

Configuration	Checkpoint
r50_nuimg_704x256	r50_nuimg_704x256.pth
r50_nuimg_704x256_625q	r50_nuimg_704x256_625q.pth
r101_nuimg_1408x512	r101_nuimg_1408x512.pth
vov99_dd3d_1600x640	vov99_dd3d_1600x640.pth
vit_eva02_1600x640	vit_eva02_1600x640.pth

Download and evaluate

Evaluation uses the project's val.py loader. It requires a CUDA GPU, the project dependencies, and the nuScenes dataset with generated sweep info files. Follow Environment and Prepare Dataset below before running the example.

Get the matching source version and install the Hub download helper:

git clone https://github.com/huiyegit/PQR3D.git
cd PQR3D
git checkout 5b105ba4a697b1f4979913ba4ebe91345952269c
pip install huggingface_hub

After preparing data/nuscenes/, download the first checkpoint into checkpoints/:

from huggingface_hub import hf_hub_download

checkpoint = hf_hub_download(
    repo_id="yhhugging/PQR3D",
    filename="checkpoints/r50_nuimg_704x256.pth",
    local_dir=".",
)
print(checkpoint)

Run the Python snippet above from the PQR3D directory, then evaluate on the nuScenes validation split:

python val.py \
    --config configs/r50_nuimg_704x256.py \
    --weights checkpoints/r50_nuimg_704x256.pth

For another model, select the matching config and checkpoint from the table above. The validation script builds the model from the MMCV config and loads the checkpoint with mmcv.runner.load_checkpoint.

Intended use and limitations

PQR3D is a research model for camera-based multi-view 3D object detection on nuScenes. The provided evaluation path processes the dataset's camera frames and temporal sweeps; it requires nuScenes files and a CUDA GPU. The repository does not provide a standalone single-image prediction interface or a from_pretrained loader. Results in the Model Zoo below are reported by the original project and have not been rerun for this Hub mirror.

Original project instructions

The following setup, dataset, training, and evaluation instructions come from the original project README.

News

• 2026-09-29: We release the arXiv version arXiv:2609.32163.
• 2026-09-25: We release the source code and pretrained weights.

Model Zoo

Setting	Pretrain	epochs	Training Cost	NDSval	NDStest	Weights	Log
r50_nuimg_704x256	nuImg	24	13h (8x4090)	58.2	-	Hub checkpoint (original)	log
r50_nuimg_704x256_625q	nuImg	48	22h (8x4090)	58.5	-	Hub checkpoint (original)	log
r101_nuimg_1408x512	nuImg	24	1d16h (8x4090)	61.2	-	Hub checkpoint (original)	log
vov99_dd3d_1600x640	DD3D	24	11d7h (8xA100)	-	68.0	Hub checkpoint (original)	log
vit_eva02_1600x640	EVA02	24	14d16h (8xA100)	-	71.6	Hub checkpoint (original)	log

Environment

Install PyTorch 2.0 + CUDA 11.8:

conda create -n pqr3d python=3.8
conda activate pqr3d
conda install pytorch==2.0.0 torchvision==0.15.0 pytorch-cuda=11.8 -c pytorch -c nvidia

Install other dependencies:

pip install openmim
mim install mmcv-full==1.6.0
mim install mmdet==2.28.2
mim install mmsegmentation==0.30.0
mim install mmdet3d==1.0.0rc6
pip install setuptools==59.5.0
pip install numpy==1.23.5

Install turbojpeg and pillow-simd to speed up data loading (optional but important):

sudo apt-get update
sudo apt-get install -y libturbojpeg
pip install pyturbojpeg
pip uninstall pillow
pip install pillow-simd==9.0.0.post1

Compile CUDA extensions:

cd models/csrc
python setup.py build_ext --inplace

Prepare Dataset

1. Download nuScenes from https://www.nuscenes.org/nuscenes and put it in data/nuscenes.
2. Download the generated info file from Google Drive.
3. Folder structure:

data/nuscenes
├── maps
├── nuscenes_infos_test_sweep.pkl
├── nuscenes_infos_train_sweep.pkl
├── nuscenes_infos_train_mini_sweep.pkl
├── nuscenes_infos_val_sweep.pkl
├── nuscenes_infos_val_mini_sweep.pkl
├── samples
├── sweeps
├── v1.0-test
└── v1.0-trainval

Training

Download pretrained weights and put it in directory pretrain/:

pretrain
├── cascade_mask_rcnn_r101_fpn_1x_nuim_20201024_134804-45215b1e.pth
├── cascade_mask_rcnn_r50_fpn_coco-20e_20e_nuim_20201009_124951-40963960.pth

Train PQR3D with 8 GPUs:

torchrun --nproc_per_node 8 train.py --config configs/r50_nuimg_704x256.py

Train PQR3D with 4 GPUs (i.e the last four GPUs):

export CUDA_VISIBLE_DEVICES=4,5,6,7
torchrun --nproc_per_node 4 train.py --config configs/r50_nuimg_704x256.py

The batch size for each GPU will be scaled automatically. So there is no need to modify the batch_size in config files.

Evaluation

Single-GPU evaluation:

export CUDA_VISIBLE_DEVICES=0
python val.py --config configs/r50_nuimg_704x256.py --weights checkpoints/r50_nuimg_704x256.pth

Multi-GPU evaluation:

export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
torchrun --nproc_per_node 8 val.py --config configs/r50_nuimg_704x256.py --weights checkpoints/r50_nuimg_704x256.pth

Timing

FPS is measured with a single GPU:

export CUDA_VISIBLE_DEVICES=0
python timing.py --config configs/r50_nuimg_704x256.py --weights checkpoints/r50_nuimg_704x256.pth

Visualization

Visualize the predicted bbox:

python viz_bbox_predictions.py --config configs/r50_nuimg_704x256.py --weights checkpoints/r50_nuimg_704x256.pth

Acknowledgements

Our implementation is based on these excellent open-source projects: SparseBEV, Sparse4D, StreamPETR, HoP, BEVFormer, BEVDet

License

The source repository includes an MIT license. nuScenes data and external pretrained backbones are distributed separately under their respective terms.