You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

p3vlmcheckpoint

Trained weights of p3vlm, a 3D vision-language model over 3D Gaussian Splatting scenes. The model answers questions about a ScanNet scene, captions it, and describes or classifies the object at a 3D point.

Code, setup, training and evaluation: https://github.com/amhalacheva/p3vlm

tune3d_stage2_posttrain/pytorch_model.bin       best stage-2 (post-training) checkpoint, selected on validation
training_data/scannet/Scan2Cap/                 ScanRefer annotations with oriented 3D boxes
    scanrefer_train_obb.json
    scanrefer_val_obb.json
training_data/scannet/LL3DA_annotations/Nr3D/   Nr3D annotations with oriented 3D boxes
    nr3d_train.json
    nr3d_val.json

Checkpoint

pytorch_model.bin contains only the parameters trained by p3vlm: the 3D–language bridge, the LoRA adapters and the fine-tuned SceneSplat encoder. The frozen weights are loaded separately from Qwen/Qwen3-VL-4B-Instruct and the SceneSplat backbone, as described in the code repository.

Training data

The ScanRefer and Nr3D files are the versions used to train this checkpoint. Each entry carries an oriented 3D box of the referred object, obb (pos_x,_y,_z,angle_z,scale_x,_y,_z), recomputed from the ScanNet instance masks of the Gaussian-splat scenes. The file layout mirrors the annotation root that the p3vlm configs read ($GSPLAN_ANNO_ROOT/scannet/...).

Usage

git clone https://github.com/amhalacheva/p3vlm.git && cd p3vlm
# environment, Qwen3-VL and SceneSplat weights, ScanNet 3DGS scenes: see the repository README

# checkpoint
huggingface-cli download amhalacheva/p3_vlm tune3d_stage2_posttrain/pytorch_model.bin --local-dir weights/p3vlm

# ScanRefer / Nr3D annotations, placed into the annotation root used by the configs
huggingface-cli download amhalacheva/p3_vlm --include "training_data/*" --local-dir /tmp/p3_vlm
cp -r /tmp/p3_vlm/training_data/scannet "${GSPLAN_ANNO_ROOT:-data_root/GSPlan_anno}/"

# evaluate or run inference
CKPT=weights/p3vlm/tune3d_stage2_posttrain bash scripts/eval.sh
python inference.py --ckpt weights/p3vlm/tune3d_stage2_posttrain --scene scene0011_00 \
    --task qa --question "What color is the chair in the kitchen?"

Data terms

The annotations are derived from ScanRefer and Nr3D / ReferIt3D, which are built on ScanNet. Use them under the terms of those datasets and cite the original works.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support