Perceive-then-Plan

Checkpoints for Perceive-then-Plan: Layout-as-Policy for Monocular 3D Scene Layout Estimation (NeurIPS 2026).

Paper · Project page · Code

Folder	Model	Built on
perceiver-2b/	Stage 1 Perceiver (2B)	Qwen3-VL-2B-Instruct, VGGT-1B

Usage

git clone https://github.com/Colezwhy/Perceive-then-Plan
cd Perceive-then-Plan/perceiver
pip install -r requirements.txt
python tools/infer.py --eval-json data/omni3d/eval.json --image-root $OMNI3D/datasets \
    --output results/predictions.json --limit 1000

from transformers import AutoProcessor
from ptp_perceiver.model import PerceiverForConditionalGeneration

model = PerceiverForConditionalGeneration.from_pretrained(
    "Colezwhy/Perceive-then-Plan", subfolder="perceiver-2b", dtype="auto",
    geometry_encoder_path="facebook/VGGT-1B")
processor = AutoProcessor.from_pretrained("Colezwhy/Perceive-then-Plan", subfolder="perceiver-2b")

License

The language and vision weights are derived from Qwen3-VL (Apache-2.0). The geometry encoder weights come from VGGT-1B and are distributed under the VGGT License (LICENSE_VGGT.txt).

Citation

@inproceedings{zhou2026perceivethenplan,
  title={Perceive-then-Plan: Layout-as-Policy for Monocular 3D Scene Layout Estimation},
  author={Zhou, Junwei and Tai, Yu-Wing},
  booktitle={Advances in Neural Information Processing Systems},
  year={2026}
}