Model Details

Official model card of the ICLR2026 oral paper "DepthLM: Metric Depth from Vision Language Models". See our github for the eval and training code.

This model card includes the 12b model of DepthLM finetuned from Pixtral.

We show for the first time that VLMs can achieve comparable accuracy with pure vision models on metric depth estimation, with standard text-based SFT and no architecture chagne, i.e., no dense prediction head or regression/regularization loss is needed. Due to the simplicity, we can use DepthLM to train a unified VLM to handle various complex 3D understanding tasks such as speed or time estimation, and metric scale camera pose estimation, which require different architecture or hand-crafted pipelines in pure vision models.

Citation

If you find our code useful for your research, please consider citing:

  @article{cai2025depthlm,
      title={DepthLM: Metric Depth from Vision Language Models},
      author={Cai, Zhipeng and Yeh, Ching-Feng and Hu, Xu and Liu, Zhuang and Meyer, Gregory and Lei, Xinjie and Zhao, Changsheng and Li, Shang-Wen and Chandra, Vikas and Shi, Yangyang},
      journal={arXiv preprint arXiv:2509.25413},
      year={2025},
  }

Contact

Zhipeng Cai, Meta Inc, homepage: https://zhipengcai.github.io/, email: czptc2h at gmail dot com.

Results

Comparison with VLMs

Comparison with pure vision models

License

DepthLM is FAIR NC licensed as of now