| --- |
| library_name: pytorch |
| license: other |
| tags: |
| - qai-hub-models |
| - qualcomm |
| - android |
| pipeline_tag: depth-estimation |
| --- |
| |
| # DepthPro: Sharp monocular metric depth in less than a second, on-device |
|
|
| Apple DepthPro is a zero-shot monocular metric depth estimator that emits high-resolution, sharp depth maps from a single 1536x1536 RGB image. The architecture is a multi-scale Vision Transformer built on Dinov2 encoders with DPT-style fusion; alongside the depth map the model predicts a per-image horizontal field of view, which downstream calibration converts into a focal length in pixels. This recipe wraps Apple's HuggingFace checkpoint (`apple/DepthPro-hf`, ~952M parameters) at the model's native 1536x1536 input resolution. |
|
|
| This is based on the implementation of DepthPro found [here](https://github.com/apple/ml-depth-pro). |
| This is a standalone recipe compatible with the [Qualcomm® AI Hub Models](https://github.com/quic/ai-hub-models) CLI — it can be compiled and evaluated on real Snapdragon devices via [Qualcomm® AI Hub Workbench](https://workbench.aihub.qualcomm.com). |
|
|
| Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device. |
|
|
|
|
| ## Setup |
| ### 1. Install the package |
| Install the base package, fetch this recipe from Hugging Face, then use the |
| `qai-hub-models` CLI to install the recipe's dependencies: |
| ```bash |
| # NOTE: 3.10 <= PYTHON_VERSION < 3.14 is supported. |
| pip install qai-hub-models |
| qai-hub-models register ashwmurt/depth_pro |
| qai-hub-models install depth_pro |
| ``` |
| `register` downloads the recipe and names it `depth_pro`, which is how every |
| command below refers to it. |
|
|
| ### 2. Configure Qualcomm® AI Hub Workbench |
| Sign-in to [Qualcomm® AI Hub Workbench](https://workbench.aihub.qualcomm.com/) with your |
| Qualcomm® ID. Once signed in navigate to `Account -> Settings -> API Token`. |
|
|
| With this API token, you can configure your client to run models on the cloud |
| hosted devices. |
| ```bash |
| qai-hub configure --api_token API_TOKEN |
| ``` |
| Navigate to [docs](https://workbench.aihub.qualcomm.com/docs/) for more information. |
|
|
| ## Run CLI Demo |
| Run the following simple CLI demo to verify the model is working end to end: |
|
|
| ```bash |
| qai-hub-models demo depth_pro |
| ``` |
| More details on the CLI tool can be found with the `--help` option. See |
| [demo.py](demo.py) for sample usage of the model including pre/post processing |
| scripts. |
|
|
| By default, the demo will run locally in PyTorch. Pass `--eval-mode on-device` to run the model on a cloud-hosted target device. |
|
|
| ## Export for on-device deployment |
| To run the model on Qualcomm® devices, you must export the model for use with an edge runtime such as |
| TensorFlow Lite, ONNX Runtime, or Qualcomm AI Engine Direct. |
| Use the following command to export the model: |
| ```bash |
| qai-hub-models export depth_pro |
| ``` |
| Additional options are documented with the `--help` option. |
|
|
| ## License |
| * The license for the original implementation of DepthPro can be found |
| [here](https://huggingface.co/apple/DepthPro/blob/main/LICENSE). |
|
|
| ## References |
| * [Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://arxiv.org/abs/2410.02073) |
| * [Source Model Implementation](https://github.com/apple/ml-depth-pro) |
|
|
| ## Community |
| * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI. |
| * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com). |
|
|
