|
Download README.md from apple/DepthPro2-coreml: direct link, hf CLI and curl.
- Browser
- Download file 5.6 kB
-
https://huggingface.co/apple/DepthPro2-coreml/resolve/main/README.md
- Command line
-
hf download hf://apple/DepthPro2-coreml/README.md
-
curl -L -o README.md https://huggingface.co/apple/DepthPro2-coreml/resolve/main/README.md
5.6 kB
| license: other | |
| license_name: apple-amlr | |
| license_link: https://github.com/apple-aiml-research/ml-depth-pro-2/blob/main/LICENSE_MODEL | |
| pipeline_tag: depth-estimation | |
| library_name: coreml | |
| tags: | |
| - depth-estimation | |
| - metric-depth | |
| - monocular-depth | |
| - video-depth | |
| - coreml | |
| <h1 align="center">Depth Pro 2 CoreML Models</h1> | |
| <p align="center"> | |
| <a href="https://github.com/apple-aiml-research/ml-depth-pro-2"><img src="https://img.shields.io/badge/GitHub-181717?style=flat-square&logo=github&logoColor=white" alt="GitHub"/></a> | |
| <a href="https://arxiv.org/abs/2611.26603"><img src="https://img.shields.io/badge/Technical%20Report-B31B1B?style=flat-square&logo=arxiv&logoColor=white" alt="Technical Report"/></a> | |
| <a href="https://apple-aiml-research.github.io/ml-depth-pro-2/"><img src="https://img.shields.io/badge/Project%20Page-666666?style=flat-square&logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSJ3aGl0ZSIgc3Ryb2tlLXdpZHRoPSIyIj48Y2lyY2xlIGN4PSIxMiIgY3k9IjEyIiByPSIxMCIvPjxwYXRoIGQ9Ik0yIDEyaDIwTTEyIDJhMTUgMTUgMCAwIDEgMCAyME0xMiAyYTE1IDE1IDAgMCAwIDAgMjAiLz48L3N2Zz4%3D" alt="Project Page"/></a> | |
| <a href="https://huggingface.co/apple/DepthPro2"><img src="https://img.shields.io/badge/All%20Sizes-FFD21E?style=flat-square&logo=huggingface&logoColor=black" alt="All Sizes"/></a> | |
| </p> | |
| Sharp, temporally consistent metric depth from a single camera, for photos and videos. These are | |
| the Core ML packages (`.mlpackage`) of the torch models on the [Depth Pro 2 page](https://huggingface.co/apple/DepthPro2). | |
| ## Models | |
| | Model | Package | Size | | |
| |:---:|---|---:| | |
| | S | [`depthpro2-s_w4.mlpackage`](https://huggingface.co/apple/DepthPro2-coreml/tree/main/depthpro2-s_w4.mlpackage) | 51 MB | | |
| | B | [`depthpro2-b_w4.mlpackage`](https://huggingface.co/apple/DepthPro2-coreml/tree/main/depthpro2-b_w4.mlpackage) | 188 MB | | |
| | L | [`depthpro2-l_w4.mlpackage`](https://huggingface.co/apple/DepthPro2-coreml/tree/main/depthpro2-l_w4.mlpackage) | 891 MB | | |
| | H+ | [`depthpro2-h_w4.mlpackage`](https://huggingface.co/apple/DepthPro2-coreml/tree/main/depthpro2-h_w4.mlpackage) | 1.6 GB | | |
| | Field of view | [`depthpro2-fov_512x896.mlpackage`](https://huggingface.co/apple/DepthPro2-coreml/tree/main/depthpro2-fov_512x896.mlpackage) | 91 MB | | |
| Larger models give sharper, steadier depth; smaller ones run faster. On a laptop, start with S. | |
| - **Window.** `w4` is a temporal window of 4 frames. The model takes one frame at a time and keeps the earlier ones in its state, so create a new state for each new video. | |
| - **Sizes.** Each depth package has one function per input size, named `res_<W>x<H>`: `res_854x480` (default), `res_640x360`, `res_896x512`, `res_1138x640` and `res_1280x720`. | |
| - **Field of view.** The field-of-view package has a single size, 896x512, for 16:9 frames. It estimates the focal length of cameras that don't report it. | |
| - **Memory.** It grows quickly with the size: S needs about 1.5 GB at 854x480 and 5 GB at 1280x720; L about 4 GB and 18 GB. On a Mac with 16 GB, use S or B, or keep to the default size. | |
| Requires Apple silicon and macOS 15 or later. | |
| ## Try it | |
| Clone the [GitHub repository](https://github.com/apple-aiml-research/ml-depth-pro-2), put a depth package (and the field-of-view package, for cameras without a known focal length) next to each other, and run the Mac demo on it: | |
| ```bash | |
| ./demo/run.sh depthpro2-s_w4.mlpackage | |
| ``` | |
| To build a package at your own size `-s HxW` and window `-w`, see the [demo's README](https://github.com/apple-aiml-research/ml-depth-pro-2/blob/main/demo/README.md): | |
| ```bash | |
| uv run --extra coreml python -m demo.export stream -m depthpro2-s -w 4 -s 480x854 | |
| ``` | |
| ## Use in Swift | |
| ```swift | |
| import CoreML | |
| let compiled = try MLModel.compileModel(at: URL(filePath: "depthpro2-s_w4.mlpackage")) | |
| let config = MLModelConfiguration() | |
| config.computeUnits = .cpuAndGPU | |
| config.functionName = "res_854x480" // omit for the default size | |
| let model = try MLModel(contentsOf: compiled, configuration: config) | |
| let state = model.makeState() // a new state starts a new video | |
| // frame: float16 MLMultiArray (1, 3, 480, 854), RGB in [0, 1] | |
| let input = try MLDictionaryFeatureProvider(dictionary: ["frame": MLFeatureValue(multiArray: frame)]) | |
| let output = try model.prediction(from: input, using: state) | |
| let inverseDepth = output.featureValue(for: "inverse_depth")!.multiArrayValue! // (1, 480, 854) | |
| ``` | |
| `inverse_depth` is normalized by the focal length: depth in meters is `f_px / (W * inverse_depth)`, with `f_px` the focal length in pixels of the H×W frame. The demo's [Preprocess.swift](https://github.com/apple-aiml-research/ml-depth-pro-2/blob/main/demo/apple/Sources/demo/Preprocess.swift) shows how to prepare camera frames. | |
| Videos and the torch models are on the [Depth Pro 2 page](https://huggingface.co/apple/DepthPro2). | |
| ## License | |
| The weights are released under the [Apple Machine Learning Research Model license](https://github.com/apple-aiml-research/ml-depth-pro-2/blob/main/LICENSE_MODEL), for | |
| research use only: non-commercial scientific research and academic development. | |
| ## Citation | |
| ```bibtex | |
| @article{DepthPro2:2026:arxiv, | |
| title = {Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth}, | |
| author = {Peiyun Hu and Marko Mihajlovic and Lyndon Duong and Lahav Lipson and Wei Dong and Ama\"{e}l Delaunoy and Stephan R. Richter and Vladlen Koltun}, | |
| journal = {arXiv preprint arXiv:2611.26603}, | |
| year = {2026}, | |
| url = {https://arxiv.org/abs/2611.26603}, | |
| } | |
| ``` | |