diff --git a/cameras/README-da3.md b/cameras/README-da3.md new file mode 100644 index 0000000000000000000000000000000000000000..171987e2750222c4b2d1469940ce8aeb2590dbb5 --- /dev/null +++ b/cameras/README-da3.md @@ -0,0 +1,145 @@ +--- +license: apache-2.0 +pipeline_tag: image-to-3d +tags: +- depth-estimation +- computer-vision +- monocular-depth +- multi-view-geometry +- pose-estimation +--- + +# Depth Anything 3: DA3-BASE + +
+ +[![Project Page](https://img.shields.io/badge/Project_Page-Depth_Anything_3-green)](https://depth-anything-3.github.io) +[![Paper](https://img.shields.io/badge/arXiv-Depth_Anything_3-red)](https://arxiv.org/abs/2511.10647) +[![Demo](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Demo-blue)](https://huggingface.co/spaces/depth-anything/Depth-Anything-3) # noqa: E501 + + +
+ +## Abstract + +We present Depth Anything 3 (DA3), a model that predicts spatially consistent geometry from an arbitrary number of visual inputs, with or without known camera poses. In pursuit of minimal modeling, DA3 yields two key insights: a single plain transformer (e.g., vanilla DINO encoder) is sufficient as a backbone without architectural specialization, and a singular depth-ray prediction target obviates the need for complex multi-task learning. Through our teacher-student training paradigm, the model achieves a level of detail and generalization on par with Depth Anything 2 (DA2). We establish a new visual geometry benchmark covering camera pose estimation, any-view geometry and visual rendering. On this benchmark, DA3 sets a new state-of-the-art across all tasks, surpassing prior SOTA VGGT by an average of 44.3% in camera pose accuracy and 25.1% in geometric accuracy. Moreover, it outperforms DA2 in monocular depth estimation. All models are trained exclusively on public academic datasets. + +## Model Description + +DA3 Base model for multi-view depth estimation and camera pose estimation. Compact foundation model with unified depth-ray representation. + +| Property | Value | +|----------|-------| +| **Model Series** | Any-view Model | +| **Parameters** | 0.12B | +| **License** | Apache 2.0 | + + + +## Capabilities + +- ✅ Relative Depth +- ✅ Pose Estimation +- ✅ Pose Conditioning + +## Quick Start + +### Installation + +```bash +git clone https://github.com/ByteDance-Seed/depth-anything-3 +cd depth-anything-3 +pip install -e . +``` + +### Basic Example + +```python +import torch +from depth_anything_3.api import DepthAnything3 + +# Load model from Hugging Face Hub +device = torch.device("cuda" if torch.cuda.is_available() else "cpu") +model = DepthAnything3.from_pretrained("depth-anything/da3-base") +model = model.to(device=device) + +# Run inference on images +images = ["image1.jpg", "image2.jpg"] # List of image paths, PIL Images, or numpy arrays +prediction = model.inference( + images, + export_dir="output", + export_format="glb" # Options: glb, npz, ply, mini_npz, gs_ply, gs_video +) + +# Access results +print(prediction.depth.shape) # Depth maps: [N, H, W] float32 +print(prediction.conf.shape) # Confidence maps: [N, H, W] float32 +print(prediction.extrinsics.shape) # Camera poses (w2c): [N, 3, 4] float32 +print(prediction.intrinsics.shape) # Camera intrinsics: [N, 3, 3] float32 +``` + +### Command Line Interface + +```bash +# Process images with auto mode +da3 auto path/to/images \ + --export-format glb \ + --export-dir output \ + --model-dir depth-anything/da3-base + +# Use backend for faster repeated inference +da3 backend --model-dir depth-anything/da3-base +da3 auto path/to/images --export-format glb --use-backend +``` + +## Model Details + +- **Developed by:** ByteDance Seed Team +- **Model Type:** Vision Transformer for Visual Geometry +- **Architecture:** Plain transformer with unified depth-ray representation +- **Training Data:** Public academic datasets only + +### Key Insights + +💎 A **single plain transformer** (e.g., vanilla DINO encoder) is sufficient as a backbone without architectural specialization. # noqa: E501 + +✨ A singular **depth-ray representation** obviates the need for complex multi-task learning. + +## Performance + +🏆 Depth Anything 3 significantly outperforms: +- **Depth Anything 2** for monocular depth estimation +- **VGGT** for multi-view depth estimation and pose estimation + +For detailed benchmarks, please refer to our [paper](https://arxiv.org/abs/2511.10647). # noqa: E501 + +## Limitations + +- The model is trained on academic datasets and may have limitations on certain domain-specific images # noqa: E501 +- Performance may vary depending on image quality, lighting conditions, and scene complexity + + +## Citation + +If you find Depth Anything 3 useful in your research or projects, please cite: + +```bibtex +@article{depthanything3, + title={Depth Anything 3: Recovering the visual space from any views}, + author={Haotong Lin and Sili Chen and Jun Hao Liew and Donny Y. Chen and Zhenyu Li and Guang Shi and Jiashi Feng and Bingyi Kang}, # noqa: E501 + journal={arXiv preprint arXiv:2511.10647}, + year={2025} +} +``` + +## Links + +- 🏠 [Project Page](https://depth-anything-3.github.io) +- 📄 [Paper](https://arxiv.org/abs/2511.10647) +- 💻 [GitHub Repository](https://github.com/ByteDance-Seed/depth-anything-3) +- 🤗 [Hugging Face Demo](https://huggingface.co/spaces/depth-anything/Depth-Anything-3) +- 📚 [Documentation](https://github.com/ByteDance-Seed/depth-anything-3#-useful-documentation) + +## Authors + +[Haotong Lin](https://haotongl.github.io/) · [Sili Chen](https://github.com/SiliChen321) · [Junhao Liew](https://liewjunhao.github.io/) · [Donny Y. Chen](https://donydchen.github.io) · [Zhenyu Li](https://zhyever.github.io/) · [Guang Shi](https://scholar.google.com/citations?user=MjXxWbUAAAAJ&hl=en) · [Jiashi Feng](https://scholar.google.com.sg/citations?user=Q8iay0gAAAAJ&hl=en) · [Bingyi Kang](https://bingykang.github.io/) # noqa: E501 \ No newline at end of file diff --git a/cameras/config.json b/cameras/config.json new file mode 100644 index 0000000000000000000000000000000000000000..4fbbc20883d49b5c1d06683753c8ad1a8d4a97e5 --- /dev/null +++ b/cameras/config.json @@ -0,0 +1,60 @@ +{ + "model_name": "da3-base", + "config": { + "__object__": { + "path": "depth_anything_3.model.da3", + "name": "DepthAnything3Net", + "args": "as_params" + }, + "net": { + "__object__": { + "path": "depth_anything_3.model.dinov2.dinov2", + "name": "DinoV2", + "args": "as_params" + }, + "name": "vitb", + "out_layers": [ + 5, + 7, + 9, + 11 + ], + "alt_start": 4, + "qknorm_start": 4, + "rope_start": 4, + "cat_token": true + }, + "head": { + "__object__": { + "path": "depth_anything_3.model.dualdpt", + "name": "DualDPT", + "args": "as_params" + }, + "dim_in": 1536, + "output_dim": 2, + "features": 128, + "out_channels": [ + 96, + 192, + 384, + 768 + ] + }, + "cam_enc": { + "__object__": { + "path": "depth_anything_3.model.cam_enc", + "name": "CameraEnc", + "args": "as_params" + }, + "dim_out": 768 + }, + "cam_dec": { + "__object__": { + "path": "depth_anything_3.model.cam_dec", + "name": "CameraDec", + "args": "as_params" + }, + "dim_in": 1536 + } + } +} \ No newline at end of file diff --git a/cameras/deps/addict-2.4.0.dist-info/INSTALLER b/cameras/deps/addict-2.4.0.dist-info/INSTALLER new file mode 100644 index 0000000000000000000000000000000000000000..a1b589e38a32041e49332e5e81c2d363dc418d68 --- /dev/null +++ b/cameras/deps/addict-2.4.0.dist-info/INSTALLER @@ -0,0 +1 @@ +pip diff --git a/cameras/deps/addict-2.4.0.dist-info/LICENSE b/cameras/deps/addict-2.4.0.dist-info/LICENSE new file mode 100644 index 0000000000000000000000000000000000000000..c9a0fae0f709d76c3246a876b03671cd866be162 --- /dev/null +++ b/cameras/deps/addict-2.4.0.dist-info/LICENSE @@ -0,0 +1,22 @@ +The MIT License (MIT) + +Copyright (c) 2014 Mats Julian Olsen + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + diff --git a/cameras/deps/addict-2.4.0.dist-info/METADATA b/cameras/deps/addict-2.4.0.dist-info/METADATA new file mode 100644 index 0000000000000000000000000000000000000000..864b661e77934b1dde26365b034e3bb5505cc1ee --- /dev/null +++ b/cameras/deps/addict-2.4.0.dist-info/METADATA @@ -0,0 +1,23 @@ +Metadata-Version: 2.1 +Name: addict +Version: 2.4.0 +Summary: Addict is a dictionary whose items can be set using both attribute and item syntax. +Home-page: https://github.com/mewwts/addict +Author: Mats Julian Olsen +Author-email: mats@plysjbyen.net +License: UNKNOWN +Platform: UNKNOWN +Classifier: Programming Language :: Python +Classifier: Programming Language :: Python :: 2.7 +Classifier: Programming Language :: Python :: 3.6 +Classifier: Programming Language :: Python :: 3.7 +Classifier: Programming Language :: Python :: 3.8 +Classifier: Programming Language :: Python :: 3.9 +Classifier: License :: OSI Approved :: MIT License +Classifier: Operating System :: OS Independent +Classifier: Intended Audience :: Developers +Classifier: Topic :: Software Development :: Libraries :: Python Modules + +Addict is a module that exposes a dictionary subclass that allows items to be set like attributes. Values are gettable and settable using both attribute and item syntax. For more info check out the README at 'github.com/mewwts/addict'. + + diff --git a/cameras/deps/addict-2.4.0.dist-info/RECORD b/cameras/deps/addict-2.4.0.dist-info/RECORD new file mode 100644 index 0000000000000000000000000000000000000000..5d72a633b641faf5996cbf5034960d4b12ac5d59 --- /dev/null +++ b/cameras/deps/addict-2.4.0.dist-info/RECORD @@ -0,0 +1,9 @@ +addict-2.4.0.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4 +addict-2.4.0.dist-info/LICENSE,sha256=ykiNM8US0LImFCCQr5DomuJmqQGik_if1kLf7JMeIsE,1085 +addict-2.4.0.dist-info/METADATA,sha256=ZjIJZ5lvhfohHxIWp79fkh0TtBsN5duX84na3bxU6vQ,1028 +addict-2.4.0.dist-info/RECORD,, +addict-2.4.0.dist-info/REQUESTED,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0 +addict-2.4.0.dist-info/WHEEL,sha256=EVRjI69F5qVjm_YgqcTXPnTAv3BfSUr0WVAHuSP3Xoo,92 +addict-2.4.0.dist-info/top_level.txt,sha256=GGfQKsz1N-bSRYzNpyp8GY8uAEqh3SWaXRbAnBUKjHA,7 +addict/__init__.py,sha256=xmlNiYYVoAD3_FfmgfOM2cc3wI17GSgsMBODbFRs_mY,233 +addict/addict.py,sha256=ScOfclaAM7ugyZoXOW2HHL9gCXt_XPe-i5bTLiY8ufU,4901 diff --git a/cameras/deps/addict-2.4.0.dist-info/REQUESTED b/cameras/deps/addict-2.4.0.dist-info/REQUESTED new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/cameras/deps/addict-2.4.0.dist-info/WHEEL b/cameras/deps/addict-2.4.0.dist-info/WHEEL new file mode 100644 index 0000000000000000000000000000000000000000..83ff02e961fce5ad7befce746ff02635e1616315 --- /dev/null +++ b/cameras/deps/addict-2.4.0.dist-info/WHEEL @@ -0,0 +1,5 @@ +Wheel-Version: 1.0 +Generator: bdist_wheel (0.35.1) +Root-Is-Purelib: true +Tag: py3-none-any + diff --git a/cameras/deps/addict-2.4.0.dist-info/top_level.txt b/cameras/deps/addict-2.4.0.dist-info/top_level.txt new file mode 100644 index 0000000000000000000000000000000000000000..b765b5f4af811ab630ffa864563a666850c266fc --- /dev/null +++ b/cameras/deps/addict-2.4.0.dist-info/top_level.txt @@ -0,0 +1 @@ +addict diff --git a/cameras/deps/addict/__init__.py b/cameras/deps/addict/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..b0af039c6298a3889a8e8b2e0329fe2313c120d0 --- /dev/null +++ b/cameras/deps/addict/__init__.py @@ -0,0 +1,10 @@ +from .addict import Dict +from .addict import Dict as Addict + + +__title__ = 'addict' +__version__ = '2.4.0' +__author__ = 'Mats Julian Olsen' +__license__ = 'MIT' +__copyright__ = 'Copyright 2014-2020 Mats Julian Olsen' +__all__ = ['Dict'] diff --git a/cameras/deps/addict/addict.py b/cameras/deps/addict/addict.py new file mode 100644 index 0000000000000000000000000000000000000000..55e02d1d17596c77a6f3642ba02eeb30971048bd --- /dev/null +++ b/cameras/deps/addict/addict.py @@ -0,0 +1,159 @@ +import copy + + +class Dict(dict): + + def __init__(__self, *args, **kwargs): + object.__setattr__(__self, '__parent', kwargs.pop('__parent', None)) + object.__setattr__(__self, '__key', kwargs.pop('__key', None)) + object.__setattr__(__self, '__frozen', False) + for arg in args: + if not arg: + continue + elif isinstance(arg, dict): + for key, val in arg.items(): + __self[key] = __self._hook(val) + elif isinstance(arg, tuple) and (not isinstance(arg[0], tuple)): + __self[arg[0]] = __self._hook(arg[1]) + else: + for key, val in iter(arg): + __self[key] = __self._hook(val) + + for key, val in kwargs.items(): + __self[key] = __self._hook(val) + + def __setattr__(self, name, value): + if hasattr(self.__class__, name): + raise AttributeError("'Dict' object attribute " + "'{0}' is read-only".format(name)) + else: + self[name] = value + + def __setitem__(self, name, value): + isFrozen = (hasattr(self, '__frozen') and + object.__getattribute__(self, '__frozen')) + if isFrozen and name not in super(Dict, self).keys(): + raise KeyError(name) + super(Dict, self).__setitem__(name, value) + try: + p = object.__getattribute__(self, '__parent') + key = object.__getattribute__(self, '__key') + except AttributeError: + p = None + key = None + if p is not None: + p[key] = self + object.__delattr__(self, '__parent') + object.__delattr__(self, '__key') + + def __add__(self, other): + if not self.keys(): + return other + else: + self_type = type(self).__name__ + other_type = type(other).__name__ + msg = "unsupported operand type(s) for +: '{}' and '{}'" + raise TypeError(msg.format(self_type, other_type)) + + @classmethod + def _hook(cls, item): + if isinstance(item, dict): + return cls(item) + elif isinstance(item, (list, tuple)): + return type(item)(cls._hook(elem) for elem in item) + return item + + def __getattr__(self, item): + return self.__getitem__(item) + + def __missing__(self, name): + if object.__getattribute__(self, '__frozen'): + raise KeyError(name) + return self.__class__(__parent=self, __key=name) + + def __delattr__(self, name): + del self[name] + + def to_dict(self): + base = {} + for key, value in self.items(): + if isinstance(value, type(self)): + base[key] = value.to_dict() + elif isinstance(value, (list, tuple)): + base[key] = type(value)( + item.to_dict() if isinstance(item, type(self)) else + item for item in value) + else: + base[key] = value + return base + + def copy(self): + return copy.copy(self) + + def deepcopy(self): + return copy.deepcopy(self) + + def __deepcopy__(self, memo): + other = self.__class__() + memo[id(self)] = other + for key, value in self.items(): + other[copy.deepcopy(key, memo)] = copy.deepcopy(value, memo) + return other + + def update(self, *args, **kwargs): + other = {} + if args: + if len(args) > 1: + raise TypeError() + other.update(args[0]) + other.update(kwargs) + for k, v in other.items(): + if ((k not in self) or + (not isinstance(self[k], dict)) or + (not isinstance(v, dict))): + self[k] = v + else: + self[k].update(v) + + def __getnewargs__(self): + return tuple(self.items()) + + def __getstate__(self): + return self + + def __setstate__(self, state): + self.update(state) + + def __or__(self, other): + if not isinstance(other, (Dict, dict)): + return NotImplemented + new = Dict(self) + new.update(other) + return new + + def __ror__(self, other): + if not isinstance(other, (Dict, dict)): + return NotImplemented + new = Dict(other) + new.update(self) + return new + + def __ior__(self, other): + self.update(other) + return self + + def setdefault(self, key, default=None): + if key in self: + return self[key] + else: + self[key] = default + return default + + def freeze(self, shouldFreeze=True): + object.__setattr__(self, '__frozen', shouldFreeze) + for key, val in self.items(): + if isinstance(val, Dict): + val.freeze(shouldFreeze) + + def unfreeze(self): + self.freeze(False) diff --git a/cameras/deps/antlr4/BufferedTokenStream.py b/cameras/deps/antlr4/BufferedTokenStream.py new file mode 100644 index 0000000000000000000000000000000000000000..aaa58c63dc02b8b0bf43398b751f1d14fef89ad9 --- /dev/null +++ b/cameras/deps/antlr4/BufferedTokenStream.py @@ -0,0 +1,302 @@ +# +# Copyright (c) 2012-2017 The ANTLR Project. All rights reserved. +# Use of this file is governed by the BSD 3-clause license that +# can be found in the LICENSE.txt file in the project root. + +# This implementation of {@link TokenStream} loads tokens from a +# {@link TokenSource} on-demand, and places the tokens in a buffer to provide +# access to any previous token by index. +# +#

+# This token stream ignores the value of {@link Token#getChannel}. If your +# parser requires the token stream filter tokens to only those on a particular +# channel, such as {@link Token#DEFAULT_CHANNEL} or +# {@link Token#HIDDEN_CHANNEL}, use a filtering token stream such a +# {@link CommonTokenStream}.

+from io import StringIO +from antlr4.Token import Token +from antlr4.error.Errors import IllegalStateException + +# need forward declaration +Lexer = None + +# this is just to keep meaningful parameter types to Parser +class TokenStream(object): + + pass + + +class BufferedTokenStream(TokenStream): + __slots__ = ('tokenSource', 'tokens', 'index', 'fetchedEOF') + + def __init__(self, tokenSource:Lexer): + # The {@link TokenSource} from which tokens for this stream are fetched. + self.tokenSource = tokenSource + + # A collection of all tokens fetched from the token source. The list is + # considered a complete view of the input once {@link #fetchedEOF} is set + # to {@code true}. + self.tokens = [] + + # The index into {@link #tokens} of the current token (next token to + # {@link #consume}). {@link #tokens}{@code [}{@link #p}{@code ]} should be + # {@link #LT LT(1)}. + # + #

This field is set to -1 when the stream is first constructed or when + # {@link #setTokenSource} is called, indicating that the first token has + # not yet been fetched from the token source. For additional information, + # see the documentation of {@link IntStream} for a description of + # Initializing Methods.

+ self.index = -1 + + # Indicates whether the {@link Token#EOF} token has been fetched from + # {@link #tokenSource} and added to {@link #tokens}. This field improves + # performance for the following cases: + # + #