Marigold V2 (fastforks)
Marigold V2 (Pavlovic, Wandel, Obukhov et al. 2026): affine-invariant log depth by one rectified-flow step of Qwen-Image-Edit-2509's 20 B transformer (NF4 4-bit, as the authors run it) with a rank-128 depth LoRA and a fine-tuned Qwen VAE decoder (depth/Log-stage2).
This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/huawei-bayerlab/marigold-v2 at main.
Author weight repos: https://huggingface.co/fastforks/marigold-v2-depth-nf4.
Task: depth. Tier: experimental. Outputs: depth. Backends: portable, kernels.
Input: one RGB image. Output: a depth or inverse-depth map. kind (a card field, result.kind) says what the numbers mean: metric = metres; scale_invariant, affine_invariant, affine_invariant_log (log depth) and affine_invariant_inverse are relative (unknown scale or shift, not metres). Metric models that use the focal length take model(image, intrinsics=K) (K [3, 3], pixels of the input): metric3d-vit-large and da3metric-large need it for correct metres (da3metric-large: metres = output x focal / 300; without intrinsics= a 300 px focal length is assumed, wrong for a real camera). unidepth-v2-vitl14, unik3d-vitl, depth-pro and MoGe estimate the camera themselves (given intrinsics= replace fx); zoedepth-nk and the relative models (da3mono-large too) take none. Normals-only models (kind none) return result.normals [B, H, W, 3] (unit vectors, camera frame: x right, y down, z forward).
Variants
Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.
| variant | weights (author link) | SHA-256 | weights licence | commercial use |
|---|---|---|---|---|
marigold-v2-depth@1 |
marigold-v2-depth-log-stage2-nf4.safetensors | 4b4aee27acdb |
Apache-2.0 | yes |
Licence
Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Licence files: LICENSE, NOTICE.
Training data: Qwen-Image-Edit-2509 (Alibaba, Apache-2.0) fine-tuned (LoRA + VAE decoder) on synthetic depth data per the Marigold V2 paper (arXiv 2609.08084); the training data terms are not checked.
- Licence note: The converted weight file (NF4 transformer, LoRA, folded constants, 2-D VAE) is hosted by fastforks under the originals' Apache-2.0 licence; NOTICE keeps both attributions and lists the changes.
Licence evidence: https://huggingface.co/huawei-bayerlab/marigold-v2/blob/main/README.md (license: apache-2.0; base model Qwen/Qwen-Image-Edit-2509)
Input size
any H x W >= 1, resized or padded inside the model
Tested up to (portable, fp16, RTX 5060 Ti 16 GB): marigold-v2-depth 1988x2880 (12.8 GiB peak allocated, 13.5 reserved)
Speed and memory
Depth, synthetic
| model | input | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
|---|---|---|---|---|---|---|---|---|---|---|
marigold-v2-depth |
994x1482 | β | OOM | 5093 | 3886 | β | 1.3x | β | 1408 / 1698 | 1498 / 1780 |
Startup and first call
| model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
|---|---|---|---|---|---|---|
marigold-v2-depth |
β | 7.80 | 6.20 | β | 5.09 | 3.89 |
torch.compile and TensorRT
| model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) |
|---|---|---|---|---|---|---|---|---|---|---|---|
marigold-v2-depth |
994x1482 | 5093 ms | deferred (build >5 min) | planned for 0.2.1 | 3886 ms | β | β | 1408 / 1698 | β | 1498 / 1780 | β |
Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).
Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.
Accuracy
On public datasets
AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | portable | kernels | note |
|---|---|---|---|---|
marigold-v2-depth |
NYUv2 test, AbsRel / delta1 | 0.0386 / 0.9840 | 0.0386 / 0.9840 | |
marigold-v2-depth |
iBims-1, AbsRel / delta1 | 0.0291 / 0.9877 | 0.0291 / 0.9877 |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 37687519, 2026-11-14.
Parity with the original code
Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.02 vs the original. GPU, bfloat16 (the default), portable and kernels: mean |diff| <= 1 % and p99 <= 5 % of the reference's median |value| (absolute error of the output depth).
Limitations
- 20 B transformer in 4 bit: 8.4 GB of weights; by default (
stream=True, CUDA) it runs in ~1-4 GB of device memory from 8.4 GB of page-locked host RAM (load fails with an error when less is free);stream=Falsekeeps it resident (about 9-11 GB of device memory, 16 GB cards: up to ~1024 px). - Diffusion model: the VAE posterior sample comes from a CPU generator seeded with
seedand differs from the original code's (unseeded, device) sample. - The enlargement to a multiple of 16 is bicubic (the original: OpenCV Lanczos); inputs with sides that are multiples of 16 are not resized.
- The output is log depth up to scale and shift (
kind = "affine_invariant_log"): align in log space, not with the other families' depth outputs. - A traced or exported module reuses the noise of its traced shape; CPU inference is possible but slow (bf16 GEMMs of a 20 B model).
Usage
from fastforks import depth
model = depth.load("marigold-v2-depth") # mode="auto": the preferred backend from recorded measurements
result = model("room.jpg")
result.depth # [H, W] float32 tensor; result.kind says what the maps mean
# models that take a camera: model("room.jpg", intrinsics=K), K [3, 3] in pixels
result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: marigold-v2-depth.
Kernels backend
depth.load("marigold-v2-depth", mode="kernels") runs our CUDA kernels (nf4). It needs a prebuilt kernel for your GPU architecture (downloaded from https://huggingface.co/fastforks/kernels on first use) and a recent NVIDIA driver; if there is none it raises BackendUnavailableError with the reason and never falls back silently. mode="auto" picks kernels when it can run and logs the choice.