Marigold V2 (fastforks)

Marigold V2 (Pavlovic, Wandel, Obukhov et al. 2026): affine-invariant log depth by one rectified-flow step of Qwen-Image-Edit-2509's 20 B transformer (NF4 4-bit, as the authors run it) with a rank-128 depth LoRA and a fine-tuned Qwen VAE decoder (depth/Log-stage2).

This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/huawei-bayerlab/marigold-v2 at main.

Author weight repos: https://huggingface.co/fastforks/marigold-v2-depth-nf4.

Task: depth. Tier: experimental. Outputs: depth. Backends: portable, kernels.

Input: one RGB image. Output: a depth or inverse-depth map. kind (a card field, result.kind) says what the numbers mean: metric = metres; scale_invariant, affine_invariant, affine_invariant_log (log depth) and affine_invariant_inverse are relative (unknown scale or shift, not metres). Metric models that use the focal length take model(image, intrinsics=K) (K [3, 3], pixels of the input): metric3d-vit-large and da3metric-large need it for correct metres (da3metric-large: metres = output x focal / 300; without intrinsics= a 300 px focal length is assumed, wrong for a real camera). unidepth-v2-vitl14, unik3d-vitl, depth-pro and MoGe estimate the camera themselves (given intrinsics= replace fx); zoedepth-nk and the relative models (da3mono-large too) take none. Normals-only models (kind none) return result.normals [B, H, W, 3] (unit vectors, camera frame: x right, y down, z forward).

Variants

Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.

variant weights (author link) SHA-256 weights licence commercial use
marigold-v2-depth@1 marigold-v2-depth-log-stage2-nf4.safetensors 4b4aee27acdb Apache-2.0 yes

Licence

Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Licence files: LICENSE, NOTICE.

Training data: Qwen-Image-Edit-2509 (Alibaba, Apache-2.0) fine-tuned (LoRA + VAE decoder) on synthetic depth data per the Marigold V2 paper (arXiv 2609.08084); the training data terms are not checked.

  • Licence note: The converted weight file (NF4 transformer, LoRA, folded constants, 2-D VAE) is hosted by fastforks under the originals' Apache-2.0 licence; NOTICE keeps both attributions and lists the changes.

Licence evidence: https://huggingface.co/huawei-bayerlab/marigold-v2/blob/main/README.md (license: apache-2.0; base model Qwen/Qwen-Image-Edit-2509)

Input size

any H x W >= 1, resized or padded inside the model Tested up to (portable, fp16, RTX 5060 Ti 16 GB): marigold-v2-depth 1988x2880 (12.8 GiB peak allocated, 13.5 reserved)

Speed and memory

Depth, synthetic

model input time (ms): original fp32 original portable kernels kernels speed-up: vs original vs portable peak memory (MiB, allocated / reserved): original portable kernels
marigold-v2-depth 994x1482 – OOM 5093 3886 – 1.3x – 1408 / 1698 1498 / 1780

Startup and first call

model load (s): original portable kernels first call (s): original portable kernels
marigold-v2-depth – 7.80 6.20 – 5.09 3.89

torch.compile and TensorRT

model input time: portable compile TensorRT fp16 kernels preparation: compile (cold first call) ONNX + engine build peak MiB (alloc / reserved): portable compile kernels TensorRT (engine + torch)
marigold-v2-depth 994x1482 5093 ms deferred (build >5 min) planned for 0.2.1 3886 ms – – 1408 / 1698 – 1498 / 1780 –

Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).

Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.

Accuracy

On public datasets

AbsRel / delta1 after a least-squares scale + shift fit (in disparity space for inverse-depth models), mean over images. Original = the original code named under Details, same checkpoint and protocol.

variant dataset portable kernels note
marigold-v2-depth NYUv2 test, AbsRel / delta1 0.0386 / 0.9840 0.0386 / 0.9840
marigold-v2-depth iBims-1, AbsRel / delta1 0.0291 / 0.9877 0.0291 / 0.9877

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 37687519, 2026-11-14.

Parity with the original code

Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.02 vs the original. GPU, bfloat16 (the default), portable and kernels: mean |diff| <= 1 % and p99 <= 5 % of the reference's median |value| (absolute error of the output depth).

Limitations

  • 20 B transformer in 4 bit: 8.4 GB of weights; by default (stream=True, CUDA) it runs in ~1-4 GB of device memory from 8.4 GB of page-locked host RAM (load fails with an error when less is free); stream=False keeps it resident (about 9-11 GB of device memory, 16 GB cards: up to ~1024 px).
  • Diffusion model: the VAE posterior sample comes from a CPU generator seeded with seed and differs from the original code's (unseeded, device) sample.
  • The enlargement to a multiple of 16 is bicubic (the original: OpenCV Lanczos); inputs with sides that are multiples of 16 are not resized.
  • The output is log depth up to scale and shift (kind = "affine_invariant_log"): align in log space, not with the other families' depth outputs.
  • A traced or exported module reuses the noise of its traced shape; CPU inference is possible but slow (bf16 GEMMs of a 20 B model).

Usage

from fastforks import depth

model = depth.load("marigold-v2-depth")  # mode="auto": the preferred backend from recorded measurements
result = model("room.jpg")
result.depth  # [H, W] float32 tensor; result.kind says what the maps mean
# models that take a camera: model("room.jpg", intrinsics=K), K [3, 3] in pixels
result.backend  # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')

Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: marigold-v2-depth.

Kernels backend

depth.load("marigold-v2-depth", mode="kernels") runs our CUDA kernels (nf4). It needs a prebuilt kernel for your GPU architecture (downloaded from https://huggingface.co/fastforks/kernels on first use) and a recent NVIDIA driver; if there is none it raises BackendUnavailableError with the reason and never falls back silently. mode="auto" picks kernels when it can run and logs the choice.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Collection including fastforks/marigold-v2