MetricStereo

Single-image stereo generation with a two-view diffusion model. Given one (left) image and a stereo baseline in metres, MetricStereo generates the matching right view directly. It uses no depth estimate, no warping and no inpainting.

Anonymous release accompanying a CVPR 2027 submission.

Code: github.com/cvpr-submission/MetricStereo

Model

  • Denoiser: two-view latent diffusion U-Net (SD-2.1 layout). Self-attention runs jointly over the tokens of both views, so the right view is generated in correspondence with the left.
  • Camera conditioning:
    • Each view's latent is concatenated with a PlΓΌcker ray map and a mask that marks the given view.
    • A 16-d rig-pose vector derived from the baseline is added to the timestep embedding.
    • A CLIP image token of the input is used as cross-attention context.
  • Normalization: cross-view synchronized normalization (SyncVN): every ResBlock's GroupNorm pools its statistics over both views.
  • Stereo-aware decoder: the SD-2 VAE decoder with a light LoRA and a cross-view attention block, so both views decode consistently.
  • Prediction: v-prediction; DPM-Solver++(2M), 25 steps uniform in log-SNR; cosine classifier-free guidance 3 β†’ 1.

Frozen inference configuration

Resolution 768 Γ— 768
Steps / sampler 25, DPM-Solver++(2M)
Guidance cosine CFG 3 β†’ 1
Precision bf16 autocast, fp32 weights
Seed image i of a benchmark uses seed i

Files

File Size SHA-256
metricstereo.safetensors 3.8 GB 071fcf5d4fc67032bf309a5713d9257078ac25dd6dc605b87d53f6b33f7ca6b8

The file holds inference weights only (unet.*, vae.* including vae.mixer.*, and schedule.*). The decoder LoRA is stored unmerged so outputs are bit-identical to the paper's pipeline on the same GPU type. The CLIP ViT-H/14 image encoder (laion2b_s32b_b79k) is fetched from open_clip on first use.

Usage

from metricstereo.pipeline import MetricStereo, read_rgb
from PIL import Image

model = MetricStereo()                                                    # loads weights/metricstereo.safetensors
right = model.generate(read_rgb("left.png"), baseline_m=0.065, seed=0)    # uint8 768x768x3
Image.fromarray(right).save("right.png")

Download the weights:

wget https://huggingface.co/cvprsubmission/metricstereo-model/resolve/main/metricstereo.safetensors -P weights/

The inference code includes:

  • a bit-exact reproduction check;
  • generation scripts for all nine benchmarks;
  • the common-protocol evaluation;
  • the per-scene baseline search of the paper's main comparison.

Results

FoundationStereo measures disparity on the generated pair in a shared frame (GT aspect ratio, short side 768). Against the ground-truth disparity on Middlebury, a pseudo-GT from the real pair on the other stereo benchmarks (their test ground truth is not public) or metric GT depth (depth benchmarks), it reports Ξ΄1, AbsRel and SAWA-H. LPIPS (vs. GT right), MEt3R and iSQoE are computed on the native pair. ↑ higher is better, ↓ lower is better.

Per-scene baseline search (main comparison of the paper): each scene's baseline is tuned against the real right view, as in StereoSpace.

Benchmark Ξ΄1 ↑ AbsRel ↓ SAWA-H ↓ LPIPS ↓ MEt3R ↓ iSQoE ↓
Booster 0.9674 0.0420 0.8263 0.2578 0.0999 0.6445
LayeredFlow 0.8798 0.0802 1.1204 0.3395 0.1382 0.6730
Middlebury 0.8807 0.0873 1.1999 0.2447 0.0958 0.6638
Spring 0.7906 0.1158 1.4121 0.1292 0.0920 0.7846

One fixed baseline per benchmark: the median ground-truth baseline on the stereo benchmarks, 10 cm on the depth benchmarks.

Benchmark Ξ΄1 ↑ AbsRel ↓ SAWA-H ↓ LPIPS ↓ MEt3R ↓ iSQoE ↓
Booster 0.9684 0.0413 0.8378 0.3097 0.0887 0.6319
LayeredFlow 0.8755 0.0814 1.1245 0.4001 0.1216 0.6565
Middlebury 0.8862 0.0895 1.2115 0.3408 0.1005 0.6740
Spring 0.7840 0.1191 1.4005 0.1394 0.0942 0.7848
KITTI 0.9617 0.0571 1.3437 – 0.0690 0.7214
Sintel 0.7019 0.2263 1.6260 – 0.1494 0.8216
ETH3D 0.9271 0.0846 1.1209 – 0.0666 0.5522
TUM 0.9109 0.0902 1.6667 – 0.0905 0.7824
NYUv2 0.9642 0.0582 1.4668 – 0.1005 0.7419

These numbers were produced on NVIDIA RTX PRO 6000 (Blackwell) GPUs, where the released code reproduces them bit-exactly (identical generated images and per-scene selections). Sampling is not bit-reproducible across GPU architectures; on an A100 the metrics change in the third decimal.

Limitations

  • Output is 768 Γ— 768.

License

CreativeML Open RAIL++-M, inherited from Stable Diffusion 2.1, on which the denoiser and VAE are based. Its use-based restrictions apply to this model and any derivatives.

Citation

@inproceedings{metricstereo2027,
  title     = {MetricStereo},
  author    = {Anonymous},
  booktitle = {Under review},
  year      = {2027}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support