MetricStereo
Single-image stereo generation with a two-view diffusion model. Given one (left) image and a stereo baseline in metres, MetricStereo generates the matching right view directly. It uses no depth estimate, no warping and no inpainting.
Anonymous release accompanying a CVPR 2027 submission.
Code: github.com/cvpr-submission/MetricStereo
Model
- Denoiser: two-view latent diffusion U-Net (SD-2.1 layout). Self-attention runs jointly over the tokens of both views, so the right view is generated in correspondence with the left.
- Camera conditioning:
- Each view's latent is concatenated with a PlΓΌcker ray map and a mask that marks the given view.
- A 16-d rig-pose vector derived from the baseline is added to the timestep embedding.
- A CLIP image token of the input is used as cross-attention context.
- Normalization: cross-view synchronized normalization (SyncVN): every ResBlock's GroupNorm pools its statistics over both views.
- Stereo-aware decoder: the SD-2 VAE decoder with a light LoRA and a cross-view attention block, so both views decode consistently.
- Prediction: v-prediction; DPM-Solver++(2M), 25 steps uniform in log-SNR; cosine classifier-free guidance 3 β 1.
Frozen inference configuration
| Resolution | 768 Γ 768 |
| Steps / sampler | 25, DPM-Solver++(2M) |
| Guidance | cosine CFG 3 β 1 |
| Precision | bf16 autocast, fp32 weights |
| Seed | image i of a benchmark uses seed i |
Files
| File | Size | SHA-256 |
|---|---|---|
metricstereo.safetensors |
3.8 GB | 071fcf5d4fc67032bf309a5713d9257078ac25dd6dc605b87d53f6b33f7ca6b8 |
The file holds inference weights only (unet.*, vae.* including vae.mixer.*, and schedule.*). The
decoder LoRA is stored unmerged so outputs are bit-identical to the paper's pipeline on the same GPU type. The CLIP ViT-H/14 image
encoder (laion2b_s32b_b79k) is fetched from open_clip on first use.
Usage
from metricstereo.pipeline import MetricStereo, read_rgb
from PIL import Image
model = MetricStereo() # loads weights/metricstereo.safetensors
right = model.generate(read_rgb("left.png"), baseline_m=0.065, seed=0) # uint8 768x768x3
Image.fromarray(right).save("right.png")
Download the weights:
wget https://huggingface.co/cvprsubmission/metricstereo-model/resolve/main/metricstereo.safetensors -P weights/
The inference code includes:
- a bit-exact reproduction check;
- generation scripts for all nine benchmarks;
- the common-protocol evaluation;
- the per-scene baseline search of the paper's main comparison.
Results
FoundationStereo measures disparity on the generated pair in a shared frame (GT aspect ratio, short side 768). Against the ground-truth disparity on Middlebury, a pseudo-GT from the real pair on the other stereo benchmarks (their test ground truth is not public) or metric GT depth (depth benchmarks), it reports Ξ΄1, AbsRel and SAWA-H. LPIPS (vs. GT right), MEt3R and iSQoE are computed on the native pair. β higher is better, β lower is better.
Per-scene baseline search (main comparison of the paper): each scene's baseline is tuned against the real right view, as in StereoSpace.
| Benchmark | Ξ΄1 β | AbsRel β | SAWA-H β | LPIPS β | MEt3R β | iSQoE β |
|---|---|---|---|---|---|---|
| Booster | 0.9674 | 0.0420 | 0.8263 | 0.2578 | 0.0999 | 0.6445 |
| LayeredFlow | 0.8798 | 0.0802 | 1.1204 | 0.3395 | 0.1382 | 0.6730 |
| Middlebury | 0.8807 | 0.0873 | 1.1999 | 0.2447 | 0.0958 | 0.6638 |
| Spring | 0.7906 | 0.1158 | 1.4121 | 0.1292 | 0.0920 | 0.7846 |
One fixed baseline per benchmark: the median ground-truth baseline on the stereo benchmarks, 10 cm on the depth benchmarks.
| Benchmark | Ξ΄1 β | AbsRel β | SAWA-H β | LPIPS β | MEt3R β | iSQoE β |
|---|---|---|---|---|---|---|
| Booster | 0.9684 | 0.0413 | 0.8378 | 0.3097 | 0.0887 | 0.6319 |
| LayeredFlow | 0.8755 | 0.0814 | 1.1245 | 0.4001 | 0.1216 | 0.6565 |
| Middlebury | 0.8862 | 0.0895 | 1.2115 | 0.3408 | 0.1005 | 0.6740 |
| Spring | 0.7840 | 0.1191 | 1.4005 | 0.1394 | 0.0942 | 0.7848 |
| KITTI | 0.9617 | 0.0571 | 1.3437 | β | 0.0690 | 0.7214 |
| Sintel | 0.7019 | 0.2263 | 1.6260 | β | 0.1494 | 0.8216 |
| ETH3D | 0.9271 | 0.0846 | 1.1209 | β | 0.0666 | 0.5522 |
| TUM | 0.9109 | 0.0902 | 1.6667 | β | 0.0905 | 0.7824 |
| NYUv2 | 0.9642 | 0.0582 | 1.4668 | β | 0.1005 | 0.7419 |
These numbers were produced on NVIDIA RTX PRO 6000 (Blackwell) GPUs, where the released code reproduces them bit-exactly (identical generated images and per-scene selections). Sampling is not bit-reproducible across GPU architectures; on an A100 the metrics change in the third decimal.
Limitations
- Output is 768 Γ 768.
License
CreativeML Open RAIL++-M, inherited from Stable Diffusion 2.1, on which the denoiser and VAE are based. Its use-based restrictions apply to this model and any derivatives.
Citation
@inproceedings{metricstereo2027,
title = {MetricStereo},
author = {Anonymous},
booktitle = {Under review},
year = {2027}
}