orcaengineer/orca-vggt-depth-bf16

ORCA VGGT depth (bfloat16, depth head only)

A smaller copy of Meta's VGGT-1B-Commercial that keeps only what multi-view depth estimation needs. It runs on a 4 GB GPU. It is used by ORCA, a pipeline for mapping coral reefs from underwater survey images, to fill the holes that stereo matching leaves in each frame's depth map.

This is a derivative of VGGT, made and shared under the VGGT License. It is not made or endorsed by Meta.

What changed from VGGT-1B-Commercial

	VGGT-1B-Commercial	This model
Heads	Camera, depth, point, track.	Depth only.
Aggregator weights	float32.	bfloat16.
Depth head weights	float32.	float32, unchanged.
Parameters	1,256.5 M	941.8 M
File size	5.03 GB	About 1.95 GB
Peak GPU memory, 8 views, depth head one frame per chunk	Does not fit 4 GB.	2.71 GB on an RTX 3050 Laptop GPU.

• No weights were retrained. The depth head is bit-identical to the original.
• The depth head must stay float32: VGGT runs its heads with autocast switched off, so they see float32 inputs.
• Measured on 10 underwater frames against the original model on an H100: depth differs by a median of 2.0 to 7.6 mm and a 95th percentile of 5.9 to 18.9 mm, at 2 to 4.5 m range. That is about the same as the original model run on a CPU instead of a GPU.

Use

Needs the vggt package from https://github.com/facebookresearch/vggt and PyTorch. No Hugging Face account or token is needed to download this model.

import functools
import torch
from vggt.models.vggt import VGGT
from vggt.utils.load_fn import load_and_preprocess_images

model = VGGT.from_pretrained("orcaengineer/orca-vggt-depth-bf16")  # config.json builds the depth-only model
model.aggregator.to(torch.bfloat16)      # weights are stored bfloat16; keep them that way
model = model.to("cuda").eval()
# One frame per chunk in the depth head keeps 8 views inside 4 GB. Output is unchanged.
head = model.depth_head
head.forward = functools.partial(type(head).forward, head, frames_chunk_size=1)

images = load_and_preprocess_images(["frame_0.jpg", "frame_1.jpg", "frame_2.jpg"]).to("cuda")
with torch.no_grad(), torch.autocast("cuda", dtype=torch.bfloat16):
    pred = model(images[None])
depth = pred["depth"][0, :, :, :, 0]  # relative depth, one map per view

• The depth is relative. ORCA scales it to metres with one median ratio against stereo depth.
• bfloat16 needs a GPU of compute capability 8.0 or newer. On older GPUs or a CPU, cast the aggregator to float32 instead.

Licence

• This model is distributed under the VGGT License, v1 of 29 July 2025, in the file LICENSE, which includes Meta's Acceptable Use Policy.
• By downloading or using this model you agree to that licence and policy. If you share this model or anything made from it, you must share it under the same licence and include a copy of it (clause 1.b.i).
• See NOTICE for the attribution.

Citation

If you publish results that use this model, acknowledge VGGT (licence clause 1.b.ii) and cite:

@inproceedings{wang2025vggt,
  title={VGGT: Visual Geometry Grounded Transformer},
  author={Wang, Jianyuan and Chen, Minghao and Karaev, Nikita and Vedaldi, Andrea and Rupprecht, Christian and Novotny, David},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2025}
}

Files

• model.safetensors: the weights. Its metadata records the source model's revision and sha256.
• config.json: builds the depth-only model.
• LICENSE: the VGGT License and Acceptable Use Policy, verbatim.
• NOTICE: attribution and the list of changes.
• SHA256SUMS: a checksum for every file.