Lux3D VGGT-1B Canonical Weights

This repository contains the canonicalized safetensors export of the VGGT-1B checkpoint used by the Lux3D runtime.

Model Details

• Model family: vggt
• Canonical file: model.safetensors
• Tensor count: 1797
• Parameter storage: F32
• Primary runtime target: Lux3D runtime
• Typical output: confidence-filtered point cloud export (.ply)
• Runtime outputs: world points, point confidence, depth, depth confidence, camera extrinsics/intrinsics, and camera pose encoding

VGGT-1B is a multi-view visual geometry transformer. This inference slice includes the DINOv2 ViT-L/14 patch backbone, 24 frame + 24 global aggregator blocks, the 4-iteration camera head, and the DPT point/depth heads. The query-point track head is intentionally excluded.

Included Files

• model.safetensors
• resolved_config.json
• manifest.json
• checksums.json

The package is about 5.0 GB (model.safetensors is 5,026,367,224 bytes).

Usage With Lux3D

The Lux3D CLI can validate and use the package once it is installed into your local model asset layout.

cargo run -p lux3d-cli -- inspect --repo-root <runtime-root> vggt
cargo run -p lux3d-cli -- run vggt --model-path <canonical-vggt-dir> --source <input-sequence> --output <output-file.ply>

The canonical package can also be auto-downloaded from this repository:

cargo run -p lux3d-cli -- run vggt --source <input-sequence> --output <output-file.ply>

Point export keeps vertices with world_points_conf > 1.0 by default. Override the threshold with --conf-threshold. Input can be a directory of RGB frames or a video file (.mp4, .mov, .avi, .mkv).

Provenance

• Upstream source model: facebook/VGGT-1B
• Canonicalization flow: tools/python_baseline/normalize_weights.py

The exact source checksum set is recorded in manifest.json. The integrity of the uploaded package is recorded in checksums.json. Canonical tensor names match the upstream checkpoint; no key remapping is applied.

Intended Use

• multi-view 3D reconstruction with Lux3D
• point cloud generation from image sequences or video
• camera pose and depth inspection through the Lux3D runtime API
• reproducible runtime validation against the upstream PyTorch reference

Validation

The Lux3D runtime was validated against the upstream PyTorch reference on CPU, CUDA, and Vulkan. CUDA parity on the 3-frame golden sample is within FP32 tolerance for patch tokens, aggregator layers, camera pose, depth, world points, confidence, and camera extrinsics/intrinsics.

Reference inference timings for 3 frames at 518x518 on an NVIDIA GeForce RTX 3060 12 GiB:

Backend	Time
CUDA	6.95 s
Vulkan	15.54 s
CPU	107.66 s

Peak CUDA memory was 9.4 GiB for 3 frames. On an NVIDIA L4 23 GiB, the peak was 7.98 GiB at 3 frames and 8.30 GiB at 12 frames.

Limitations

• The upstream VGGT-1B checkpoint is non-commercial (CC-BY-NC-4.0).
• The query-point track head is not implemented in this inference slice; its weights are never loaded.
• Metal inference is not tested in this package; CPU, CUDA, and Vulkan are the verified backends.
• This package contains canonical runtime artifacts only, not the upstream training or demo stack.
• The Lux3D contract JSON is shipped with the runtime, not inside this weight package.
• Inference is F32-only in the current Lux3D runtime.

License

• Upstream model page: facebook/VGGT-1B
• Upstream license: CC-BY-NC-4.0 (non-commercial)

Commercial use requires the separately distributed facebook/VGGT-1B-Commercial checkpoint. Review the upstream terms before redistribution or commercial use.

Citation

@article{wang2025vggt,
  title={VGGT: Visual Geometry Grounded Transformer},
  author={Wang, Jianyuan and Chen, Minghao and Karaev, Nikita and Vedaldi, Andrea and Rupprecht, Christian and Novotny, David},
  journal={arXiv preprint arXiv:2503.11651},
  year={2025}
}