File size: 3,733 Bytes
7ced6cb bfca7c8 7ced6cb 7888994 7ced6cb 7888994 7ced6cb 7888994 7ced6cb bfca7c8 7ced6cb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | ---
license: other
license_name: fair-noncommercial-research-license
license_link: https://huggingface.co/Jingong/GradSplat/blob/main/LICENSE
library_name: gradsplat
pipeline_tag: image-to-3d
tags:
- gaussian-splatting
- novel-view-synthesis
- 3d-reconstruction
- feed-forward
---
# GradSplat
**GradSplat** reconstructs a scene as 3D Gaussians from a few images in one forward pass, and refines its own prediction
by learned gradient steps on the input views inside that pass, first on opacity and color and then on the shape of the
Gaussians. No target view is used, so the steps run unchanged at inference.
[Paper](https://arxiv.org/abs/2610.38951) · [Code](https://github.com/Jingong-Chen/GradSplat) · [Project page](https://jingong-chen.github.io/GradSplat/)
The repository holds the two sets of weights of the paper:
| File | Training data | Load with |
|:---|:---|:---|
| `model.safetensors` | RealEstate10K | `GradSplat.from_pretrained("Jingong/GradSplat")` |
| `model.dl3dv.safetensors` | RealEstate10K, then 12,000 steps on DL3DV | `GradSplat.from_pretrained("Jingong/GradSplat", variant="dl3dv")` |
## Usage
```bash
git clone https://github.com/Jingong-Chen/GradSplat.git && cd GradSplat && pip install -e .
```
```python
from gradsplat import GradSplat
from gradsplat.io import load_views
model = GradSplat.from_pretrained("Jingong/GradSplat").to("cuda").eval()
views = load_views("examples/re10k_dining_room/images", cameras="examples/re10k_dining_room/cameras.json")
out = model.infer(views.images, views.c2w, views.K) # without cameras: model.infer(views.images)
gaussians = out["gaussians"] # one Gaussian per input pixel
```
## Model
- 1.22 B parameters: a VGGT-Ω backbone, a camera head, a dense head with a Gaussian branch, a shape head and two
learned updates (three appearance steps and five shape steps).
- Input: 2 or more overlapping views of 224 × 224 pixels (center-cropped), with or without their cameras.
- Output: one 3D Gaussian per input pixel (center, scale, rotation, opacity and spherical harmonics of degree 1), and
the predicted cameras.
- Training: initialized from VGGT-Ω and trained for 178,000 steps in four stages on the training split of
RealEstate10K, with 2 to 12 input views per sample. `model.dl3dv.safetensors` continues for 12,000 steps on DL3DV.
## Results (from the paper)
| | Narrow protocol (7,128 scenes) | Wide protocol (1,600 scenes) |
|:---|:---:|:---:|
| PSNR / SSIM / LPIPS, with input poses | 33.683 / 0.9593 / 0.0640 | 26.755 / 0.8993 / 0.1301 |
| PSNR / SSIM / LPIPS, without input poses | 26.914 / 0.8863 / 0.1053 | 20.901 / 0.6945 / 0.2426 |
RealEstate10K with six input and eight target views per scene at 224 × 224 pixels, with `model.safetensors`. On
DL3DV (139 scenes of the index released with ReSplat) and ScanNet++ without fine-tuning: DL3DV 24.787 / 0.8448 / 0.1919
and ScanNet++ 21.609 / 0.7540 / 0.3184. With `model.dl3dv.safetensors`: DL3DV 26.104 / 0.8802 / 0.1505.
## License
The weights are trained from the VGGT-Ω weights and are released under the FAIR Noncommercial Research License of
VGGT-Ω ([LICENSE](LICENSE)), for noncommercial research use. The [code](https://github.com/Jingong-Chen/GradSplat) is
released under the MIT License, except for the code of VGGT-Ω and the two files that adapt it.
## Citation
```bibtex
@article{chen2026gradsplat,
title = {GradSplat: Gradients Inside the Forward Pass for Feed-Forward 3D Gaussian Splatting},
author = {Chen, Jingong and Jiang, Linlian and Li, Guangyuan and Zuo, Xinxin and
Wang, Yang and Pack, Chulwoo and Gao, Kyle and Won, Kwanghee},
journal = {arXiv preprint arXiv:2610.38951},
year = {2026}
}
```
|