File size: 3,010 Bytes
655c00b
4e72893
 
 
 
655c00b
 
 
4e72893
 
 
655c00b
 
4e72893
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
---
title: GaussianRenderFormer
emoji: ✨
colorFrom: pink
colorTo: blue
sdk: gradio
sdk_version: 6.31.0
app_file: app.py
python_version: "3.12"
short_description: Transformer-based rendering of 3D Gaussian splats
startup_duration_timeout: 30m
---

# GaussianRenderFormer

**Transformer-based rendering of 3D Gaussian splats** — pick a Gaussian Splatting
scene, point an orbit camera at it, and a transformer renders the view. No
rasterizer, no per-scene optimization: each Gaussian's 14 parameters become one
scene token, a view decoder cross-attends over 4-px image patches, and a DPT
head produces the image.

- Paper: [arXiv:2611.05593](https://arxiv.org/abs/2611.05593) — Valencia Levy & Benaim, 2026
- Code: [github.com/SVLwoof/gaussianformer](https://github.com/SVLwoof/gaussianformer) (MIT)
- Checkpoint: [shahafvl/gaussianformer](https://huggingface.co/shahafvl/gaussianformer) (MIT)

## What you can do here

- **Example scenes** — three synthetic splats authored for this demo (a faceted
  gem, a glazed vase, a ringed planet, each ≤ 20k Gaussians) rendered through
  the exact PLY → `read_ply` → `normalize` path the model uses in production.
- **Your own splat** — upload a 3DGS `.ply` (raw optimizer fields: log scales,
  wxyz quaternions, SH colors, opacity logits). Tell the app the file's up axis
  and it centers, re-orients and scales the object into the model's frame.
  Splats larger than 20k Gaussians are pruned by an importance score; the model
  was trained on ~20k-Gaussian objects and denser inputs lose quality.

The camera orbits the object's center: azimuth, elevation and distance
(the model was trained at distances 1.15–2.45, 45° FOV, 512 px).

## Deviations from the reference pipeline

The reference `load_ply` prunes with gsplat-projected radii and then fine-tunes
the kept Gaussians against the full splat for ~1500 iterations. A CUDA
rasterizer with a JIT build does not fit this demo's runtime, so pruning here
uses an analytic approximation of the same importance score (opacity ×
projected area estimated from pinhole geometry, × scale^0.1) without the
fine-tune. Per the model card, large pruned splats can "lose coverage and turn
dark" without the fine-tune — the bundled examples are ≤ 20k Gaussians and so
are never pruned.

## Example scenes & licensing

The three example scenes are original synthetic splats generated procedurally
for this demo (no captured data), so they carry no third-party license
restrictions. The model was trained on
[Objaverse_Splats](https://huggingface.co/datasets/ShapeSplats/Objaverse_Splats),
whose objects each keep their creator's license — that is why the demo does not
bundle any of the paper's evaluation scenes.

## Limitations (from the model card)

Trained on isolated single objects centered, Y-up and scaled into
[-0.45, 0.45]³, at 512 px and 45° FOV; denser inputs lose quality; colors are
view-independent (only each Gaussian's DC color is used); fine detail renders
softer than a rasterizer.