Spaces:
Running on Zero
Running on Zero
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,13 +1,65 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
sdk_version: 6.31.0
|
| 8 |
-
python_version: '3.12'
|
| 9 |
app_file: app.py
|
| 10 |
-
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: GaussianRenderFormer
|
| 3 |
+
emoji: ✨
|
| 4 |
+
colorFrom: pink
|
| 5 |
+
colorTo: blue
|
| 6 |
sdk: gradio
|
| 7 |
sdk_version: 6.31.0
|
|
|
|
| 8 |
app_file: app.py
|
| 9 |
+
python_version: "3.12"
|
| 10 |
+
short_description: Transformer-based rendering of 3D Gaussian splats
|
| 11 |
+
startup_duration_timeout: 30m
|
| 12 |
---
|
| 13 |
|
| 14 |
+
# GaussianRenderFormer
|
| 15 |
+
|
| 16 |
+
**Transformer-based rendering of 3D Gaussian splats** — pick a Gaussian Splatting
|
| 17 |
+
scene, point an orbit camera at it, and a transformer renders the view. No
|
| 18 |
+
rasterizer, no per-scene optimization: each Gaussian's 14 parameters become one
|
| 19 |
+
scene token, a view decoder cross-attends over 4-px image patches, and a DPT
|
| 20 |
+
head produces the image.
|
| 21 |
+
|
| 22 |
+
- Paper: [arXiv:2611.05593](https://arxiv.org/abs/2611.05593) — Valencia Levy & Benaim, 2026
|
| 23 |
+
- Code: [github.com/SVLwoof/gaussianformer](https://github.com/SVLwoof/gaussianformer) (MIT)
|
| 24 |
+
- Checkpoint: [shahafvl/gaussianformer](https://huggingface.co/shahafvl/gaussianformer) (MIT)
|
| 25 |
+
|
| 26 |
+
## What you can do here
|
| 27 |
+
|
| 28 |
+
- **Example scenes** — three synthetic splats authored for this demo (a faceted
|
| 29 |
+
gem, a glazed vase, a ringed planet, each ≤ 20k Gaussians) rendered through
|
| 30 |
+
the exact PLY → `read_ply` → `normalize` path the model uses in production.
|
| 31 |
+
- **Your own splat** — upload a 3DGS `.ply` (raw optimizer fields: log scales,
|
| 32 |
+
wxyz quaternions, SH colors, opacity logits). Tell the app the file's up axis
|
| 33 |
+
and it centers, re-orients and scales the object into the model's frame.
|
| 34 |
+
Splats larger than 20k Gaussians are pruned by an importance score; the model
|
| 35 |
+
was trained on ~20k-Gaussian objects and denser inputs lose quality.
|
| 36 |
+
|
| 37 |
+
The camera orbits the object's center: azimuth, elevation and distance
|
| 38 |
+
(the model was trained at distances 1.15–2.45, 45° FOV, 512 px).
|
| 39 |
+
|
| 40 |
+
## Deviations from the reference pipeline
|
| 41 |
+
|
| 42 |
+
The reference `load_ply` prunes with gsplat-projected radii and then fine-tunes
|
| 43 |
+
the kept Gaussians against the full splat for ~1500 iterations. A CUDA
|
| 44 |
+
rasterizer with a JIT build does not fit this demo's runtime, so pruning here
|
| 45 |
+
uses an analytic approximation of the same importance score (opacity ×
|
| 46 |
+
projected area estimated from pinhole geometry, × scale^0.1) without the
|
| 47 |
+
fine-tune. Per the model card, large pruned splats can "lose coverage and turn
|
| 48 |
+
dark" without the fine-tune — the bundled examples are ≤ 20k Gaussians and so
|
| 49 |
+
are never pruned.
|
| 50 |
+
|
| 51 |
+
## Example scenes & licensing
|
| 52 |
+
|
| 53 |
+
The three example scenes are original synthetic splats generated procedurally
|
| 54 |
+
for this demo (no captured data), so they carry no third-party license
|
| 55 |
+
restrictions. The model was trained on
|
| 56 |
+
[Objaverse_Splats](https://huggingface.co/datasets/ShapeSplats/Objaverse_Splats),
|
| 57 |
+
whose objects each keep their creator's license — that is why the demo does not
|
| 58 |
+
bundle any of the paper's evaluation scenes.
|
| 59 |
+
|
| 60 |
+
## Limitations (from the model card)
|
| 61 |
+
|
| 62 |
+
Trained on isolated single objects centered, Y-up and scaled into
|
| 63 |
+
[-0.45, 0.45]³, at 512 px and 45° FOV; denser inputs lose quality; colors are
|
| 64 |
+
view-independent (only each Gaussian's DC color is used); fine detail renders
|
| 65 |
+
softer than a rasterizer.
|