multimodalart HF Staff commited on
Commit
4e72893
·
verified ·
1 Parent(s): 5d6ef79

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +59 -7
README.md CHANGED
@@ -1,13 +1,65 @@
1
  ---
2
- title: Gaussianrenderformer
3
- emoji: 👁
4
- colorFrom: yellow
5
- colorTo: green
6
  sdk: gradio
7
  sdk_version: 6.31.0
8
- python_version: '3.12'
9
  app_file: app.py
10
- pinned: false
 
 
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: GaussianRenderFormer
3
+ emoji: ✨
4
+ colorFrom: pink
5
+ colorTo: blue
6
  sdk: gradio
7
  sdk_version: 6.31.0
 
8
  app_file: app.py
9
+ python_version: "3.12"
10
+ short_description: Transformer-based rendering of 3D Gaussian splats
11
+ startup_duration_timeout: 30m
12
  ---
13
 
14
+ # GaussianRenderFormer
15
+
16
+ **Transformer-based rendering of 3D Gaussian splats** — pick a Gaussian Splatting
17
+ scene, point an orbit camera at it, and a transformer renders the view. No
18
+ rasterizer, no per-scene optimization: each Gaussian's 14 parameters become one
19
+ scene token, a view decoder cross-attends over 4-px image patches, and a DPT
20
+ head produces the image.
21
+
22
+ - Paper: [arXiv:2611.05593](https://arxiv.org/abs/2611.05593) — Valencia Levy & Benaim, 2026
23
+ - Code: [github.com/SVLwoof/gaussianformer](https://github.com/SVLwoof/gaussianformer) (MIT)
24
+ - Checkpoint: [shahafvl/gaussianformer](https://huggingface.co/shahafvl/gaussianformer) (MIT)
25
+
26
+ ## What you can do here
27
+
28
+ - **Example scenes** — three synthetic splats authored for this demo (a faceted
29
+ gem, a glazed vase, a ringed planet, each ≤ 20k Gaussians) rendered through
30
+ the exact PLY → `read_ply` → `normalize` path the model uses in production.
31
+ - **Your own splat** — upload a 3DGS `.ply` (raw optimizer fields: log scales,
32
+ wxyz quaternions, SH colors, opacity logits). Tell the app the file's up axis
33
+ and it centers, re-orients and scales the object into the model's frame.
34
+ Splats larger than 20k Gaussians are pruned by an importance score; the model
35
+ was trained on ~20k-Gaussian objects and denser inputs lose quality.
36
+
37
+ The camera orbits the object's center: azimuth, elevation and distance
38
+ (the model was trained at distances 1.15–2.45, 45° FOV, 512 px).
39
+
40
+ ## Deviations from the reference pipeline
41
+
42
+ The reference `load_ply` prunes with gsplat-projected radii and then fine-tunes
43
+ the kept Gaussians against the full splat for ~1500 iterations. A CUDA
44
+ rasterizer with a JIT build does not fit this demo's runtime, so pruning here
45
+ uses an analytic approximation of the same importance score (opacity ×
46
+ projected area estimated from pinhole geometry, × scale^0.1) without the
47
+ fine-tune. Per the model card, large pruned splats can "lose coverage and turn
48
+ dark" without the fine-tune — the bundled examples are ≤ 20k Gaussians and so
49
+ are never pruned.
50
+
51
+ ## Example scenes & licensing
52
+
53
+ The three example scenes are original synthetic splats generated procedurally
54
+ for this demo (no captured data), so they carry no third-party license
55
+ restrictions. The model was trained on
56
+ [Objaverse_Splats](https://huggingface.co/datasets/ShapeSplats/Objaverse_Splats),
57
+ whose objects each keep their creator's license — that is why the demo does not
58
+ bundle any of the paper's evaluation scenes.
59
+
60
+ ## Limitations (from the model card)
61
+
62
+ Trained on isolated single objects centered, Y-up and scaled into
63
+ [-0.45, 0.45]³, at 512 px and 45° FOV; denser inputs lose quality; colors are
64
+ view-independent (only each Gaussian's DC color is used); fine detail renders
65
+ softer than a rasterizer.