multimodalart's picture
multimodalart HF Staff
Upload README.md with huggingface_hub
4e72893 verified
|
Raw History Blame Contribute Delete
3.01 kB

A newer version of the Gradio SDK is available: 6.32.0

Upgrade
metadata
title: GaussianRenderFormer
emoji: ✨
colorFrom: pink
colorTo: blue
sdk: gradio
sdk_version: 6.31.0
app_file: app.py
python_version: '3.12'
short_description: Transformer-based rendering of 3D Gaussian splats
startup_duration_timeout: 30m

GaussianRenderFormer

Transformer-based rendering of 3D Gaussian splats — pick a Gaussian Splatting scene, point an orbit camera at it, and a transformer renders the view. No rasterizer, no per-scene optimization: each Gaussian's 14 parameters become one scene token, a view decoder cross-attends over 4-px image patches, and a DPT head produces the image.

What you can do here

  • Example scenes — three synthetic splats authored for this demo (a faceted gem, a glazed vase, a ringed planet, each ≤ 20k Gaussians) rendered through the exact PLY → read_ply → normalize path the model uses in production.
  • Your own splat — upload a 3DGS .ply (raw optimizer fields: log scales, wxyz quaternions, SH colors, opacity logits). Tell the app the file's up axis and it centers, re-orients and scales the object into the model's frame. Splats larger than 20k Gaussians are pruned by an importance score; the model was trained on ~20k-Gaussian objects and denser inputs lose quality.

The camera orbits the object's center: azimuth, elevation and distance (the model was trained at distances 1.15–2.45, 45° FOV, 512 px).

Deviations from the reference pipeline

The reference load_ply prunes with gsplat-projected radii and then fine-tunes the kept Gaussians against the full splat for ~1500 iterations. A CUDA rasterizer with a JIT build does not fit this demo's runtime, so pruning here uses an analytic approximation of the same importance score (opacity × projected area estimated from pinhole geometry, × scale^0.1) without the fine-tune. Per the model card, large pruned splats can "lose coverage and turn dark" without the fine-tune — the bundled examples are ≤ 20k Gaussians and so are never pruned.

Example scenes & licensing

The three example scenes are original synthetic splats generated procedurally for this demo (no captured data), so they carry no third-party license restrictions. The model was trained on Objaverse_Splats, whose objects each keep their creator's license — that is why the demo does not bundle any of the paper's evaluation scenes.

Limitations (from the model card)

Trained on isolated single objects centered, Y-up and scaled into [-0.45, 0.45]³, at 512 px and 45° FOV; denser inputs lose quality; colors are view-independent (only each Gaussian's DC color is used); fine detail renders softer than a rasterizer.