Spaces:
Running on Zero
Download README.md from ritianyu/SplatAnything: direct link, hf CLI and curl.
- Browser
- Download file 16.1 kB
-
https://huggingface.co/spaces/ritianyu/SplatAnything/resolve/main/README.md
- Command line
-
hf download hf://spaces/ritianyu/SplatAnything/README.md
-
curl -L -o README.md https://huggingface.co/spaces/ritianyu/SplatAnything/resolve/main/README.md
A newer version of the Gradio SDK is available: 6.32.0
title: SplatAnything (InfiniSplat v3)
emoji: 🌐
colorFrom: blue
colorTo: gray
sdk: gradio
sdk_version: 6.5.1
python_version: '3.10'
app_file: space_app.py
pinned: false
SplatAnything
Gaussian Spacing Normalization for Feed-Forward 3DGS across Arbitrary Scenes
Upload one image or several overlapping images of one static scene, extract images from a video, or upload one full 360° panorama.
Select RGB-only to predict cameras, or Pose-conditioned to supply camera poses
and intrinsics in prior_camera.json. Street, Rig-Recon and Aerial default to 504 × 504; Object, Indoor and Pan-Recon default to 756 × 756. Width and height presets are 504, 756, 896 and 1022; you can also type each dimension manually (positive multiples of 14).
Single-image RGB-only reconstruction is supported; additional overlapping views provide more coverage. Uploading your own images resets the resolution to at most 896 × 896. Each dimension is capped by the smallest corresponding dimension in the uploaded set, then rounded down to a multiple of 14 (for example, 800 × 600 becomes 798 × 588). Adding or deleting images recomputes these defaults; clearing the upload restores 896 × 896. Captured camera photos and extracted video frames follow the same rule. Original files are preserved, and you can still edit width and height manually.
The app renders native 3D Gaussian splats in a PlayCanvas viewer. It provides Gaussian PLY, standalone HTML, and a complete ZIP containing point clouds, cameras, processed RGB, predicted depth, and the effective configuration. Result links expire after 24 hours. Anyone given a result link can access that result until it expires; download it to keep a copy.
Run locally
Install a CUDA-compatible PyTorch build first, then:
python -m pip install -r requirements.txt
npm ci
python app.py --host 0.0.0.0 --port 8081
# Open http://127.0.0.1:8081
You can also run python space_app.py; it installs the pinned viewer package
automatically and listens on 0.0.0.0:7860. This repository contains the app, inference code, examples, and viewer
dependencies. It runs independently of the GitHub checkout.
Checkpoints
Configure the server with complete student and WorldMirror-2 geometry weights. Local defaults are:
checkpoints/splatanything.ckpt
checkpoints/worldmirror2.safetensors
checkpoints/skyseg.onnx # optional, for sky removal
Use SPLATANYTHING_CKPT, WORLDMIRROR2_CKPT, and SPLATANYTHING_SKY_CKPT
to select other local files. The local preparation can link to existing weights;
the external links and checkpoints are ignored by Git and are not portable uploads.
For a hosted Space, upload the two inference checkpoints to a Hugging Face model repository, then configure these Space Variables:
| Variable | Value |
|---|---|
SPLATANYTHING_MODEL_REPO |
Your model repository ID (owner/model) |
SPLATANYTHING_MODEL_REVISION |
Model commit or revision (default main) |
SPLATANYTHING_MODEL_FILE |
Student filename (default splatanything.ckpt) |
SPLATANYTHING_GEOMETRY_FILE |
Geometry filename (default worldmirror2.safetensors) |
For a private model repository, add HF_TOKEN as a Space Secret with read
access. Tokens stay on the server. Sky removal uses skyseg.onnx, downloaded and
cached on first use from the model repository (default ritianyu/SplatAnything).
SPLATANYTHING_SKY_REPO, SPLATANYTHING_SKY_REVISION, and SPLATANYTHING_SKY_FILE
can select a separate sky model; SPLATANYTHING_SKY_CKPT overrides this with a
local ONNX file. The CPU session is reused. Sky segmentation runs before the
ZeroGPU reservation, with its time recorded as sky_and_input_preprocessing.
Street and Rig-Recon examples enable Remove sky by default; other categories
start with it disabled. You can change the checkbox before reconstruction.
The shared configs/inference.yaml uses a default voxel size of 0.0018 and a 99.5 scale percentile with the supplied fine-tuned checkpoint. Both camera-mode flags are selected
together by the UI. CLI examples use python inference.py --example object/1 --mode pose;
use --list_examples to inspect the compact examples included in this bundle.
Hugging Face runtime
This folder is self-contained: it includes the inference source, Gradio UI, portable configuration, small examples, and the npm lockfile. It does not depend on a sibling GitHub checkout, private filesystem paths, or startup Git clones.
- Push this folder to a new Gradio Space after reviewing it.
- Select GPU hardware and configure the model variables above.
- For ZeroGPU, set
SPLATANYTHING_ZEROGPU=1and select ZeroGPU hardware. Itsspaces.GPUadapter reserves the GPU only for prediction; model placement happens at startup and PLY/HTML/ZIP exports run afterward on the CPU. - For a dedicated GPU, leave
SPLATANYTHING_ZEROGPU=0(the default).
Max images selects the first N input frames; 0 or an empty value uses all frames. User uploads, video frames and camera captures reset it to 0. Video extraction samples 2–32 frames.
SPLATANYTHING_DEVICE=cpu is available for small local smoke tests.
SPLATANYTHING_CONFIG selects another complete inference YAML.
GPU usage is subject to the hardware and quota settings of the hosted Space.
The original example images live in the dataset
ritianyu/SplatAnything_Demo,
under examples/CATEGORY/SCENE/. Set SPLATANYTHING_EXAMPLES_REPO=ritianyu/SplatAnything_Demo
and SPLATANYTHING_EXAMPLES_REVISION to a dataset commit. space_app.py downloads
the original images and camera files to a local cache before starting the studio.
This keeps the Space source below its repository storage limit while retaining
all original views and image quality. Local runs use the existing examples/
folder; SPLATANYTHING_EXAMPLES_DIR or --examples_dir can select another folder.
Input cameras
Camera IDs must match the image filename stems. Supply OpenCV 4 × 4 camera-to-world poses and 3 × 3 intrinsics in original-image pixels. The app updates intrinsics during cropping and resizing. See prepare_data.md for camera conversion and validation.
The scene library has six categories: Object, Indoor, Street, Aerial, Rig-Recon, and Pan-Recon. Examples are loaded directly from examples/CATEGORY/SCENE/images/ and the matching prior_camera.json, when available. Indoor scenes use RGB-only mode. Street, Rig-Recon and Aerial scenes use 504 × 504; Object, Indoor and Pan-Recon use 756 × 756. The studio displays small previews for faster loading; Max images controls how many original views are reconstructed. Selecting a card loads the scene; reconstruction starts when you click the button.
The studio opens with no selected data and a white viewer with an animated SplatAnything infinity mark. Clear resets the current example, uploads, camera captures, camera JSON and result preview for your session, restoring the idle animation. The animation uses local SVG/CSS, has no visible text, and respects reduced-motion preferences. Selecting an example loads its images and settings; the right-hand viewer shows a still captured from the matching HTML’s initial Gaussian camera. Click Explore 3D to download and open the precomputed scene from SplatAnything_Demo. HTML files are fetched only on this click, verified against the pinned dataset revision, and cached for reuse. Click Reconstruct scene to run new inference with the current inputs and settings. Example covers use square crops for Object, Indoor, Street, Aerial, and Rig-Recon; Pan-Recon keeps the original panorama aspect ratio. These covers affect previews only: reconstruction uses the complete original images. Captions sit below the image.
Video frame extraction is a CPU operation and resets the mode to RGB-only.
Browser camera
In Images, click Open camera and allow camera access. Capture one image or several overlapping views of one static scene; each snapshot is added to the input preview. The first snapshot starts a new RGB-only scene, replacing any previously selected example or uploads. Later snapshots append to that camera scene. Use New set to start over and Close camera to release the camera; closing it keeps the captured photos available for reconstruction. Switching input tabs or selecting an example also closes the camera.
The camera belongs to the person visiting the page. It opens only after clicking the button, and the live video stays in the browser; only captured photos are uploaded. Browser camera access requires HTTPS (including hosted Hugging Face Spaces) or localhost, plus camera permission. A plain HTTP server IP address cannot access the camera. For the local demo, open http://localhost:8081 on the client machine, with port forwarding if the server is remote.
360° panoramas
Open the 360° Panorama tab and upload a full 360° × 180° equirectangular JPG, PNG, or WebP image with a 2:1 aspect ratio. Click Reconstruct panorama: the demo first generates 24 perspective views (eight headings at three elevations, 60° FOV) and matching camera intrinsics and poses, then runs Pose-conditioned reconstruction automatically. No camera JSON upload is required. The resolution and export settings apply to panorama reconstruction too. Conversion runs on the CPU before GPU inference.
The converter is included in this repository at scripts/convert_panorama.py, with its implementation in src/utils/panorama.py. It can also prepare an input scene from the command line:
python scripts/convert_panorama.py \
--input /path/to/panorama.jpg \
--output /path/to/prepared_scene \
--size 896
The resulting scene contains images/, prior_camera.json, and panorama.json. Apply EXIF orientation to the source image before uploading or converting it.
Updating example previews
assets/scene-previews/catalog.json matches each local CATEGORY/SCENE directory to the same numbered directory in the dataset. Its 24 WebP files are actual initial-camera Gaussian renders, packaged with the app so selecting examples does not download large HTML files. The live app does not require Chromium.
To rebuild previews after updating the examples or dataset, install the optional build tooling and run:
npm install --no-save playwright
npx playwright install chromium
python scripts/prepare_scene_previews.py
The script verifies the downloaded HTML hashes and uses the viewer’s captureFrame API at time zero, then publishes the new posters and catalog. Restart the demo afterward.
Development
Maintain the Gradio UI, ZeroGPU adapter, viewer, and demo examples in this repository. The GitHub repository contains the command-line inference release; it does not include the Gradio app or Space packaging scripts. This Space is self-contained and does not load code from a sibling checkout.
Run the demo regression checks with:
python -m unittest discover -s tests -v
Attribution
The demo structure follows InfiniSplat2, with SplatAnything's own inference pipeline and input camera format. Native Gaussian viewing uses PlayCanvas splat-transform (installed separately, pinned to 2.6.0). Original component notices are preserved in License.txt and Notice.txt.
Inference speed and progress
Inference Setting → Max images uses the first N frames: Object/Indoor/Pan-Recon default to 24, Street/Aerial to 100, Rig-Recon to 150. Uploaded images default to 0 (all frames); uploading a panorama selects 756 × 756 and its 24 generated views; a limit larger than the scene uses all available frames. Previewing precomputed examples remains independent of this setting.
GPU servers preload the two models in the background after startup and reuse them across requests. Set SPLATANYTHING_PRELOAD=0 to retain loading on the first request. ZeroGPU keeps its existing startup initialization and GPU scheduling. The inline progress indicator reports actual pipeline stages without obscuring the scene.
The interactive Gaussian viewer takes priority: prediction is followed by native Gaussian PLY export and a fast compressed PLY preview, displayed in a shared, cached SuperSplat viewer. Standalone HTML, raw points.ply, depth maps and processed RGB files are exported by a bounded CPU background worker. The download panel reports when all files are ready. Gaussian PLY is available immediately. Standalone HTML downloads wait until the background HTML export completes.
Full result waits for the background export to finish, then builds and caches the complete ZIP on its first download. It never packages a partially written result. If additional exports fail, the viewer and Gaussian PLY remain available and the app reports the failure. Archives use fast compression and keep already-compressed image formats as-is. Depth, RGB and raw points remain included; model precision and decoder chunk sizes are unchanged.
A local comparison used the same 4.64 million Gaussians, a 960 × 600 Chromium software WebGL viewport, a shared 100 Mbps server-side bandwidth limit, and three runs per format. From an already exported native PLY to the first rendered frame, conversion plus loading took 30.2 s for native PLY, 13.1 s for compressed PLY, 21.9 s for SOG, and 25.9 s for standalone HTML (sums of conversion and loading medians). These measurements exclude model inference and native PLY export; client hardware and network conditions affect results. On an unrestricted local connection, native PLY avoided conversion and had the shortest combined wait. Compressed PLY was selected for the hosted demo. No Gaussians are removed by this conversion; the native PLY remains available for full-precision download.
timings.json records preparation, model loading/lock wait, prediction (including preprocessing and CPU transfer), gaussian_export, preview_compression, and elapsed preview_ready. After the additional files finish, it also records html_export, auxiliary_export and elapsed all_files_ready. Background exports run on CPU after the ZeroGPU prediction reservation has ended.
Inference Setting → Voxel size ranges from 0.0015 to 0.0020, defaults to 0.0018, with step 0.0001. This updates the renderer's actual voxel merge size for each request, and is recorded in run.json and config.yaml. Coarser merging usually reduces Gaussian counts and HTML/PLY size, with a tradeoff in fine detail. It does not reduce the encoder's input tokens or the raw point/depth/RGB output sizes. Precomputed example HTMLs retain their published settings; click Reconstruct scene to apply a new voxel size.
Inference Setting → max_scale_pct (%) accepts a percentage greater than 0 and
up to 100, default 99.5. Gaussians above this scale percentile are removed before
voxel merging: 99.5 trims approximately the largest 0.5%, while 100 keeps all scales
under the percentile filter. Equal scale values at the cutoff can affect the exact
fraction removed. The existing filter_large_scale_infer config flag must remain
true for scale filtering (the supplied config enables it).
filter_flying_points_infer is off by default. Enable it to filter pixels near
relative depth discontinuities before point-cloud export and Gaussian voxel merging.
Both settings apply to RGB-only, pose-conditioned, video-frame, and panorama
reconstruction, and are recorded in run.json and the exported config.yaml.
Depth and RGB exports remain unchanged. Changing these settings takes effect on
the next reconstruction; precomputed example viewers retain their published filters.