--- title: SplatAnything (InfiniSplat v3) emoji: 🌐 colorFrom: blue colorTo: gray sdk: gradio sdk_version: 6.5.1 python_version: "3.10" app_file: space_app.py pinned: false --- # SplatAnything **Gaussian Spacing Normalization for Feed-Forward 3DGS across Arbitrary Scenes** [Project page](https://zju3dv.github.io/SplatAnything/) · [Paper](https://arxiv.org/html/2611.08264) Upload one image or several overlapping images of one static scene, extract images from a video, or upload one full 360° panorama. Select **RGB-only** to predict cameras, or **Pose-conditioned** to supply camera poses and intrinsics in `prior_camera.json`. Street, Rig-Recon and Aerial default to 504 × 504; Object, Indoor and Pan-Recon default to 756 × 756. Width and height presets are 504, 756, 896 and 1022; you can also type each dimension manually (positive multiples of 14). Single-image RGB-only reconstruction is supported; additional overlapping views provide more coverage. Uploading your own images resets the resolution to at most 896 × 896. Each dimension is capped by the smallest corresponding dimension in the uploaded set, then rounded down to a multiple of 14 (for example, 800 × 600 becomes 798 × 588). Adding or deleting images recomputes these defaults; clearing the upload restores 896 × 896. Captured camera photos and extracted video frames follow the same rule. Original files are preserved, and you can still edit width and height manually. The app renders native 3D Gaussian splats in a PlayCanvas viewer. It provides Gaussian PLY, standalone HTML, and a complete ZIP containing point clouds, cameras, processed RGB, predicted depth, and the effective configuration. Result links expire after 24 hours. Anyone given a result link can access that result until it expires; download it to keep a copy. ## Run locally Install a CUDA-compatible PyTorch build first, then: ```bash python -m pip install -r requirements.txt npm ci python app.py --host 0.0.0.0 --port 8081 # Open http://127.0.0.1:8081 ``` You can also run `python space_app.py`; it installs the pinned viewer package automatically and listens on `0.0.0.0:7860`. This repository contains the app, inference code, examples, and viewer dependencies. It runs independently of the GitHub checkout. ## Checkpoints Configure the server with complete student and WorldMirror-2 geometry weights. Local defaults are: ```text checkpoints/splatanything.ckpt checkpoints/worldmirror2.safetensors checkpoints/skyseg.onnx # optional, for sky removal ``` Use `SPLATANYTHING_CKPT`, `WORLDMIRROR2_CKPT`, and `SPLATANYTHING_SKY_CKPT` to select other local files. The local preparation can link to existing weights; the external links and checkpoints are ignored by Git and are not portable uploads. For a hosted Space, upload the two inference checkpoints to a Hugging Face **model** repository, then configure these Space Variables: | Variable | Value | |---|---| | `SPLATANYTHING_MODEL_REPO` | Your model repository ID (`owner/model`) | | `SPLATANYTHING_MODEL_REVISION` | Model commit or revision (default `main`) | | `SPLATANYTHING_MODEL_FILE` | Student filename (default `splatanything.ckpt`) | | `SPLATANYTHING_GEOMETRY_FILE` | Geometry filename (default `worldmirror2.safetensors`) | For a private model repository, add `HF_TOKEN` as a Space **Secret** with read access. Tokens stay on the server. Sky removal uses `skyseg.onnx`, downloaded and cached on first use from the model repository (default `ritianyu/SplatAnything`). `SPLATANYTHING_SKY_REPO`, `SPLATANYTHING_SKY_REVISION`, and `SPLATANYTHING_SKY_FILE` can select a separate sky model; `SPLATANYTHING_SKY_CKPT` overrides this with a local ONNX file. The CPU session is reused. Sky segmentation runs before the ZeroGPU reservation, with its time recorded as `sky_and_input_preprocessing`. Street and Rig-Recon examples enable **Remove sky** by default; other categories start with it disabled. You can change the checkbox before reconstruction. The shared `configs/inference.yaml` uses a default voxel size of 0.0018 and a 99.5 scale percentile with the supplied fine-tuned checkpoint. Both camera-mode flags are selected together by the UI. CLI examples use `python inference.py --example object/1 --mode pose`; use `--list_examples` to inspect the compact examples included in this bundle. ## Hugging Face runtime This folder is self-contained: it includes the inference source, Gradio UI, portable configuration, small examples, and the npm lockfile. It does not depend on a sibling GitHub checkout, private filesystem paths, or startup Git clones. 1. Push this folder to a new **Gradio Space** after reviewing it. 2. Select GPU hardware and configure the model variables above. 3. For **ZeroGPU**, set `SPLATANYTHING_ZEROGPU=1` and select ZeroGPU hardware. Its `spaces.GPU` adapter reserves the GPU only for prediction; model placement happens at startup and PLY/HTML/ZIP exports run afterward on the CPU. 4. For a dedicated GPU, leave `SPLATANYTHING_ZEROGPU=0` (the default). Max images selects the first N input frames; 0 or an empty value uses all frames. User uploads, video frames and camera captures reset it to 0. Video extraction samples 2–32 frames. `SPLATANYTHING_DEVICE=cpu` is available for small local smoke tests. `SPLATANYTHING_CONFIG` selects another complete inference YAML. GPU usage is subject to the hardware and quota settings of the hosted Space. The original example images live in the dataset [`ritianyu/SplatAnything_Demo`](https://huggingface.co/datasets/ritianyu/SplatAnything_Demo/tree/main/examples), under `examples/CATEGORY/SCENE/`. Set `SPLATANYTHING_EXAMPLES_REPO=ritianyu/SplatAnything_Demo` and `SPLATANYTHING_EXAMPLES_REVISION` to a dataset commit. `space_app.py` downloads the original images and camera files to a local cache before starting the studio. This keeps the Space source below its repository storage limit while retaining all original views and image quality. Local runs use the existing `examples/` folder; `SPLATANYTHING_EXAMPLES_DIR` or `--examples_dir` can select another folder. ## Input cameras Camera IDs must match the image filename stems. Supply OpenCV 4 × 4 camera-to-world poses and 3 × 3 intrinsics in original-image pixels. The app updates intrinsics during cropping and resizing. See [prepare_data.md](prepare_data.md) for camera conversion and validation. The scene library has six categories: **Object, Indoor, Street, Aerial, Rig-Recon, and Pan-Recon**. Examples are loaded directly from `examples/CATEGORY/SCENE/images/` and the matching `prior_camera.json`, when available. Indoor scenes use RGB-only mode. Street, Rig-Recon and Aerial scenes use 504 × 504; Object, Indoor and Pan-Recon use 756 × 756. The studio displays small previews for faster loading; Max images controls how many original views are reconstructed. Selecting a card loads the scene; reconstruction starts when you click the button. The studio opens with no selected data and a white viewer with an animated SplatAnything infinity mark. **Clear** resets the current example, uploads, camera captures, camera JSON and result preview for your session, restoring the idle animation. The animation uses local SVG/CSS, has no visible text, and respects reduced-motion preferences. Selecting an example loads its images and settings; the right-hand viewer shows a still captured from the matching HTML’s initial Gaussian camera. Click **Explore 3D** to download and open the precomputed scene from [SplatAnything_Demo](https://huggingface.co/datasets/ritianyu/SplatAnything_Demo). HTML files are fetched only on this click, verified against the pinned dataset revision, and cached for reuse. Click **Reconstruct scene** to run new inference with the current inputs and settings. Example covers use square crops for Object, Indoor, Street, Aerial, and Rig-Recon; Pan-Recon keeps the original panorama aspect ratio. These covers affect previews only: reconstruction uses the complete original images. Captions sit below the image. Video frame extraction is a CPU operation and resets the mode to RGB-only. ## Browser camera In **Images**, click **Open camera** and allow camera access. Capture one image or several overlapping views of one static scene; each snapshot is added to the input preview. The first snapshot starts a new RGB-only scene, replacing any previously selected example or uploads. Later snapshots append to that camera scene. Use **New set** to start over and **Close camera** to release the camera; closing it keeps the captured photos available for reconstruction. Switching input tabs or selecting an example also closes the camera. The camera belongs to the person visiting the page. It opens only after clicking the button, and the live video stays in the browser; only captured photos are uploaded. Browser camera access requires **HTTPS** (including hosted Hugging Face Spaces) or **localhost**, plus camera permission. A plain HTTP server IP address cannot access the camera. For the local demo, open `http://localhost:8081` on the client machine, with port forwarding if the server is remote. ## 360° panoramas Open the **360° Panorama** tab and upload a full 360° × 180° equirectangular JPG, PNG, or WebP image with a 2:1 aspect ratio. Click **Reconstruct panorama**: the demo first generates 24 perspective views (eight headings at three elevations, 60° FOV) and matching camera intrinsics and poses, then runs **Pose-conditioned** reconstruction automatically. No camera JSON upload is required. The resolution and export settings apply to panorama reconstruction too. Conversion runs on the CPU before GPU inference. The converter is included in this repository at [scripts/convert_panorama.py](scripts/convert_panorama.py), with its implementation in [src/utils/panorama.py](src/utils/panorama.py). It can also prepare an input scene from the command line: ```bash python scripts/convert_panorama.py \ --input /path/to/panorama.jpg \ --output /path/to/prepared_scene \ --size 896 ``` The resulting scene contains `images/`, `prior_camera.json`, and `panorama.json`. Apply EXIF orientation to the source image before uploading or converting it. ## Updating example previews `assets/scene-previews/catalog.json` matches each local `CATEGORY/SCENE` directory to the same numbered directory in the dataset. Its 24 WebP files are actual initial-camera Gaussian renders, packaged with the app so selecting examples does not download large HTML files. The live app does not require Chromium. To rebuild previews after updating the examples or dataset, install the optional build tooling and run: ```bash npm install --no-save playwright npx playwright install chromium python scripts/prepare_scene_previews.py ``` The script verifies the downloaded HTML hashes and uses the viewer’s `captureFrame` API at time zero, then publishes the new posters and catalog. Restart the demo afterward. ## Development Maintain the Gradio UI, ZeroGPU adapter, viewer, and demo examples in this repository. The GitHub repository contains the command-line inference release; it does not include the Gradio app or Space packaging scripts. This Space is self-contained and does not load code from a sibling checkout. Run the demo regression checks with: ```bash python -m unittest discover -s tests -v ``` ## Attribution The demo structure follows [InfiniSplat2](https://huggingface.co/spaces/Edisoneh/InfiniSplat2), with SplatAnything's own inference pipeline and input camera format. Native Gaussian viewing uses [PlayCanvas splat-transform](https://github.com/playcanvas/splat-transform) (installed separately, pinned to 2.6.0). Original component notices are preserved in [License.txt](License.txt) and [Notice.txt](Notice.txt). ## Inference speed and progress Inference Setting → **Max images** uses the first N frames: Object/Indoor/Pan-Recon default to 24, Street/Aerial to 100, Rig-Recon to 150. Uploaded images default to 0 (all frames); uploading a panorama selects 756 × 756 and its 24 generated views; a limit larger than the scene uses all available frames. Previewing precomputed examples remains independent of this setting. GPU servers preload the two models in the background after startup and reuse them across requests. Set `SPLATANYTHING_PRELOAD=0` to retain loading on the first request. ZeroGPU keeps its existing startup initialization and GPU scheduling. The inline progress indicator reports actual pipeline stages without obscuring the scene. The interactive Gaussian viewer takes priority: prediction is followed by native Gaussian PLY export and a fast compressed PLY preview, displayed in a shared, cached SuperSplat viewer. Standalone HTML, raw `points.ply`, depth maps and processed RGB files are exported by a bounded CPU background worker. The download panel reports when all files are ready. Gaussian PLY is available immediately. Standalone HTML downloads wait until the background HTML export completes. **Full result** waits for the background export to finish, then builds and caches the complete ZIP on its first download. It never packages a partially written result. If additional exports fail, the viewer and Gaussian PLY remain available and the app reports the failure. Archives use fast compression and keep already-compressed image formats as-is. Depth, RGB and raw points remain included; model precision and decoder chunk sizes are unchanged. A local comparison used the same 4.64 million Gaussians, a 960 × 600 Chromium software WebGL viewport, a shared 100 Mbps server-side bandwidth limit, and three runs per format. From an already exported native PLY to the first rendered frame, conversion plus loading took 30.2 s for native PLY, **13.1 s for compressed PLY**, 21.9 s for SOG, and 25.9 s for standalone HTML (sums of conversion and loading medians). These measurements exclude model inference and native PLY export; client hardware and network conditions affect results. On an unrestricted local connection, native PLY avoided conversion and had the shortest combined wait. Compressed PLY was selected for the hosted demo. No Gaussians are removed by this conversion; the native PLY remains available for full-precision download. `timings.json` records preparation, model loading/lock wait, prediction (including preprocessing and CPU transfer), `gaussian_export`, `preview_compression`, and elapsed `preview_ready`. After the additional files finish, it also records `html_export`, `auxiliary_export` and elapsed `all_files_ready`. Background exports run on CPU after the ZeroGPU prediction reservation has ended. Inference Setting → **Voxel size** ranges from **0.0015 to 0.0020**, defaults to **0.0018**, with step **0.0001**. This updates the renderer's actual voxel merge size for each request, and is recorded in `run.json` and `config.yaml`. Coarser merging usually reduces Gaussian counts and HTML/PLY size, with a tradeoff in fine detail. It does not reduce the encoder's input tokens or the raw point/depth/RGB output sizes. Precomputed example HTMLs retain their published settings; click Reconstruct scene to apply a new voxel size. Inference Setting → **max_scale_pct (%)** accepts a percentage greater than 0 and up to 100, default **99.5**. Gaussians above this scale percentile are removed before voxel merging: 99.5 trims approximately the largest 0.5%, while 100 keeps all scales under the percentile filter. Equal scale values at the cutoff can affect the exact fraction removed. The existing `filter_large_scale_infer` config flag must remain true for scale filtering (the supplied config enables it). **filter_flying_points_infer** is off by default. Enable it to filter pixels near relative depth discontinuities before point-cloud export and Gaussian voxel merging. Both settings apply to RGB-only, pose-conditioned, video-frame, and panorama reconstruction, and are recorded in `run.json` and the exported `config.yaml`. Depth and RGB exports remain unchanged. Changing these settings takes effect on the next reconstruction; precomputed example viewers retain their published filters.