Spaces:
Running on Zero
Running on Zero
docs: link Gaussian inspector
Browse files
README.md
CHANGED
|
@@ -1,246 +1,69 @@
|
|
| 1 |
---
|
| 2 |
-
title: Simam3D
|
| 3 |
emoji: 🧭
|
| 4 |
colorFrom: indigo
|
| 5 |
colorTo: blue
|
| 6 |
-
sdk:
|
| 7 |
-
|
| 8 |
-
suggested_hardware:
|
| 9 |
-
pinned:
|
| 10 |
-
tags:
|
| 11 |
-
- image-to-3d
|
| 12 |
-
- depth-estimation
|
| 13 |
-
- gaussian-splatting
|
| 14 |
-
- multi-view-reconstruction
|
| 15 |
models:
|
| 16 |
- depth-anything/Depth-Anything-V2-Small-hf
|
| 17 |
- depth-anything/DA3NESTED-GIANT-LARGE-1.1
|
| 18 |
---
|
| 19 |
|
| 20 |
-
# Simam3D
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
and
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
For paired depth arrays, run `python evaluate_depth.py prediction.npy
|
| 76 |
-
ground_truth.npy`; for a JSONL dataset, use `python evaluate_depth_dataset.py
|
| 77 |
-
pairs.jsonl`. For exported fused or Gaussian geometry, run `python
|
| 78 |
-
evaluate_pointcloud.py exported.ply reference.ply --threshold 0.05`, or use
|
| 79 |
-
`evaluate_pointcloud_dataset.py` for a JSONL set. Before a calibrated
|
| 80 |
-
multi-view experiment, validate `capture.json` with `python
|
| 81 |
-
validate_capture_manifest.py capture.json --check-files`. The schema and a
|
| 82 |
-
copyable example are in [CAPTURE_MANIFEST_SCHEMA.md](CAPTURE_MANIFEST_SCHEMA.md).
|
| 83 |
-
For COLMAP text outputs, use `python import_colmap.py cameras.txt images.txt
|
| 84 |
-
image_root capture.json --held-out IMAGE_ID ...` to create the manifest first.
|
| 85 |
-
These tools retain raw versus median-scaled depth results, declared geometric
|
| 86 |
-
thresholds, held-out splits, and failed samples; they do not turn synthetic
|
| 87 |
-
fixtures or synthetic camera poses into quality claims.
|
| 88 |
-
|
| 89 |
-
Multiple manifests can be compared with `python summarize_runs.py output`,
|
| 90 |
-
which emits backend, view-count, fusion, coverage, reprojection, fallback, and
|
| 91 |
-
provenance fields as JSON. Failed manifests are included with their status,
|
| 92 |
-
failing stage, and structured error details.
|
| 93 |
-
The dependency-free regression harness can be run with
|
| 94 |
-
`python benchmark_pipeline.py --output benchmark_output`. It repeats the CPU
|
| 95 |
-
fusion fixture at several resolutions and records signatures plus fusion
|
| 96 |
-
metrics. This proves deterministic geometry/export behavior only; it is not a
|
| 97 |
-
learned reconstruction-quality benchmark.
|
| 98 |
-
On Windows, run `powershell -ExecutionPolicy Bypass -File run_checks.ps1` before
|
| 99 |
-
publishing a change; this is the canonical compile and regression gate.
|
| 100 |
-
The DA3 package-layout resolver is tested independently in `da3_compat.py`, so
|
| 101 |
-
optional upstream import changes can be diagnosed without loading model
|
| 102 |
-
weights.
|
| 103 |
-
The model-free geometry path can also be exercised with
|
| 104 |
-
`python demo_pipeline.py --output demo_output`; it writes deterministic fused
|
| 105 |
-
and Gaussian artifacts for CPU-only integration testing.
|
| 106 |
-
The demo also emits a schema-3 manifest, so it can be checked with the same
|
| 107 |
-
`validate_manifest.py` tool as a model-backed run.
|
| 108 |
-
See [TROUBLESHOOTING.md](TROUBLESHOOTING.md) for the scheduler, smoke-test, and
|
| 109 |
-
browser-versus-server failure decision tree.
|
| 110 |
-
The research direction and required ablations are outlined in
|
| 111 |
-
[PAPER_OUTLINE.md](PAPER_OUTLINE.md).
|
| 112 |
-
The verified milestone history is recorded in [CHANGELOG.md](CHANGELOG.md).
|
| 113 |
-
Fusion metrics also report the number of distinct dominant source views in the
|
| 114 |
-
fused result, making view collapse visible without overstating provenance.
|
| 115 |
-
They report the fraction of voxels supported by multiple views as a separate
|
| 116 |
-
cross-view consistency signal, plus the number of source views with any
|
| 117 |
-
support in the fused result.
|
| 118 |
-
It also emits a dependency-free Gaussian-splat PLY initialized from the fused
|
| 119 |
-
points. Splat opacity and initial scale are confidence-weighted, allowing the
|
| 120 |
-
representation to expose uncertainty before any learned Gaussian optimization.
|
| 121 |
-
Each run includes a JSON manifest recording the backend, model, runtime,
|
| 122 |
-
number of views, elapsed time, and generated artifact paths.
|
| 123 |
-
The UI also accepts optional additional images so DA3 can process a small
|
| 124 |
-
multi-view batch in one request; fusion and novel-view synthesis remain
|
| 125 |
-
separate, explicitly tracked stages.
|
| 126 |
-
When the V2 fallback is selected, additional images use a deterministic
|
| 127 |
-
synthetic orbit pose prior and the run manifest records that limitation; only
|
| 128 |
-
DA3 camera metadata is treated as camera-aware geometry.
|
| 129 |
-
Hosted requests are bounded to eight views and a 1536-pixel maximum input side
|
| 130 |
-
by default, with `SIMAM3D_MAX_VIEWS` and `SIMAM3D_MAX_INPUT_SIDE` available for
|
| 131 |
-
controlled deployments. Inference runs under `torch.inference_mode()` to
|
| 132 |
-
avoid retaining training graphs. The heavy inference event is limited to one
|
| 133 |
-
concurrent request and guarded by a process lock, reducing avoidable V2/DA3
|
| 134 |
-
memory contention.
|
| 135 |
-
Fusion resolution is configurable with `SIMAM3D_FUSION_VOXEL_SIZE` (default
|
| 136 |
-
`0.02`) and `SIMAM3D_FUSION_MAX_POINTS` (default `150000`); both values are
|
| 137 |
-
recorded in the run manifest.
|
| 138 |
-
The provenance bitmask supports up to 30 views because it is exported as a
|
| 139 |
-
signed 32-bit integer.
|
| 140 |
-
|
| 141 |
-
The geometry layer is isolated in `simam3d_core.py`, a dependency-free NumPy
|
| 142 |
-
module with deterministic voxel fusion and confidence metrics. Its smoke tests
|
| 143 |
-
can run without CUDA or model weights:
|
| 144 |
-
|
| 145 |
-
```bash
|
| 146 |
-
python -c "import runpy; ns=runpy.run_path('tests/test_core.py'); [fn() for name,fn in ns.items() if name.startswith('test_')]"
|
| 147 |
-
```
|
| 148 |
-
|
| 149 |
-
Relative depth is normalized using a consistent inverse-depth convention for
|
| 150 |
-
the current scaffold, so exported geometry is useful for visualization but is
|
| 151 |
-
not yet metrically scaled. DA3 camera calibration and
|
| 152 |
-
multi-view pose quality should be evaluated before treating measurements as
|
| 153 |
-
physical dimensions.
|
| 154 |
-
|
| 155 |
-
## Planned backends
|
| 156 |
-
|
| 157 |
-
- **Depth Anything 3**: any-view depth, camera pose, confidence, and direct Gaussian estimation.
|
| 158 |
-
- **TRELLIS.2**: primary high-quality mesh and structured-latent backend.
|
| 159 |
-
- **Hunyuan3D 2.1**: alternative shape and PBR-texture backend.
|
| 160 |
-
- **gsplat**: Gaussian initialization, confidence-weighted fusion, and rendering.
|
| 161 |
-
|
| 162 |
-
Extend3D remains a research reference only. Simam3D will use adapters around
|
| 163 |
-
these maintained projects so each backend can be tested independently and
|
| 164 |
-
replaced without rewriting the UI or export layer.
|
| 165 |
-
|
| 166 |
-
## Companion prototypes
|
| 167 |
-
|
| 168 |
-
- [SimamTwin3D](https://huggingface.co/spaces/junaid-simamdigital/SimamTwin3D):
|
| 169 |
-
consent-first multi-view human-reference capture audit. It records hashes and
|
| 170 |
-
capture coverage; it does not claim a perfect digital twin.
|
| 171 |
-
- [SimamAnything2GS](https://huggingface.co/spaces/junaid-simamdigital/SimamAnything2GS):
|
| 172 |
-
arbitrary image/video ingest with bounded FFmpeg sampling and a CPU 2.5D
|
| 173 |
-
preview. It is an ingest smoke test, not learned 3D reconstruction.
|
| 174 |
-
|
| 175 |
-
These prototypes are intentionally separate so experimental identity/video
|
| 176 |
-
workflows and general Gaussian reconstruction do not weaken Simam3D's evidence
|
| 177 |
-
and provenance contract.
|
| 178 |
-
The falsifiable experiment queue is documented in
|
| 179 |
-
[RESEARCH_BACKLOG.md](RESEARCH_BACKLOG.md).
|
| 180 |
-
The acceptance criteria for depth, fusion, Gaussian, and hidden-side claims
|
| 181 |
-
are documented in [EVALUATION_PROTOCOL.md](EVALUATION_PROTOCOL.md).
|
| 182 |
-
|
| 183 |
-
## SimamDigital publishing
|
| 184 |
-
|
| 185 |
-
The live demo currently belongs to the `junaid-simamdigital` user namespace.
|
| 186 |
-
For prominent company branding, keep this Space linked from the
|
| 187 |
-
[SimamDigital organization](https://huggingface.co/SimamDigital), and later
|
| 188 |
-
move it to `SimamDigital/Simam3D` once the organization profile and collection
|
| 189 |
-
are ready. The `pinned` flag keeps the Space near the top of its owning profile.
|
| 190 |
-
|
| 191 |
-
## Hosting notes
|
| 192 |
-
|
| 193 |
-
This repository is currently packaged as a Docker Space, so its T4/L4/A10G
|
| 194 |
-
hardware choices are metered Space hardware. Hugging Face ZeroGPU is a
|
| 195 |
-
separate Gradio-SDK-only hosting mode; selecting `zero-a10g` on this Docker
|
| 196 |
-
Space is not a drop-in replacement. A future ZeroGPU variant would need a
|
| 197 |
-
Gradio-SDK packaging pass, the `spaces.GPU` function boundary, and the
|
| 198 |
-
ZeroGPU-supported Torch/Python matrix. The Docker version remains the
|
| 199 |
-
reproducible target for calibrated DA3 experiments.
|
| 200 |
-
The app already includes an optional no-op-safe `spaces.GPU` boundary; it does
|
| 201 |
-
not convert this Docker Space into a ZeroGPU Space by itself.
|
| 202 |
-
See [ZERO_GPU_MIGRATION.md](ZERO_GPU_MIGRATION.md) for the separate-variant
|
| 203 |
-
steps and acceptance checks.
|
| 204 |
-
|
| 205 |
-
## Roadmap
|
| 206 |
-
|
| 207 |
-
1. Depth baseline, GLB, PLY, camera metadata, confidence fusion, and Gaussian initialization (implemented).
|
| 208 |
-
2. Validate DA3NESTED-GIANT-LARGE-1.1 camera-aware and direct-Gaussian outputs against the V2 fallback (verified on the public GPU Space for one-, two-, and three-view requests).
|
| 209 |
-
3. Add four-view completion with source-view preservation and confidence-weighted fusion.
|
| 210 |
-
4. Add TRELLIS.2 and Hunyuan3D backend selection.
|
| 211 |
-
5. Add gsplat optimization and interactive splat preview.
|
| 212 |
-
|
| 213 |
-
The UI defaults to the V2 stable baseline for a first successful smoke test;
|
| 214 |
-
DA3 remains available as the explicit camera-aware research backend. The
|
| 215 |
-
public GPU Space installs the pinned DA3 dependency and keeps the explicit V2
|
| 216 |
-
fallback. The Docker image still treats DA3 as optional: `requirements-da3.txt`
|
| 217 |
-
can be layered into a research image when the upstream API is known to be
|
| 218 |
-
compatible. This keeps the baseline Docker build independent of the
|
| 219 |
-
experimental Git dependency.
|
| 220 |
-
The Dockerfile exposes the same opt-in as
|
| 221 |
-
`--build-arg SIMAM3D_INSTALL_DA3=1` for reproducible GPU research builds.
|
| 222 |
-
The optional DA3 source is pinned to a specific upstream commit rather than
|
| 223 |
-
tracking a moving branch.
|
| 224 |
-
The container also includes a health check for the Gradio HTTP process, so a
|
| 225 |
-
reachable UI can be separated from later model-inference failures.
|
| 226 |
-
Gradio is launched with detailed error display enabled, while the manifest and
|
| 227 |
-
container logs remain the authoritative diagnostic record.
|
| 228 |
-
Failed requests also leave a redacted failure manifest with the failing stage,
|
| 229 |
-
backend, runtime, software versions, input hashes, and exception details for
|
| 230 |
-
reproducible diagnosis.
|
| 231 |
-
The Diagnostics panel can list those reports for download after a failed run.
|
| 232 |
-
The Diagnostics panel checks the optional DA3 package and the expected
|
| 233 |
-
`DepthAnything3` symbol across both known upstream import locations, so an
|
| 234 |
-
installed but incompatible DA3 package is visible before model inference.
|
| 235 |
-
When DA3 is selected, its depth, confidence, extrinsics, and intrinsics shapes
|
| 236 |
-
are validated at the adapter boundary; malformed results trigger the labeled V2
|
| 237 |
-
fallback before any geometry export begins, and the exact fallback exception is
|
| 238 |
-
stored in the run manifest for diagnosis.
|
| 239 |
-
Backend switching also releases the previously loaded heavyweight model before
|
| 240 |
-
loading the next one, reducing avoidable GPU memory pressure in long-lived
|
| 241 |
-
Space processes.
|
| 242 |
-
Additional views accept both filepath strings and Gradio `FileData` uploads;
|
| 243 |
-
unreadable files now produce a direct input error before inference starts.
|
| 244 |
-
The baseline requirements also pin the CUDA-compatible Torch pair and bound
|
| 245 |
-
dependency major versions so rebuilds do not silently change the inference or
|
| 246 |
-
export contract.
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Simam3D ZeroGPU
|
| 3 |
emoji: 🧭
|
| 4 |
colorFrom: indigo
|
| 5 |
colorTo: blue
|
| 6 |
+
sdk: gradio
|
| 7 |
+
app_file: app.py
|
| 8 |
+
suggested_hardware: zero-a10g
|
| 9 |
+
pinned: false
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
models:
|
| 11 |
- depth-anything/Depth-Anything-V2-Small-hf
|
| 12 |
- depth-anything/DA3NESTED-GIANT-LARGE-1.1
|
| 13 |
---
|
| 14 |
|
| 15 |
+
# Simam3D ZeroGPU
|
| 16 |
+
|
| 17 |
+
Gradio-SDK deployment variant of Simam3D for Hugging Face ZeroGPU. It reuses
|
| 18 |
+
the same tested depth, fusion, Gaussian export, provenance, and failure-report
|
| 19 |
+
code as the Docker Space. ZeroGPU availability and queue time are platform
|
| 20 |
+
controlled; this variant does not claim a successful model run until a real
|
| 21 |
+
request returns a validated manifest.
|
| 22 |
+
|
| 23 |
+
## Verified capabilities
|
| 24 |
+
|
| 25 |
+
- Depth Anything V2 stable-baseline inference on GPU.
|
| 26 |
+
- Depth Anything 3 camera-aware inference using the Nested checkpoint.
|
| 27 |
+
- Direct DA3 Gaussian export with deterministic opacity-based point capping.
|
| 28 |
+
- GLB, point-cloud, fused-point-cloud, Gaussian PLY, uncertainty, turntable,
|
| 29 |
+
evidence-card, and reproducibility-manifest outputs.
|
| 30 |
+
- Input-diversity and fusion-support diagnostics that distinguish duplicate,
|
| 31 |
+
near-duplicate, synthetic-prior, and native-camera evidence.
|
| 32 |
+
|
| 33 |
+
See [HOSTED_VERIFICATION.md](HOSTED_VERIFICATION.md) for live run IDs and
|
| 34 |
+
metrics, [PAPER_OUTLINE.md](PAPER_OUTLINE.md) for the research protocol, and
|
| 35 |
+
[RESEARCH_BACKLOG.md](RESEARCH_BACKLOG.md) for falsifiable follow-up ideas.
|
| 36 |
+
The claim boundaries and acceptance criteria are defined in
|
| 37 |
+
[EVALUATION_PROTOCOL.md](EVALUATION_PROTOCOL.md).
|
| 38 |
+
|
| 39 |
+
For public project stories, see the [Simam AI Lab case-study series](CASE_STUDIES/README.md)
|
| 40 |
+
and the [LinkedIn launch pack](CASE_STUDIES/LINKEDIN-POSTS.md).
|
| 41 |
+
|
| 42 |
+
Maintainers can reproduce the no-model hosted check with
|
| 43 |
+
[`hosted_smoke_test.py`](hosted_smoke_test.py), and use
|
| 44 |
+
[`TROUBLESHOOTING.md`](TROUBLESHOOTING.md) for failure-stage guidance.
|
| 45 |
+
|
| 46 |
+
For quantitative work, use [`evaluate_depth.py`](evaluate_depth.py) and
|
| 47 |
+
[`evaluate_depth_dataset.py`](evaluate_depth_dataset.py) for depth arrays,
|
| 48 |
+
[`evaluate_pointcloud.py`](evaluate_pointcloud.py) and
|
| 49 |
+
[`evaluate_pointcloud_dataset.py`](evaluate_pointcloud_dataset.py) for exported
|
| 50 |
+
3D/Gaussian positions, and validate calibrated inputs with
|
| 51 |
+
[`validate_capture_manifest.py`](validate_capture_manifest.py). The copyable
|
| 52 |
+
manifest format is documented in
|
| 53 |
+
[`CAPTURE_MANIFEST_SCHEMA.md`](CAPTURE_MANIFEST_SCHEMA.md).
|
| 54 |
+
Use [`inspect_gaussian_ply.py`](inspect_gaussian_ply.py) to inspect exported
|
| 55 |
+
Gaussian parameters and evidence fields.
|
| 56 |
+
COLMAP text models can be converted with
|
| 57 |
+
[`import_colmap.py`](import_colmap.py) before validation.
|
| 58 |
+
|
| 59 |
+
## Important limitations
|
| 60 |
+
|
| 61 |
+
This Space does not claim metric-scale reconstruction, learned Gaussian
|
| 62 |
+
optimization, verified unseen-side geometry, or generated novel-view images.
|
| 63 |
+
The displayed turntable is a point/Gaussian export preview, not proof that the
|
| 64 |
+
hidden side of an object was reconstructed correctly.
|
| 65 |
+
|
| 66 |
+
The public GPU deployment installs the pinned DA3 package so both the
|
| 67 |
+
camera-aware path and the V2 baseline can be tested. Start with the explicit
|
| 68 |
+
export smoke test to separate UI/GPU scheduling from model inference; a DA3
|
| 69 |
+
failure is reported and falls back to V2 rather than being hidden.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|