Simam3D-GPU / PAPER_OUTLINE.md
junaid-simamdigital's picture
research: align paper outline with evaluation tools
5c8dbf5 verified
|
Raw History Blame Contribute Delete
4.14 kB

A newer version of the Gradio SDK is available: 6.30.0

Upgrade

Working paper outline: Hypothesis-guided hidden-side reconstruction

This is an experiment plan, not a claim of completed results.

Proposed title

Simam3D: Evidence- and Hypothesis-Guided Reconstruction Beyond the Visible View

Research question

Can a reconstruction system use a user's prior about unseen content to choose informative views and complete a scene while keeping observed geometry, generated geometry, uncertainty, and lighting evidence separate?

Implemented scaffold

  • Depth and confidence prediction with a stable V2 baseline and optional DA3 camera-aware adapter.
  • Relative-depth projection, synthetic orbit priors, confidence-weighted voxel fusion, Gaussian initialization, and source-view bitmasks.
  • Uncertainty, projection-coverage, and reprojection diagnostics.
  • Depth-normal photometric light hypothesis and shareable reveal artifacts.
  • Manifest claims contract, artifact hashes, and run summarization.
  • Reproducible depth evaluation for paired arrays and JSONL datasets, with raw versus median-scaled reporting.
  • Reproducible point-cloud/Gaussian evaluation using symmetric Chamfer, precision, recall, and F-score, plus calibrated-capture manifest validation.

Preliminary reproducibility evidence

The public GPU deployment has completed native DA3 requests with one, two, and three submitted views. The one-view regression returned the complete 14-output bundle in 21.1 seconds on an RTX PRO 6000 MIG slice and recorded gaussian_source=da3_direct with 2,048 exported Gaussians. A three-view synthetic stress test also returned the complete bundle in 28.9 seconds, with native DA3 pose/confidence and 2,048 exported Gaussians; its manifest reported 34.1% mean projection coverage, 0.626 relative-depth reprojection MAE, and low_multi_view_support.

These are execution, provenance, and diagnostic observations on controlled inputs. They do not establish reconstruction quality, metric scale, unseen-side accuracy, or improvement over a baseline. The exact run IDs, hardware, hashes, and validation procedure are recorded in HOSTED_VERIFICATION.md.

Proposed contributions to test

  1. Prior-conditioned acquisition: record a user hypothesis before proposing the next view; compare against fixed orbit and random-view policies.
  2. Evidence-weighted completion: let observed points dominate generated points, with opacity and scale controlled by confidence and provenance.
  3. Lighting-consistent completion: reject candidate generated views whose estimated depth-normal light direction disagrees with observed views.
  4. Uncertainty-driven interaction: request a real photograph at the view with the largest predicted evidence gap, then measure how much uncertainty decreases.

Required ablations

  • V2 depth versus DA3 depth where the DA3 installation and model revision are recorded.
  • Uniform fusion versus confidence-weighted fusion.
  • No user prior versus user prior.
  • Random view selection versus uncertainty-guided selection.
  • Geometry-only scoring versus geometry-plus-lighting scoring.

Evaluation protocol

Use a named multi-view dataset with held-out cameras. Report held-out depth, RGB, geometry, and coverage metrics, plus provenance precision: the fraction of points labeled observed that are supported by an input view. Report failure cases separately for reflective, textureless, transparent, and highly occluded objects. Do not call relative-depth outputs metric reconstruction.

Before scoring, validate the capture manifest with validate_capture_manifest.py; then use evaluate_depth_dataset.py and evaluate_pointcloud_dataset.py to retain per-sample failures and aggregate metrics. The current synthetic geometry benchmark is a contract test for projection/fusion determinism, not a learned-model quality result.

Honest success criterion

The method is successful only if held-out-view metrics and provenance-aware ablations improve over the selected baselines under the same compute budget. The current Space provides instrumentation for this test; it does not yet provide those learned completion results.