Simam3D-GPU / PAPER_OUTLINE.md
junaid-simamdigital's picture
research: align paper outline with evaluation tools
5c8dbf5 verified
|
Raw History Blame Contribute Delete
4.14 kB
# Working paper outline: Hypothesis-guided hidden-side reconstruction
This is an experiment plan, not a claim of completed results.
## Proposed title
**Simam3D: Evidence- and Hypothesis-Guided Reconstruction Beyond the Visible View**
## Research question
Can a reconstruction system use a user's prior about unseen content to choose
informative views and complete a scene while keeping observed geometry,
generated geometry, uncertainty, and lighting evidence separate?
## Implemented scaffold
- Depth and confidence prediction with a stable V2 baseline and optional DA3
camera-aware adapter.
- Relative-depth projection, synthetic orbit priors, confidence-weighted voxel
fusion, Gaussian initialization, and source-view bitmasks.
- Uncertainty, projection-coverage, and reprojection diagnostics.
- Depth-normal photometric light hypothesis and shareable reveal artifacts.
- Manifest claims contract, artifact hashes, and run summarization.
- Reproducible depth evaluation for paired arrays and JSONL datasets, with raw
versus median-scaled reporting.
- Reproducible point-cloud/Gaussian evaluation using symmetric Chamfer,
precision, recall, and F-score, plus calibrated-capture manifest validation.
## Preliminary reproducibility evidence
The public GPU deployment has completed native DA3 requests with one, two,
and three submitted views. The one-view regression returned the complete
14-output bundle in 21.1 seconds on an RTX PRO 6000 MIG slice and recorded
`gaussian_source=da3_direct` with 2,048 exported Gaussians. A three-view
synthetic stress test also returned the complete bundle in 28.9 seconds, with
native DA3 pose/confidence and 2,048 exported Gaussians; its manifest reported
34.1% mean projection coverage, 0.626 relative-depth reprojection MAE, and
`low_multi_view_support`.
These are execution, provenance, and diagnostic observations on controlled
inputs. They do not establish reconstruction quality, metric scale,
unseen-side accuracy, or improvement over a baseline. The exact run IDs,
hardware, hashes, and validation procedure are recorded in
`HOSTED_VERIFICATION.md`.
## Proposed contributions to test
1. **Prior-conditioned acquisition:** record a user hypothesis before proposing
the next view; compare against fixed orbit and random-view policies.
2. **Evidence-weighted completion:** let observed points dominate generated
points, with opacity and scale controlled by confidence and provenance.
3. **Lighting-consistent completion:** reject candidate generated views whose
estimated depth-normal light direction disagrees with observed views.
4. **Uncertainty-driven interaction:** request a real photograph at the view
with the largest predicted evidence gap, then measure how much uncertainty
decreases.
## Required ablations
- V2 depth versus DA3 depth where the DA3 installation and model revision are
recorded.
- Uniform fusion versus confidence-weighted fusion.
- No user prior versus user prior.
- Random view selection versus uncertainty-guided selection.
- Geometry-only scoring versus geometry-plus-lighting scoring.
## Evaluation protocol
Use a named multi-view dataset with held-out cameras. Report held-out depth,
RGB, geometry, and coverage metrics, plus provenance precision: the fraction of
points labeled observed that are supported by an input view. Report failure
cases separately for reflective, textureless, transparent, and highly
occluded objects. Do not call relative-depth outputs metric reconstruction.
Before scoring, validate the capture manifest with
`validate_capture_manifest.py`; then use `evaluate_depth_dataset.py` and
`evaluate_pointcloud_dataset.py` to retain per-sample failures and aggregate
metrics. The current synthetic geometry benchmark is a contract test for
projection/fusion determinism, not a learned-model quality result.
## Honest success criterion
The method is successful only if held-out-view metrics and provenance-aware
ablations improve over the selected baselines under the same compute budget.
The current Space provides instrumentation for this test; it does not yet
provide those learned completion results.