Spaces:
Running on Zero
Running on Zero
docs: add point cloud evaluation
Browse files- EVALUATION_PROTOCOL.md +7 -0
EVALUATION_PROTOCOL.md
CHANGED
|
@@ -14,6 +14,7 @@ not results on a named reconstruction benchmark.
|
|
| 14 |
| Depth is quantitatively accurate | Named dataset split, metric definitions, baseline, confidence interval | Not yet measured |
|
| 15 |
| Depth evaluation is reproducible | Paired `.npy` prediction/ground-truth arrays, mask, orientation flag, standard metrics | Implemented in `evaluate_depth.py`; dataset run still pending |
|
| 16 |
| Gaussian export is direct from a model | Native source, candidate/export counts, portable file inspection | DA3 direct path verified |
|
|
|
|
| 17 |
| Gaussian reconstruction is higher quality | Held-out views, renderer, PSNR/SSIM/LPIPS or equivalent, baseline | Not yet claimed |
|
| 18 |
| Hidden-side geometry is correct | Held-out unseen-side views and a predeclared threshold | Not yet claimed |
|
| 19 |
|
|
@@ -77,6 +78,12 @@ run `python evaluate_depth.py prediction.npy ground_truth.npy`. Use
|
|
| 77 |
`median_scaled` results; only the former is appropriate when metric scale is
|
| 78 |
known.
|
| 79 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
## Current honest conclusion
|
| 81 |
|
| 82 |
Simam3D demonstrates a reproducible depth/fusion/export workbench, native DA3
|
|
|
|
| 14 |
| Depth is quantitatively accurate | Named dataset split, metric definitions, baseline, confidence interval | Not yet measured |
|
| 15 |
| Depth evaluation is reproducible | Paired `.npy` prediction/ground-truth arrays, mask, orientation flag, standard metrics | Implemented in `evaluate_depth.py`; dataset run still pending |
|
| 16 |
| Gaussian export is direct from a model | Native source, candidate/export counts, portable file inspection | DA3 direct path verified |
|
| 17 |
+
| Exported geometry is quantitatively close | Reference point cloud, declared threshold, symmetric Chamfer and F-score | Implemented in `evaluate_pointcloud.py`; reference run pending |
|
| 18 |
| Gaussian reconstruction is higher quality | Held-out views, renderer, PSNR/SSIM/LPIPS or equivalent, baseline | Not yet claimed |
|
| 19 |
| Hidden-side geometry is correct | Held-out unseen-side views and a predeclared threshold | Not yet claimed |
|
| 20 |
|
|
|
|
| 78 |
`median_scaled` results; only the former is appropriate when metric scale is
|
| 79 |
known.
|
| 80 |
|
| 81 |
+
For point-cloud or Gaussian position evaluation, use
|
| 82 |
+
`python evaluate_pointcloud.py exported.ply reference.ply --threshold 0.05`.
|
| 83 |
+
The evaluator reports both directions of nearest-neighbour error, precision,
|
| 84 |
+
recall, and F-score. A reference capture and predeclared coordinate alignment
|
| 85 |
+
are required before interpreting these numbers as reconstruction quality.
|
| 86 |
+
|
| 87 |
## Current honest conclusion
|
| 88 |
|
| 89 |
Simam3D demonstrates a reproducible depth/fusion/export workbench, native DA3
|