junaid-simamdigital commited on
Commit
171ae61
·
verified ·
1 Parent(s): 5847084

docs: add point cloud evaluation

Browse files
Files changed (1) hide show
  1. EVALUATION_PROTOCOL.md +7 -0
EVALUATION_PROTOCOL.md CHANGED
@@ -14,6 +14,7 @@ not results on a named reconstruction benchmark.
14
  | Depth is quantitatively accurate | Named dataset split, metric definitions, baseline, confidence interval | Not yet measured |
15
  | Depth evaluation is reproducible | Paired `.npy` prediction/ground-truth arrays, mask, orientation flag, standard metrics | Implemented in `evaluate_depth.py`; dataset run still pending |
16
  | Gaussian export is direct from a model | Native source, candidate/export counts, portable file inspection | DA3 direct path verified |
 
17
  | Gaussian reconstruction is higher quality | Held-out views, renderer, PSNR/SSIM/LPIPS or equivalent, baseline | Not yet claimed |
18
  | Hidden-side geometry is correct | Held-out unseen-side views and a predeclared threshold | Not yet claimed |
19
 
@@ -77,6 +78,12 @@ run `python evaluate_depth.py prediction.npy ground_truth.npy`. Use
77
  `median_scaled` results; only the former is appropriate when metric scale is
78
  known.
79
 
 
 
 
 
 
 
80
  ## Current honest conclusion
81
 
82
  Simam3D demonstrates a reproducible depth/fusion/export workbench, native DA3
 
14
  | Depth is quantitatively accurate | Named dataset split, metric definitions, baseline, confidence interval | Not yet measured |
15
  | Depth evaluation is reproducible | Paired `.npy` prediction/ground-truth arrays, mask, orientation flag, standard metrics | Implemented in `evaluate_depth.py`; dataset run still pending |
16
  | Gaussian export is direct from a model | Native source, candidate/export counts, portable file inspection | DA3 direct path verified |
17
+ | Exported geometry is quantitatively close | Reference point cloud, declared threshold, symmetric Chamfer and F-score | Implemented in `evaluate_pointcloud.py`; reference run pending |
18
  | Gaussian reconstruction is higher quality | Held-out views, renderer, PSNR/SSIM/LPIPS or equivalent, baseline | Not yet claimed |
19
  | Hidden-side geometry is correct | Held-out unseen-side views and a predeclared threshold | Not yet claimed |
20
 
 
78
  `median_scaled` results; only the former is appropriate when metric scale is
79
  known.
80
 
81
+ For point-cloud or Gaussian position evaluation, use
82
+ `python evaluate_pointcloud.py exported.ply reference.ply --threshold 0.05`.
83
+ The evaluator reports both directions of nearest-neighbour error, precision,
84
+ recall, and F-score. A reference capture and predeclared coordinate alignment
85
+ are required before interpreting these numbers as reconstruction quality.
86
+
87
  ## Current honest conclusion
88
 
89
  Simam3D demonstrates a reproducible depth/fusion/export workbench, native DA3