yxma commited on
Commit
71bf0be
·
verified ·
1 Parent(s): 053e7a3

force recovery: methods, evaluation, debug log

Browse files
Files changed (1) hide show
  1. results.html +11 -9
results.html CHANGED
@@ -69,11 +69,10 @@ better of the two. Nothing but the image→gradient step differs.</p>
69
  <figure><img src="assets/truncation.png" alt="truncated presses">
70
  <figcaption>What the row above excludes. A press is <b>truncated</b> when its
71
  contact core reaches an image border: part of the indentation is outside the
72
- frame, the free-boundary solve has a slope running off the edge and nothing on
73
- the far side to stop it, and the height ramps. 14.5% of truncated frames
74
- reconstruct deeper than the 4.25&nbsp;mm gel is thick, against 0%
75
- of frames imaged whole. Their depth is not identifiable from the image, so no
76
- reconstruction can fix them.</figcaption></figure>
77
 
78
  <p>Excluding them is what the headline row buys: on GelSight Mini, CNC presses, 0-20 N
79
  calibration-free scores ρ&nbsp;0.996
@@ -84,15 +83,14 @@ evaluation does not exist</b>: no dataset here contains that many fully imaged
84
  presses.</p>
85
 
86
  <h3>All frames</h3>
87
- <p class="dim">The same protocol without that exclusion, for reference.</p>
88
  <div class="tablewrap"><table><thead><tr><th>dataset</th><th>n</th><th>LUT</th><th>calibration-free</th><th>shuffle floor</th><th>groups fitted</th></tr></thead><tbody>
89
  <tr><td>GelSight Mini, CNC presses, 0-20 N</td><td>2786</td><td>0.660</td><td><b>0.863</b></td><td class='dim'>+0.026 / +0.019</td><td class='dim'>6/6</td></tr><tr><td>FoTa cnc_Mini</td><td>2499</td><td>0.401</td><td><b>0.491</b></td><td class='dim'>+0.060 / +0.047</td><td class='dim'>6/6</td></tr><tr><td>FEATS (MARKER gel)</td><td>390</td><td><b>0.775</b></td><td>0.685</td><td class='dim'>-0.003 / +0.067</td><td class='dim'>4/6</td></tr><tr><td>Sparsh / Meta, 10 gel pads</td><td>960</td><td>0.456</td><td><b>0.588</b></td><td class='dim'>+0.092 / +0.099</td><td class='dim'>8/8</td></tr><tr><td>FeelAnyForce</td><td>5410</td><td>0.850</td><td><b>0.862</b></td><td class='dim'>+0.640 / +0.636</td><td class='dim'>42/42</td></tr></tbody></table></div>
90
 
91
  <p class="dim">The shuffle floor is an absolute ρ — what this protocol scores
92
  when the labels are permuted inside each group. Subtract it before reading a
93
- cell. React's own production number carries a fitted position gain field on top
94
- and is not in this table for that reason; it is on the
95
- <a href="method.html">method</a> page.</p>
96
 
97
  <figure><img src="assets/pred_vs_gt.png" alt="predicted vs ground-truth force">
98
  <figcaption>Held-out prediction against ground truth, shared axes per row.
@@ -114,5 +112,9 @@ frames, mean difference 0.84&nbsp;N. <b>The published dataset still
114
  carries the LUT column</b>: switching it needs 36 episodes reprocessed.</p>
115
 
116
  <h2>Error analysis</h2>
 
 
 
 
117
  <figure><img src="assets/errors_cnc_mini_26.png" alt="cnc_mini_26 errors"><figcaption>GelSight Mini CNC — the ten worst held-out frames and, as a control, the five best. Relative error is |pred−true| over the dataset's force span (19.50&nbsp;N); median 1.1%, p90 3.7%, worst 11.6%.</figcaption></figure><figure><img src="assets/errors_cnc.png" alt="cnc errors"><figcaption>FoTa cnc_Mini — the ten worst held-out frames and, as a control, the five best. Relative error is |pred−true| over the dataset's force span (4.01&nbsp;N); median 3.8%, p90 10.9%, worst 48.3%.</figcaption></figure><figure><img src="assets/errors_feats.png" alt="feats errors"><figcaption>FEATS (marker) — the ten worst held-out frames and, as a control, the five best. Relative error is |pred−true| over the dataset's force span (34.11&nbsp;N); median 5.7%, p90 22.1%, worst 67.9%.</figcaption></figure><figure><img src="assets/errors_sparsh.png" alt="sparsh errors"><figcaption>Sparsh / Meta — the ten worst held-out frames and, as a control, the five best. Relative error is |pred−true| over the dataset's force span (0.98&nbsp;N); median 3.1%, p90 12.0%, worst 30.0%.</figcaption></figure><figure><img src="assets/errors_faf.png" alt="faf errors"><figcaption>FeelAnyForce — the ten worst held-out frames and, as a control, the five best. Relative error is |pred−true| over the dataset's force span (17.43&nbsp;N); median 1.6%, p90 12.7%, worst 48.1%.</figcaption></figure>
118
  </div></body></html>
 
69
  <figure><img src="assets/truncation.png" alt="truncated presses">
70
  <figcaption>What the row above excludes. A press is <b>truncated</b> when its
71
  contact core reaches an image border: part of the indentation is outside the
72
+ frame, so the free-boundary solve runs off the edge with nothing beyond it to
73
+ stop the ramp. 14.5% of truncated frames reconstruct deeper than the
74
+ 4.25&nbsp;mm gel is thick, against 0% of frames imaged whole.
75
+ Their depth is not identifiable, so no reconstruction can fix them.</figcaption></figure>
 
76
 
77
  <p>Excluding them is what the headline row buys: on GelSight Mini, CNC presses, 0-20 N
78
  calibration-free scores ρ&nbsp;0.996
 
83
  presses.</p>
84
 
85
  <h3>All frames</h3>
86
+ <p class="dim">The same protocol without that exclusion.</p>
87
  <div class="tablewrap"><table><thead><tr><th>dataset</th><th>n</th><th>LUT</th><th>calibration-free</th><th>shuffle floor</th><th>groups fitted</th></tr></thead><tbody>
88
  <tr><td>GelSight Mini, CNC presses, 0-20 N</td><td>2786</td><td>0.660</td><td><b>0.863</b></td><td class='dim'>+0.026 / +0.019</td><td class='dim'>6/6</td></tr><tr><td>FoTa cnc_Mini</td><td>2499</td><td>0.401</td><td><b>0.491</b></td><td class='dim'>+0.060 / +0.047</td><td class='dim'>6/6</td></tr><tr><td>FEATS (MARKER gel)</td><td>390</td><td><b>0.775</b></td><td>0.685</td><td class='dim'>-0.003 / +0.067</td><td class='dim'>4/6</td></tr><tr><td>Sparsh / Meta, 10 gel pads</td><td>960</td><td>0.456</td><td><b>0.588</b></td><td class='dim'>+0.092 / +0.099</td><td class='dim'>8/8</td></tr><tr><td>FeelAnyForce</td><td>5410</td><td>0.850</td><td><b>0.862</b></td><td class='dim'>+0.640 / +0.636</td><td class='dim'>42/42</td></tr></tbody></table></div>
89
 
90
  <p class="dim">The shuffle floor is an absolute ρ — what this protocol scores
91
  when the labels are permuted inside each group. Subtract it before reading a
92
+ cell. React's production number adds a fitted position gain field and is
93
+ therefore on the <a href="method.html">method</a> page instead.</p>
 
94
 
95
  <figure><img src="assets/pred_vs_gt.png" alt="predicted vs ground-truth force">
96
  <figcaption>Held-out prediction against ground truth, shared axes per row.
 
112
  carries the LUT column</b>: switching it needs 36 episodes reprocessed.</p>
113
 
114
  <h2>Error analysis</h2>
115
+ <p>The ten worst held-out frames of each dataset reconstruct as well as the
116
+ five best — same gradient dipoles, same compact depth, no ramping. The residual
117
+ is in the depth→force fit, not in image→depth, so a better reconstruction will
118
+ not move these frames.</p>
119
  <figure><img src="assets/errors_cnc_mini_26.png" alt="cnc_mini_26 errors"><figcaption>GelSight Mini CNC — the ten worst held-out frames and, as a control, the five best. Relative error is |pred−true| over the dataset's force span (19.50&nbsp;N); median 1.1%, p90 3.7%, worst 11.6%.</figcaption></figure><figure><img src="assets/errors_cnc.png" alt="cnc errors"><figcaption>FoTa cnc_Mini — the ten worst held-out frames and, as a control, the five best. Relative error is |pred−true| over the dataset's force span (4.01&nbsp;N); median 3.8%, p90 10.9%, worst 48.3%.</figcaption></figure><figure><img src="assets/errors_feats.png" alt="feats errors"><figcaption>FEATS (marker) — the ten worst held-out frames and, as a control, the five best. Relative error is |pred−true| over the dataset's force span (34.11&nbsp;N); median 5.7%, p90 22.1%, worst 67.9%.</figcaption></figure><figure><img src="assets/errors_sparsh.png" alt="sparsh errors"><figcaption>Sparsh / Meta — the ten worst held-out frames and, as a control, the five best. Relative error is |pred−true| over the dataset's force span (0.98&nbsp;N); median 3.1%, p90 12.0%, worst 30.0%.</figcaption></figure><figure><img src="assets/errors_faf.png" alt="faf errors"><figcaption>FeelAnyForce — the ten worst held-out frames and, as a control, the five best. Relative error is |pred−true| over the dataset's force span (17.43&nbsp;N); median 1.6%, p90 12.7%, worst 48.1%.</figcaption></figure>
120
  </div></body></html>