Dongyun Zou commited on
Add B200 v-kmeans vsmooth ablation results to nunchux_low_bit_attention_blackwell.html
Browse files
blackwell_attention_slides/nunchux_low_bit_attention_blackwell.html
CHANGED
|
@@ -312,12 +312,23 @@ prmt.b32 P, p0, p1, 0x5410;</pre>
|
|
| 312 |
|
| 313 |
<section class="slide">
|
| 314 |
<div class="eyebrow">B200 / v-smooth ablation</div>
|
| 315 |
-
<h2>
|
| 316 |
-
<
|
| 317 |
-
|
| 318 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 319 |
</div>
|
| 320 |
-
<p class="math-note">
|
| 321 |
</section>
|
| 322 |
|
| 323 |
<section class="slide">
|
|
|
|
| 312 |
|
| 313 |
<section class="slide">
|
| 314 |
<div class="eyebrow">B200 / v-smooth ablation</div>
|
| 315 |
+
<h2>V-kmeans + V-smooth: Stable Accuracy Gain</h2>
|
| 316 |
+
<table>
|
| 317 |
+
<thead>
|
| 318 |
+
<tr><th>Config</th><th>Latency</th><th>Speedup vs bf16</th><th>TFLOPs</th><th>RMSE</th><th>Cosine</th></tr>
|
| 319 |
+
</thead>
|
| 320 |
+
<tbody>
|
| 321 |
+
<tr><td>bf16 FA4</td><td>68.793 ms</td><td>1.000x</td><td>1422.4</td><td>0</td><td>1.000000</td></tr>
|
| 322 |
+
<tr><td>no v-kmeans / no v-smooth</td><td><span class="red">51.800 ms</span></td><td><span class="red">1.329x</span></td><td><span class="red">1890.3</span></td><td>8.704e-3</td><td>0.999918</td></tr>
|
| 323 |
+
<tr><td>v-kmeans + v-smooth</td><td>57.506 ms</td><td>1.197x</td><td>1702.7</td><td><span class="red">7.997e-3</span></td><td><span class="red">0.999928</span></td></tr>
|
| 324 |
+
</tbody>
|
| 325 |
+
</table>
|
| 326 |
+
<div class="metric">
|
| 327 |
+
<div class="box"><div class="num">1.197x</div><div class="label">still faster than bf16 FA4 with v-kmeans + v-smooth</div></div>
|
| 328 |
+
<div class="box"><div class="num">8.1%</div><div class="label">RMSE reduction: 8.704e-3 -> 7.997e-3</div></div>
|
| 329 |
+
<div class="box"><div class="num">1.110x</div><div class="label">latency cost vs no v-kmeans / no v-smooth</div></div>
|
| 330 |
</div>
|
| 331 |
+
<p class="math-note">B200, full Wan2.1 p1 set: 18 attention-only workloads. QK-INT8 with Hadamard + Q per-token + K block-128, PV-FP8, exp approximation ON (linmant). Metrics are vs bf16 FA4 attention output.</p>
|
| 332 |
</section>
|
| 333 |
|
| 334 |
<section class="slide">
|