Update README.md
Browse files
README.md
CHANGED
|
@@ -389,13 +389,67 @@ frozen — so the block-DC residual collapses to the `fp16` storage floor
|
|
| 389 |
<10 ms per tensor). It spends **no** SSE budget (the pin can only lower
|
| 390 |
unweighted block SSE) and runs identically on the CPU and CUDA paths.
|
| 391 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 392 |
**DC/vesica drain knob map (and where they act):**
|
| 393 |
|
| 394 |
| Knob | Config field | Stage | Cost model |
|
| 395 |
| :--- | :--- | :--- | :--- |
|
| 396 |
| Spectral shaping | `dc_lambda`, `vw_lambda` (λ_dc=λ_vw=1.0 default) | Phases 1–4 (in-objective) | arises inside the weighted objective itself |
|
| 397 |
| Thermal λ-tilt drain | `therm_passes`, `therm_lambda_tilt` | Phase 4.5 (opt-in) | spends SSE under `sse_budget`, seeded & deterministic |
|
| 398 |
-
| Zero-DC Pin | `pin_dc` | Phase 4.7 (opt-in) | **exact & free**
|
| 399 |
| Row-aware carry nudge | `apply_dc_carry`, `dc_decay`, `sse_budget`, `carry_cumulative` | Phase 4.8 (on by default) | exactly bounded per-block budget, residual funneled downstream |
|
| 400 |
|
| 401 |
For maximal coherent-residual drainage enable thermal (with tilt) + the
|
|
@@ -445,5 +499,3 @@ stats = write_quantized_gguf(
|
|
| 445 |
)
|
| 446 |
print(stats)
|
| 447 |
```
|
| 448 |
-
|
| 449 |
-
---
|
|
|
|
| 389 |
<10 ms per tensor). It spends **no** SSE budget (the pin can only lower
|
| 390 |
unweighted block SSE) and runs identically on the CPU and CUDA paths.
|
| 391 |
|
| 392 |
+
**Zero-DC Pin for the format quantizers (Q4_0 / Q3_K):** those formats
|
| 393 |
+
carry a single scale per block and no additive degree of freedom, so the
|
| 394 |
+
exact closed-form re-solve is degenerate there. Instead the pin performs a
|
| 395 |
+
deterministic **code-flip exchange**: each round picks, per block, the
|
| 396 |
+
flip (direction + position) with the best `ΔSSE / |ΔDC|` ratio and applies
|
| 397 |
+
it until the block has spent at most `pin_budget` of its starting SSE or
|
| 398 |
+
the residual reaches the floor of half a flip quantum. Measured on real
|
| 399 |
+
tensors (blk.0, CUDA path):
|
| 400 |
+
|
| 401 |
+
| Format | Config | RMSE | Block-DC RMS |
|
| 402 |
+
| :--- | :--- | :---: | :---: |
|
| 403 |
+
| `Q3_K` (`blk.0.ffn_down`) | base | $1.554\times10^{-3}$ | $1.96\times10^{-2}$ |
|
| 404 |
+
| `Q3_K` + pin 5% | `pin_budget=0.05` | $1.560\times10^{-3}$ (+0.4%) | $3.55\times10^{-3}$ (**−82%**) |
|
| 405 |
+
| `Q4_0` (`blk.0.attn_qkv`) | base | $1.223\times10^{-3}$ | $5.82\times10^{-3}$ |
|
| 406 |
+
| `Q4_0` + pin 5% | `pin_budget=0.05` | $1.231\times10^{-3}$ (+0.7%) | $3.36\times10^{-3}$ (−42%) |
|
| 407 |
+
| `Q4_0` + pin 10% | `pin_budget=0.10` | $1.239\times10^{-3}$ (+1.3%) | $2.18\times10^{-3}$ (−63%) |
|
| 408 |
+
|
| 409 |
+
`pin_budget` (default 0.05, `0` disables the spend) bounds the per-block
|
| 410 |
+
SSE the exchange may spend; the Q2_K closed-form pin never consults it.
|
| 411 |
+
The pin is bit-deterministic, applies after thermal refinement, runs
|
| 412 |
+
fully on-GPU on the CUDA paths (`pin_dc_codes_uniform_cuda` in
|
| 413 |
+
`spectral.py`) and on NumPy on the CPU paths — Q3_K's residual lands near
|
| 414 |
+
the DC floor because its signed `sc` unit grid has DC slack on both sides,
|
| 415 |
+
while Q4_0's unsigned `[0, 15]` nibble grid is bounded by the code-16
|
| 416 |
+
edge and settles higher.
|
| 417 |
+
|
| 418 |
+
The same single invocation covers Q2_K and the format quantizers —
|
| 419 |
+
`--device cuda` (or `--device auto`, which picks CUDA when available)
|
| 420 |
+
routes the format pipelines to the CUDA path and the pin runs on-GPU
|
| 421 |
+
automatically, no extra flags needed:
|
| 422 |
+
|
| 423 |
+
```bash
|
| 424 |
+
# Pure Q3_K, full CUDA pipeline (thermal + on-GPU zero-DC pin):
|
| 425 |
+
python3 -m hpc_quantize_v2 \
|
| 426 |
+
--gguf model.gguf \
|
| 427 |
+
--format q3_k \
|
| 428 |
+
--device cuda \
|
| 429 |
+
--imatrix-gguf imatrix.gguf \
|
| 430 |
+
--thermal 4 \
|
| 431 |
+
--pin-dc \
|
| 432 |
+
--output model-Q3K.gguf \
|
| 433 |
+
--verbose
|
| 434 |
+
|
| 435 |
+
# Pure Q4_0 (pin included via --pin-dc):
|
| 436 |
+
python3 -m hpc_quantize_v2 --gguf model.gguf --format q4_0 --device cuda --pin-dc --output model-Q4_0.gguf
|
| 437 |
+
```
|
| 438 |
+
|
| 439 |
+
In the GUI, tick **Pin DC residual to 0** and pick the CUDA device from the
|
| 440 |
+
device selector — the checkbox enables the pin for whichever format each
|
| 441 |
+
tensor is quantized to (closed-form re-solve for Q2_K superblocks,
|
| 442 |
+
code-flip exchange for Q4_0/Q3_K blocks). `pin_budget` itself is an
|
| 443 |
+
`HPCConfig` field (0.05 default) for API users, GUI/CLI defaults keep the
|
| 444 |
+
sensible order of magnitude.
|
| 445 |
+
|
| 446 |
**DC/vesica drain knob map (and where they act):**
|
| 447 |
|
| 448 |
| Knob | Config field | Stage | Cost model |
|
| 449 |
| :--- | :--- | :--- | :--- |
|
| 450 |
| Spectral shaping | `dc_lambda`, `vw_lambda` (λ_dc=λ_vw=1.0 default) | Phases 1–4 (in-objective) | arises inside the weighted objective itself |
|
| 451 |
| Thermal λ-tilt drain | `therm_passes`, `therm_lambda_tilt` | Phase 4.5 (opt-in) | spends SSE under `sse_budget`, seeded & deterministic |
|
| 452 |
+
| Zero-DC Pin | `pin_dc`, `pin_budget` | Phase 4.7 (opt-in) | Q2_K: **exact & free** (closed-form, SSE non-increasing) · Q4_0/Q3_K: code-flip exchange bounded by `pin_budget` of block SSE |
|
| 453 |
| Row-aware carry nudge | `apply_dc_carry`, `dc_decay`, `sse_budget`, `carry_cumulative` | Phase 4.8 (on by default) | exactly bounded per-block budget, residual funneled downstream |
|
| 454 |
|
| 455 |
For maximal coherent-residual drainage enable thermal (with tilt) + the
|
|
|
|
| 499 |
)
|
| 500 |
print(stats)
|
| 501 |
```
|
|
|
|
|
|