CompressedGemma commited on
Commit
cd8b7c9
·
verified ·
1 Parent(s): 399da0d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +55 -3
README.md CHANGED
@@ -389,13 +389,67 @@ frozen — so the block-DC residual collapses to the `fp16` storage floor
389
  <10 ms per tensor). It spends **no** SSE budget (the pin can only lower
390
  unweighted block SSE) and runs identically on the CPU and CUDA paths.
391
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
392
  **DC/vesica drain knob map (and where they act):**
393
 
394
  | Knob | Config field | Stage | Cost model |
395
  | :--- | :--- | :--- | :--- |
396
  | Spectral shaping | `dc_lambda`, `vw_lambda` (λ_dc=λ_vw=1.0 default) | Phases 1–4 (in-objective) | arises inside the weighted objective itself |
397
  | Thermal λ-tilt drain | `therm_passes`, `therm_lambda_tilt` | Phase 4.5 (opt-in) | spends SSE under `sse_budget`, seeded & deterministic |
398
- | Zero-DC Pin | `pin_dc` | Phase 4.7 (opt-in) | **exact & free** — SSE non-increasing, DC → fp16 floor |
399
  | Row-aware carry nudge | `apply_dc_carry`, `dc_decay`, `sse_budget`, `carry_cumulative` | Phase 4.8 (on by default) | exactly bounded per-block budget, residual funneled downstream |
400
 
401
  For maximal coherent-residual drainage enable thermal (with tilt) + the
@@ -445,5 +499,3 @@ stats = write_quantized_gguf(
445
  )
446
  print(stats)
447
  ```
448
-
449
- ---
 
389
  <10 ms per tensor). It spends **no** SSE budget (the pin can only lower
390
  unweighted block SSE) and runs identically on the CPU and CUDA paths.
391
 
392
+ **Zero-DC Pin for the format quantizers (Q4_0 / Q3_K):** those formats
393
+ carry a single scale per block and no additive degree of freedom, so the
394
+ exact closed-form re-solve is degenerate there. Instead the pin performs a
395
+ deterministic **code-flip exchange**: each round picks, per block, the
396
+ flip (direction + position) with the best `ΔSSE / |ΔDC|` ratio and applies
397
+ it until the block has spent at most `pin_budget` of its starting SSE or
398
+ the residual reaches the floor of half a flip quantum. Measured on real
399
+ tensors (blk.0, CUDA path):
400
+
401
+ | Format | Config | RMSE | Block-DC RMS |
402
+ | :--- | :--- | :---: | :---: |
403
+ | `Q3_K` (`blk.0.ffn_down`) | base | $1.554\times10^{-3}$ | $1.96\times10^{-2}$ |
404
+ | `Q3_K` + pin 5% | `pin_budget=0.05` | $1.560\times10^{-3}$ (+0.4%) | $3.55\times10^{-3}$ (**−82%**) |
405
+ | `Q4_0` (`blk.0.attn_qkv`) | base | $1.223\times10^{-3}$ | $5.82\times10^{-3}$ |
406
+ | `Q4_0` + pin 5% | `pin_budget=0.05` | $1.231\times10^{-3}$ (+0.7%) | $3.36\times10^{-3}$ (−42%) |
407
+ | `Q4_0` + pin 10% | `pin_budget=0.10` | $1.239\times10^{-3}$ (+1.3%) | $2.18\times10^{-3}$ (−63%) |
408
+
409
+ `pin_budget` (default 0.05, `0` disables the spend) bounds the per-block
410
+ SSE the exchange may spend; the Q2_K closed-form pin never consults it.
411
+ The pin is bit-deterministic, applies after thermal refinement, runs
412
+ fully on-GPU on the CUDA paths (`pin_dc_codes_uniform_cuda` in
413
+ `spectral.py`) and on NumPy on the CPU paths — Q3_K's residual lands near
414
+ the DC floor because its signed `sc` unit grid has DC slack on both sides,
415
+ while Q4_0's unsigned `[0, 15]` nibble grid is bounded by the code-16
416
+ edge and settles higher.
417
+
418
+ The same single invocation covers Q2_K and the format quantizers —
419
+ `--device cuda` (or `--device auto`, which picks CUDA when available)
420
+ routes the format pipelines to the CUDA path and the pin runs on-GPU
421
+ automatically, no extra flags needed:
422
+
423
+ ```bash
424
+ # Pure Q3_K, full CUDA pipeline (thermal + on-GPU zero-DC pin):
425
+ python3 -m hpc_quantize_v2 \
426
+ --gguf model.gguf \
427
+ --format q3_k \
428
+ --device cuda \
429
+ --imatrix-gguf imatrix.gguf \
430
+ --thermal 4 \
431
+ --pin-dc \
432
+ --output model-Q3K.gguf \
433
+ --verbose
434
+
435
+ # Pure Q4_0 (pin included via --pin-dc):
436
+ python3 -m hpc_quantize_v2 --gguf model.gguf --format q4_0 --device cuda --pin-dc --output model-Q4_0.gguf
437
+ ```
438
+
439
+ In the GUI, tick **Pin DC residual to 0** and pick the CUDA device from the
440
+ device selector — the checkbox enables the pin for whichever format each
441
+ tensor is quantized to (closed-form re-solve for Q2_K superblocks,
442
+ code-flip exchange for Q4_0/Q3_K blocks). `pin_budget` itself is an
443
+ `HPCConfig` field (0.05 default) for API users, GUI/CLI defaults keep the
444
+ sensible order of magnitude.
445
+
446
  **DC/vesica drain knob map (and where they act):**
447
 
448
  | Knob | Config field | Stage | Cost model |
449
  | :--- | :--- | :--- | :--- |
450
  | Spectral shaping | `dc_lambda`, `vw_lambda` (λ_dc=λ_vw=1.0 default) | Phases 1–4 (in-objective) | arises inside the weighted objective itself |
451
  | Thermal λ-tilt drain | `therm_passes`, `therm_lambda_tilt` | Phase 4.5 (opt-in) | spends SSE under `sse_budget`, seeded & deterministic |
452
+ | Zero-DC Pin | `pin_dc`, `pin_budget` | Phase 4.7 (opt-in) | Q2_K: **exact & free** (closed-form, SSE non-increasing) · Q4_0/Q3_K: code-flip exchange bounded by `pin_budget` of block SSE |
453
  | Row-aware carry nudge | `apply_dc_carry`, `dc_decay`, `sse_budget`, `carry_cumulative` | Phase 4.8 (on by default) | exactly bounded per-block budget, residual funneled downstream |
454
 
455
  For maximal coherent-residual drainage enable thermal (with tilt) + the
 
499
  )
500
  print(stats)
501
  ```