Update README.md
5defcdf verified 2 days ago
30.4 kB
---
license: mit
---

![image](https://cdn-uploads.huggingface.co/production/uploads/69de1a493a21cb7c029dc694/BvRBBP2uBGxYhQnsxH6jZ.png)
GUI screencap

# Developer Note

This requires a CUDA GPU, expect a 50GB model to take around 3-4 hours to quantize on a 3060 12GB.

I prefer to llama-quantize to 8_0, then to generate a imatrix from that 8_0, and finally to quantize the 8_0 file to Q2K.
# Lattice Repair
This is NOT yet included in the quantizers automated pipeline but DOES function.
It is invoked by doing the following:
python3 lattice_repair.py \
        --source BF16GGUF.gguf \
        --quant QUANTGGUF.gguf \
        --output OUTPUT.gguf \
        --force \
        --thresh 1.0 \
        --fix-frames \
        --dc-lambda 5.0 \
        --ulp-radius 8 \
        --refit-batch 512 \
        --catalog ./lr_moved.csv
      Or wherever you want to point the weight dumps.
Output looks like this:

  blk.11.attn_output.weight [Q2_K] outliers 60829/31457280 moved 26504 (blocks 23737/122880) frame-limited 34410 in 28363 blocks, frames fixed 13822 | rmse 4.2478e-03->4.1842e-03 max|e| 5.834e-01->3.314e-01 mean|DC| 1.380e-03->4.602e-03

Numbers to pay attention to: rmse 4.2478e-03->4.1842e-03          max|e| 5.834e-01->3.314e-01.

RMSE ends up cut a little, Max Error is absolutely shattered and cut in half recovering critical function, at the cost of DC.

DC is my heuristic to assemble the LLM with a initial pass, then Lattice Repair polishes things off.

Q2K and Q3K don't really have a lot of space to move, so the weights are mostly recalculated with fix-frame to bring them closer to the source GGUF.


# HPC-Quantize V2

## Holographic Phase Contraction, Geometry-Preserving Weight Analogs, and Algebraic Coupled Bilinear Factorization for Ultra-Low-Bit (`Q2_K`) LLM Quantization
**HPC-Quantize V2** is a standalone Python/NumPy ultra-low-bit quantization engine and GGUF re-quantizer (with a real-time telemetry desktop GUI in `gui.py`) built for **2-bit class (`Q2_K`, 2.625 bpw)** compression of large language models.
> **Core thesis:** At 2 bits per weight (4 levels $q \in \{0,1,2,3\}$ and 4-bit sub-block scales $Ls, Lm \in \{0..15\}$), quantization is not an independent rounding problem and should not rely on brute-force grid scans. It is a **coupled bilinear factorization and global sequence-optimization problem** best solved by extracting discrete lattice coordinates from **geometry-preserving weight analogs**, projecting them onto the true weights via **closed-form orthogonalized sufficient statistics**, and optimizing cross-block state trajectories with a **6-state sequential Sieve and 36-state Viterbi dynamic program**.
HPC-Quantize V2 (`hpc_quantize_v2/` + `gui.py`) has **zero C or `ctypes` dependencies**, reads and writes stock `llama.cpp`-compatible `.gguf` models and `.gguf`/`.dat` importance matrices (`iMatrix`), scales across all CPU cores with **bit-for-bit determinism**, and requires no custom inference kernels.
---
## Key Highlights in V2
1. **Geometry-Preserving Weight Analogs (`hpc_quantize_v2/analog.py`)**:
   Constructs a portfolio of surrogate weight blocks $\tilde{x}$ that preserve exact sub-block first and second importance-weighted moments ($\mu_j, \sigma_j^2$), rank order, unit sub-block direction ($\cos\theta = 1$), total block energy, and the $2\times 2$ D6/Vesica fold Gram matrix while regularizing heavy-tailed outliers toward the 4-level `Q2_K` lattice.
2. **Coupled Orthogonal-Basis Bilinear `Q2_K` Factorization (`hpc_quantize_v2/factored.py`)**:
   Replaces independent 1-D scale/offset heuristics with an orthogonalized sub-block decomposition $\hat{x}_{j,k} = s_j(q_{j,k} - \bar{q}_j) + (s_j\bar{q}_j - t_j)$, coupled offset compensation $t_j^{\text{coupled}} = \max(0, (d \cdot Ls_j \cdot S_{wq,j} - S_{x,j})/S_{w,j})$, rank-1 algebraic 1-D code refinement ($S_{vv} - S_{lv}^2/S_{l2}$), and a single-pass vectorized 36-seed candidate lattice evaluator (`81x` faster than per-seed loops).
3. **Algebraic Coupled Phase-4 Block Assembly (`hpc_quantize_v2/pipeline.py`)**:
   Replaces legacy brute-force $16\times 16$ sub-block scans, $9\times 9$ weight-level ULP loops, and 256-pass greedy code searches with closed-form sufficient-statistic projections (`1,200–1,475 blocks/sec` in Phase 4, achieving **+2.64% lower objective error** than brute-force Phase 4 in **1/9th the time**).
4. **Bit-Identical Multi-Core Execution (`--workers 0`)**:
    Distributes block-independent candidate and analog solves across all CPU cores via zero-copy `fork` shared memory while keeping cross-block Sieve, 36-state Viterbi, and row-aware DC carry strictly sequential — producing **100% bit-for-bit identical** GGUF output regardless of worker count.
5. **Real-Time Telemetry GUI (`gui.py`) & Direct GGUF-to-GGUF Pipeline (`hpc_quantize_v2/gguf.py`)**:
    Ingests `F32`, `F16`, `BF16`, `Q8_0`, `Q4_0`, and `Q2_K` source GGUFs plus `llama.cpp` iMatrix files, displays live intra-tensor stage/block progress, dual **Tensor ETA** and block-weighted **Whole-Model ETA**, live `RMSE`/`RMSE/σ`, Sieve temperature, and floor hits, and emits a stock `llama.cpp` `Q2_K` GGUF.
6. **The Coherent-Residual Drain & the `Q2_K` Free Lunch (`spectral.py`, `anneal.py`, `q2k.py`)**:
    A four-stage drain of the *coherent* error channel — in-objective DC/vesica shaping, an opt-in thermal `λ`-tilt drain, a row-aware sequential DC carry nudge, and the closed-form **Zero-DC Pin** (Phase 4.7): a pure $O(n)$ re-solve of the superblock scale pair on the exact `Σe = 0` line that **drops the block-DC residual by ~94% while slightly *lowering* RMSE** — for free, with zero SSE budget.
---

## Why Standard `Q2_K` Quantization Breaks Down
A `Q2_K` superblock encodes 256 weights in 84 bytes (16 sub-blocks of 16 weights each):

$$
\hat{x}_{16j + k} = d \cdot Ls_j \cdot q_{16j + k} - d_{\min} \cdot Lm_j,
\qquad
q \in \{0,1,2,3\}, \quad Ls_j, Lm_j \in \{0,\ldots,15\}, \quad d, d_{\min} \in \mathbb{F}_{16}.
$$
When factoring raw LLM weights directly into this bilinear form, three structural failure modes occur:
1. **Intra-sub-block outlier collapse**: A single outlier weight in a 16-weight sub-block stretches the span $x_{\max} - x_{\min}$, forcing 13–15 of the 16 weights into a single bin ($q=1$ or $q=2$) and reducing effective precision below 1 bit.
2. **Inter-sub-block scale crushing**: A single high-dispersion sub-block inflates the shared superblock scale $d$, forcing $Ls_j \in \{1, 2\}$ across the remaining 15 sub-blocks.
3. **Scale-offset cross-term neglect**: Because $q_{j,k} \in \{0,1,2,3\}$ is non-negative, its importance-weighted mean $\bar{q}_j = \frac{\sum_k w_{j,k} q_{j,k}}{\sum_k w_{j,k}}$ is strictly positive. Rounding the scale $s_j \to d \cdot Ls_j$ shifts the sub-block mean by $(d \cdot Ls_j - s_j)\bar{q}_j$. Factoring $s_j \approx d \cdot Ls_j$ and $t_j \approx d_{\min} \cdot Lm_j$ independently leaves this mean shift uncompensated.
---
## Mathematical Architecture of HPC-Quantize V2
### 1. Geometry-Preserving Weight Analogs (`analog.py`)
To prevent outliers from collapsing the discrete code assignment $q \in \{0,1,2,3\}$ while respecting the true linear-layer geometry, V2 constructs a portfolio of **geometry-preserving analogs** $\tilde{x}$ of each eligible weight block before extracting candidate lattice coordinates:
#### A. Sub-block Iso-Moment Quantile-Companded Analog (`subblock_isomoment_analog`)
For each 16-weight sub-block $j$ with importance weights $w_{j,k}$:

$$
\mu_j = \frac{\sum_k w_{j,k} x_{j,k}}{\sum_k w_{j,k}},
\qquad
\sigma_j^2 = \frac{\sum_k w_{j,k}(x_{j,k} - \mu_j)^2}{\sum_k w_{j,k}},
\qquad
z_{j,k} = \frac{x_{j,k} - \mu_j}{\sigma_j}.
$$
Standardized coordinates are mapped through an importance-weighted hyperbolic transport blended with ideal 4-level quantiles $q^{\text{ideal}}_r \in [-1.35, +1.35]$:

$$
\tilde{z}_{j,k} = (1 - \alpha)\,\tau \tanh\!\left(\frac{(w_{j,k}/\bar{w}_j)^\beta z_{j,k}}{\tau}\right) + \alpha\, q^{\text{ideal}}_{\text{rank}(z_{j,k})},
$$

and then **orthogonally projected back onto the exact sub-block sphere**:

$$
\tilde{x}_{j,k} = \mu_j + \sigma_j \frac{\tilde{z}_{j,k} - \bar{z}_j}{\sqrt{\frac{1}{W_j}\sum_k w_{j,k}(\tilde{z}_{j,k} - \bar{z}_j)^2}}.
$$

This preserves the exact sub-block weighted mean $\mu_j$, weighted variance $\sigma_j^2$, and rank ordering while equalizing spacing across the 4 quantization levels.

#### B. D6 / Vesica Gram-Preserving Fold Analog (`vesica_gram_analog`)
Decomposes the 256-weight superblock into symmetric (vesica) and antisymmetric (wave) fold modes:

$$
v_p = \frac{1}{2}(x_p + x_{p+128}),
\qquad
u_p = \frac{1}{2}(x_p - x_{p+128}),
\qquad p \in \{0,\ldots,127\}.
$$

Let $G \in \mathbb{R}^{2\times 2}$ be the importance-weighted covariance (Gram) matrix of $(v, u)$. V2 whitens $(v, u)$ via $G^{-1/2}$, applies smooth hyperbolic saturation along the principal axes, re-orthogonalizes, and colors back via $G^{1/2}$ so that:

$$
\operatorname{Gram}_w(\tilde{v}, \tilde{u}) = \operatorname{Gram}_w(v, u),
\qquad
\mathbb{E}_w[\tilde{v}] = \mathbb{E}_w[v],
\qquad
\mathbb{E}_w[\tilde{u}] = \mathbb{E}_w[u].
$$

#### C. Cross-Sub-block Scale-Lattice Analog (`subscale_lattice_analog`)
Tempers extreme sub-block standard deviations $\sigma_j \mapsto \sigma_j^\gamma \sigma_{\text{block}}^{1-\gamma}$ so all 16 sub-blocks fit cleanly into the shared 4-bit $\{0..15\}$ scale lattice while preserving every sub-block's unit direction $\hat{u}_j$ ($\cos\theta = 1$), mean $\mu_j$, and total superblock weighted energy.
---
### 2. Coupled Orthogonal-Basis Bilinear `Q2_K` Factorization (`factored.py`)

Given candidate discrete codes $q_{j,k} \in \{0,1,2,3\}$ (from raw starts and the analog portfolio), V2 projects the continuous and 4-bit scale parameters directly onto the **true weights** $x$:
1. **Orthogonalized Sub-block Basis**:
   Rewriting $\hat{x}_{j,k} = s_j(q_{j,k} - \bar{q}_j) + (s_j\bar{q}_j - t_j)$ diagonalizes the $2\times 2$ sub-block normal equations. The true sensitivity of sub-block $j$ to scale error $(s_j - d \cdot Ls_j)^2$ is the centered variance weight:
   $$
   V_{qq,j} = \sum_{k=0}^{15} w_{j,k}(q_{j,k} - \bar{q}_j)^2 = S_{qq,j} - S_{wq,j}\bar{q}_j.
   $$
2. **Monotone Rank-1 1-D Factorization (`factor_1d`)**:
   Factors $s_j \approx d \cdot Ls_j$ under weights $V_{qq,j}$ using multi-start Lloyd iterations followed by an exact vectorized rank-1 coordinate sweep maximizing $S_{lv}^2 / S_{l2}$ (since $\sum_j w_j(v_j - d^* L_j)^2 = S_{vv} - S_{lv}^2/S_{l2}$).
3. **Coupled Offset Target**:
   Once $(d, Ls_j)$ are chosen, the optimal offset for sub-block $j$ is shifted in closed form to absorb the scale rounding residual:
   $$
   t_j^{\text{coupled}} = \max\!\left(0,\; \frac{d \cdot Ls_j \cdot S_{wq,j} - S_{x,j}}{S_{w,j}}\right)
   = \max\!\left(0,\; t_j + \bar{q}_j(d \cdot Ls_j - s_j)\right),
   $$
   which is then factored into $d_{\min} \cdot Lm_j$ under weights $S_{w,j}$.
---
### 3. Batched Algebraic Phase-4 Assembly (`pipeline.py`)
Instead of brute-force $16\times 16$ sub-block scans and $9\times 9$ weight-level ULP searches, Phase 4 refines both the Viterbi-selected state and the analog-factored state across all blocks in vectorized batches using three exact algebraic identities:
1. **Closed-Form Coupled Sub-scale Projection**:
   From sufficient statistics $(V_{qq,j}, C_{qx,j}, S_{wq,j}, S_{x,j}, S_{w,j})$, the unconstrained scale minimizer is $s_j^* = \max(0, C_{qx,j}/V_{qq,j})$. Evaluating the local integer neighborhood $Ls_j \in \operatorname{round}(s_j^*/d) + \{-3,\ldots,+3\}$ paired with its closed-form coupled offset $Lm_j \in \operatorname{round}(t_j^*(Ls_j)/m) + \{-1,0,+1\}$ replaces $16\times 16 = 256$ brute-force pairs with $21$ algebraic states.
2. **Vesica + DC Augmented $2\times 2$ Cramér Solve & $O(1)$ Scalar Quadratic ULP Polish**:
   With $A_i = Ls_{i\gg 4} q_i$ and $B_i = Lm_{i\gg 4}$, the full extended objective (weighted SSE + Vesica fold pairs + DC carry) is an exact quadratic form in $(d, m)$:
   $$
   J(d, m) = \text{const} + \tilde{S}_{aa} d^2 - 2\tilde{S}_{ab} d m + \tilde{S}_{bb} m^2 - 2\tilde{S}_{xa} d + 2\tilde{S}_{xb} m,
   $$
   where $(\tilde{S}_{aa}, \tilde{S}_{ab}, \tilde{S}_{bb}, \tilde{S}_{xa}, \tilde{S}_{xb})$ include the Vesica pair sums $(A_p + A_{p+128}, B_p + B_{p+128}, x_p + x_{p+128})$ and the row DC sum $(\sum_i A_i, \sum_i B_i, \sum_i x_i + \text{carry})$. The continuous optimum $(d^*, m^*)$ is solved by $2\times 2$ Cramér's rule and the $\mathbb{F}_{16}$ ULP neighborhood is polished in $O(1)$ scalar arithmetic without touching the 256 weights.
3. **Closed-Form 1-D Quadratic Vesica/DC Code Projection**:
   Each weight's optimal discrete step $\Delta q_k^*$ under the combined weighted SSE + Vesica + DC objective is computed directly from the 1-D quadratic minimizer:
   $$
   \Delta q_k^* = \operatorname{round}\!\left(\frac{d_{\text{sub},k}\left(w_k e_k + \frac{\lambda_{\text{vw}}}{N}(e_k + e_{k\oplus 128}) + \frac{\lambda_{\text{dc}}}{N} D\right)}{\left(w_k + \frac{\lambda_{\text{vw}} + \lambda_{\text{dc}}}{N}\right) d_{\text{sub},k}^2}\right).
   $$
---
### 4. The Coherent-Residual Drain: DC & Vesica Error Handling (`spectral.py`, `anneal.py`, `q2k.py`)
#### A. Residual anatomy: why "SSE-optimal" is not enough
For a finished `Q2_K` superblock, the per-element error is
$$
e_i = x_i - \big(d \cdot Ls_{\lfloor i/16 \rfloor}\, q_i - d_{\min} \cdot Lm_{\lfloor i/16 \rfloor}\big),
\qquad i \in \{0,\ldots,255\}.
$$
SSE-optimal per-element quantizers are unbiased only *on average over huge ensembles*. At 2 bits per weight, with the `(d, d_min)` pair stored in `fp16`, the residual of a single superblock has two coherent components with scale far above the white-noise floor:
1. **Block DC** $D = \sum_{i=0}^{255} e_i$ — the systematic per-superblock offset error.
2. **Vesica pairs** $v_p = e_p + e_{p+128}$, $p \in \{0,\ldots,127\}$ — the first fold level of the dyadic/Hadamard error pyramid, i.e. the dominant low-frequency coherent residual shared by the symmetric halves of a superblock.
Measured on this stack: the *unpinned* pipeline leaves block-DC RMS $\approx 2.7\times 10^{-2}$ against a total RMSE of $\approx 4.1\times 10^{-3}$ — the coherent channel is **~7× louder than the entire random error**, and because it is coherent it *accumulates across layers* in inference instead of averaging out. V2 therefore treats draining $D$ and $v_p$ as a first-class problem, with four independent, composable mechanisms:
#### B. Mechanism 1 — In-objective spectral shaping (Phases 1–4, always on)
The block objective augments importance-weighted SSE with the DC/vesica fold penalty
$$
E = \sum_i w_i e_i^2 \;+\; \frac{\lambda_{\text{vw}}}{n} \sum_{p=0}^{127} v_p^2 \;+\; \frac{\lambda_{\text{dc}}}{n} \,(D + \text{carry})^2 ,
$$
(`dc_lambda`, `vw_lambda`; the full vectorized fold pyramid generalizes this to every dyadic level). All three algebraic Phase-4 solvers — the coupled sub-scale projection, the $2\times 2$ Cramér scale/offset solve, and the 1-D quadratic code descent — minimize this extended form exactly, so the *lattice solution space itself* is biased toward low coherent residual before any separate drain pass runs.
#### C. Mechanism 2 — Thermal `λ`-tilt drain (Phase 4.5, opt-in, `--thermal` + `--thermal-tilt`)
With `therm_passes > 0`, the exact heat-bath Gibbs chain sweeps the code lattice with the *extended* energy. Every accepted move is exact on the quantization lattice, so multiplying the spectral lambdas by the zero-residual tilt (`therm_lambda_tilt` ≥ 1) makes the chain deliberately **spend weighted SSE to buy coherent-residual drainage** — it walks $(\text{SSE}, D, v_p)$ along the exact Pareto frontier of the block. Endpoint behavior at large tilts: the coherent residual is driven to the lattice floor and the remaining error is (almost) spectrally white — measured on this model's `ssm_out`: **DC RMS −97% at tilt $10^3$ for ~8× SSE**. The pass is monotone per block (extended energy never regresses) and seeded/deterministic.
#### D. Mechanism 3 — Row-aware sequential DC carry nudge (Phase 4.8, on by default)
`_apply_dc_nudge` sweeps superblocks strictly left-to-right (row topology of the weight matrix, `row_width` blocks per row). Each block's codes are re-projected by `dc_nudge_codes` — a bounded-SSE-budget greedy descent that cancels the *incoming* DC carry `dc_decay · rolling` in exchange for as little objective cost as the `sse_budget` cap allows; whatever DC the block cannot cheaply cancel becomes part of the *rolling* carry handed to the next superblock (reset at row boundaries; `carry_cumulative` selects reset-to-current vs accumulate). Net effect: **the residual DC of the whole tensor gets funneled into whichever blocks have the cheapest 2-bit codes to absorb it**, instead of staying distributed uniformly. Cost is exactly bounded by construction; the sweep is deterministic and single-pass.
#### E. Mechanism 4 — The Zero-DC Pin, a.k.a. the "`Q2_K` Free Lunch" (Phase 4.7, opt-in, `pin_dc`)
This is a **pure closed-form bonus**: with codes, $Ls$, $Lm$ frozen at their final refined state, the remaining freedom $(d, m)$ (with $m \equiv d_{\min}$, the fp16 minimum field) is re-solved *subject to exact zero DC*. Two degrees of freedom, two constraints ($\min \sum e_i^2$, $\sum e_i = 0$) → an exact solution, no lattice search, no budget.
Write $a_i = Ls_{\lfloor i/16\rfloor} c_i$ and $b_i = Lm_{\lfloor i/16\rfloor}$ so $e_i = x_i - d\,a_i + m\,b_i$, and let $S_x = \sum_i x_i$, $S_a = \sum_i a_i$, $S_b = \sum_i b_i$. The constraint $\sum_i e_i = 0$ pins $m$ onto a line parameterized by $d$:
$$
m(d) = \frac{d\,S_a - S_x}{S_b}
\quad\Rightarrow\quad
e_i(d) = u_i - d\,v_i,
\quad
u_i = x_i - b_i \tfrac{S_x}{S_b},\;
v_i = a_i - b_i \tfrac{S_a}{S_b}.
$$
Along that line the block energy
$$
\sum_i e_i^2(d) = \|u\|^2 \;-\; 2d\,\langle u, v\rangle \;+\; d^2 \|v\|^2
$$
is a convex 1-D quadratic with exact minimizer
$$
d^* = \frac{\langle u, v\rangle}{\langle v, v\rangle},
\qquad
m^* = \frac{d^* S_a - S_x}{S_b}.
$$
**Geometric meaning:** $v$ is the component of the code vector $a$ orthogonal to the all-ones direction (the DC direction), and $u$ is the analogous component of the data — so $d^*$ is the exact OLS scale of orthogonally-projected data against orthogonally-projected codes. The pin is self-dominating: moving from the input $(d, m)$ to $(d^*, m^*)$ travels *along* the zero-DC line, so it is at worst the zero move — **SSE is non-increasing, $|D|$ is non-increasing, only the fp16 storage rounding of the `(d, m)` fields limits the final DC**. Blocks where the translation degree of freedom degenerates ($S_b = 0$ or $\langle v, v\rangle = 0$) keep their input parameters; the map is fp16-idempotent (re-pinning an already-pinned state is a no-op).
Its practical effect (live `blk.0.ssm_out.weight`, 8,192 superblocks, RTX 3060 path):

| Configuration | RMSE | Block-DC RMS | mean $\lvert D \rvert$ |
| :--- | :---: | :---: | :---: |
| Base pipeline (Phases 1–4 + carry nudge) | $4.068\times 10^{-3}$ | $2.72\times 10^{-2}$ | $1.96\times 10^{-2}$ |
| Base + Zero-DC Pin (Phase 4.7) | $4.045\times 10^{-3}$ | $1.70\times 10^{-3}$ | $1.36\times 10^{-3}$ |

i.e. **DC RMS −93.8%, RMSE −0.6%, at <10 ms for the whole tensor** (a handful of $O(n_B \cdot 256)$ reductions). The weighted iMatrix-objective does rise slightly (~5% on `wSSE` for the pinned sample) — that is the point: it is an *exchange*, trading part of the importance-weighted objective for a near-zero coherent channel, and it composes safely with the drain stages of Mechanisms 1–3 (recommended order: thermal `λ`-tilt drain → pin → row-aware carry nudge, which is exactly the pipeline's Stage order 4.5 → 4.7 → 4.8).
---
### 5. End-to-End HPC Pipeline
```text
Source Weights + Optional iMatrix
        │
        ▼
Kurtosis-Gated Clamp & Batched GGML Seeds (make_qkx2_batch / make_qp_batch)
        │
        ▼
Vectorized Superblock WLS + Multi-Core Analog-Guided Factored Candidate Field
        │
        ▼
6-State Sequential Sieve (Bounded Neighbor Parity Back-Action + Log-Slack)
        │
        ▼
36-State Viterbi Global Sequence Optimization (Free-Energy Bins + Manhattan Prior)
        │
        ▼
Batched Algebraic Phase-4 Coupled Assembly + Quadratic ULP Polish + Closed-Form Code Projection
        │
        ▼
Opt-In Thermal Drain (Exact Heat-Bath Sweep, Zero-Residual λ-Tilt — Phase 4.5)
        │
        ▼
Zero-DC Pin / "Q2_K Free Lunch" (Closed-Form dc=0 Re-Solve of (d, m) — Phase 4.7)
        │
        ▼
Row-Aware Sequential DC Residual Nudge & Monotone Local-Safety Floors (Phase 4.8)
        │
        ▼
Standard Q2_K GGUF Output
```
---

## Measured Benchmarks

Evaluated across 48 synthetic blocks (4 distributions) and a full row (47 superblocks = 12,032 weights) of `blk.0.ffn_down.weight` from `Raon.gguf` weighted by `imatrix.gguf`:

| Benchmark Suite | Mean Objective Reduction | Median Reduction | Min Reduction | Worse Cases |
| :--- | :---: | :---: | :---: | :---: |
| **Gaussian** ($\mathcal{N}(0, 0.02^2)$) | **+8.85%** | +8.45% | +3.17% | `0 / 12` |
| **Heavy-Tailed** ($\text{Student-}t_3$) | **+10.25%** | +8.30% | +5.04% | `0 / 12` |
| **Offset-Sparse** (50% density + DC shift) | **+14.07%** | +14.83% | +4.16% | `0 / 12` |
| **Sparse** (10% density) | **+35.40%** | +32.78% | +17.01% | `0 / 12` |
| **Real `Raon.gguf` (`blk.0.ffn_down.weight` + `iMatrix`)** | **+15.96%** | — | — | `0 / 47` |
Reproduce at any time with:
```bash
python3 -m hpc_quantize_v2 --bench --bench-per-kind 12
```
---
## Quickstart
### Requirements
Only **Python 3.10+** and **NumPy** (`tkinter` is included with standard Python installations for the GUI). No C compiler or shared libraries are needed.
```bash
pip install numpy
```
### 1. Desktop GUI with Live Telemetry (`gui.py`)
Launch the graphical interface to browse a source `.gguf`, attach an `.imatrix`/`.gguf` importance file, inspect eligible tensors, tune analog/HPC parameters, set CPU worker count, copy the equivalent CLI command, and monitor live quantization with dual progress bars (**Model Progress** and **Tensor Superblocks**) and dual ETAs (**Tensor ETA** and **Model ETA**):
```bash
python3 gui.py
```
Or pre-populate paths on launch:
```bash
python3 gui.py --gguf Raon.gguf --imatrix imatrix.gguf --output Raon-Q2K.gguf
```
### 2. Command-Line Interface (`hpc_quantize_v2`)
**Quantize a full GGUF model to a `Q2_K` GGUF with an iMatrix (using all CPU cores by default):**
```bash
python3 -m hpc_quantize_v2 \
  --gguf Raon.gguf \
  --all-q2k \
  --imatrix-gguf imatrix.gguf \
  --output Raon-Q2K.gguf \
  --pin-dc \
  --workers 0 \
  --verbose
```
**Inspect tensors in a GGUF file:**
```bash
python3 -m hpc_quantize_v2 --gguf Raon.gguf --list-tensors
```
**Smoke-test a single tensor into a valid GGUF (copies remaining tensors verbatim):**
```bash
python3 -m hpc_quantize_v2 \
  --gguf Raon.gguf \
  --tensor blk.0.ssm_beta.weight \
  --imatrix-gguf imatrix.gguf \
  --output Raon-smoke.gguf
```
**Quantize a NumPy `.npy` weight array to `.npz`:**
```bash
python3 -m hpc_quantize_v2 --input weights.npy --output weights_q2k.npz --verbose
```
**Optional thermal refinery (opt-in, seeded, deterministic per `--seed`):**
```bash
python3 -m hpc_quantize_v2 --gguf model.gguf --all-q2k --thermal 4 --verbose
```
`--thermal N` runs `N` heat-bath annealing passes over each block's code
lattice after Phase 4 (Phase 4.5), trading a tiny amount of weighted SSE for
large reductions in the rolling row-wise DC residual (measured on real GGUF
tensors: extended objective ~1.3% better, DC penalty ~40% lower at ~4% time
cost per 4 passes).  `0` (default) keeps the deterministic pipeline output
bit-for-bit.  The full knob set (`therm_passes`, `therm_sweeps`,
`therm_t0`, `therm_t_decay`, `therm_seed`) lives on `HPCConfig`; the pass is
monotone per block and never regresses a block's extended energy.
The refinery also covers the format quantizers: with `therm_passes > 0` the
finished Q4_0 / Q1_0 / Q3_K blocks are re-swept on their own 16/2/8-level
lattices (signed fp16 scales included) with extended-nonincreasing
acceptance under the `sse_budget` cap — on CPU and CUDA alike, and with the
same bit-exact seeded behavior.

**Zero-residual ("0-error") anneal (`therm_lambda_tilt`, GUI "Zero-Residual
Tilt", CLI `--thermal-tilt`):** a multiplier (≥1, 1=off) applied to
`dc_lambda`/`vw_lambda` while annealing.  Because every Gibbs move is exact
on the quantization lattice, tilting the spectral lambdas makes the chain
spend weighted SSE to drain the *coherent* residual — row DC and vesica
pairs — toward the lattice floor (measured on this model's `ssm_out`:
DC RMS −97% at tilt 1e3 for ~8× SSE).  The endpoint is a weight matrix
whose remaining error is (almost) spectrally white: no residual systematic
error left to accumulate across layers.

**Q2_K Free Lunch — Zero-DC Pin (`pin_dc`, GUI "Pin DC residual to 0 (Q2_K
free lunch)", CLI `--pin-dc`, API `pin_dc=True`):** after Phase 4 (and
thermal, when enabled) the pin re-solves each superblock's `(d, d_min)`
pair in closed form on the exact `Σe = 0` line — codes and 4-bit scales
frozen — so the block-DC residual collapses to the `fp16` storage floor
(measured: **DC RMS −94%, RMSE −0.6%**, block SSE non-increasing,
<10 ms per tensor).  It spends **no** SSE budget (the pin can only lower
unweighted block SSE) and runs identically on the CPU and CUDA paths.

**Zero-DC Pin for the format quantizers (Q4_0 / Q3_K):** those formats
carry a single scale per block and no additive degree of freedom, so the
exact closed-form re-solve is degenerate there.  Instead the pin performs a
deterministic **code-flip exchange**: each round picks, per block, the
flip (direction + position) with the best `ΔSSE / |ΔDC|` ratio and applies
it until the block has spent at most `pin_budget` of its starting SSE or
the residual reaches the floor of half a flip quantum.  Measured on real
tensors (blk.0, CUDA path):

| Format | Config | RMSE | Block-DC RMS |
| :--- | :--- | :---: | :---: |
| `Q3_K` (`blk.0.ffn_down`) | base | $1.554\times10^{-3}$ | $1.96\times10^{-2}$ |
| `Q3_K` + pin 5% | `pin_budget=0.05` | $1.560\times10^{-3}$ (+0.4%) | $3.55\times10^{-3}$ (**−82%**) |
| `Q4_0` (`blk.0.attn_qkv`) | base | $1.223\times10^{-3}$ | $5.82\times10^{-3}$ |
| `Q4_0` + pin 5% | `pin_budget=0.05` | $1.231\times10^{-3}$ (+0.7%) | $3.36\times10^{-3}$ (−42%) |
| `Q4_0` + pin 10% | `pin_budget=0.10` | $1.239\times10^{-3}$ (+1.3%) | $2.18\times10^{-3}$ (−63%) |

`pin_budget` (default 0.05, `0` disables the spend) bounds the per-block
SSE the exchange may spend; the Q2_K closed-form pin never consults it.
The pin is bit-deterministic, applies after thermal refinement, runs
fully on-GPU on the CUDA paths (`pin_dc_codes_uniform_cuda` in
`spectral.py`) and on NumPy on the CPU paths — Q3_K's residual lands near
the DC floor because its signed `sc` unit grid has DC slack on both sides,
while Q4_0's unsigned `[0, 15]` nibble grid is bounded by the code-16
edge and settles higher.
The same single invocation covers Q2_K and the format quantizers —
`--device cuda` (or `--device auto`, which picks CUDA when available)
routes the format pipelines to the CUDA path and the pin runs on-GPU
automatically, no extra flags needed:

```bash
# Pure Q3_K, full CUDA pipeline (thermal + on-GPU zero-DC pin):
python3 -m hpc_quantize_v2 \
  --gguf model.gguf \
  --format q3_k \
  --device cuda \
  --imatrix-gguf imatrix.gguf \
  --thermal 4 \
  --pin-dc \
  --output model-Q3K.gguf \
  --verbose
# Pure Q4_0 (pin included via --pin-dc):
python3 -m hpc_quantize_v2 --gguf model.gguf --format q4_0 --device cuda --pin-dc --output model-Q4_0.gguf
```

In the GUI, tick **Pin DC residual to 0** and pick the CUDA device from the
device selector — the checkbox enables the pin for whichever format each
tensor is quantized to (closed-form re-solve for Q2_K superblocks,
code-flip exchange for Q4_0/Q3_K blocks).  `pin_budget` itself is an
`HPCConfig` field (0.05 default) for API users, GUI/CLI defaults keep the
sensible order of magnitude.

**DC/vesica drain knob map (and where they act):**

| Knob | Config field | Stage | Cost model |
| :--- | :--- | :--- | :--- |
| Spectral shaping | `dc_lambda`, `vw_lambda` (λ_dc=λ_vw=1.0 default) | Phases 1–4 (in-objective) | arises inside the weighted objective itself |
| Thermal λ-tilt drain | `therm_passes`, `therm_lambda_tilt` | Phase 4.5 (opt-in) | spends SSE under `sse_budget`, seeded & deterministic |
| Zero-DC Pin | `pin_dc`, `pin_budget` | Phase 4.7 (opt-in) | Q2_K: **exact & free** (closed-form, SSE non-increasing) · Q4_0/Q3_K: code-flip exchange bounded by `pin_budget` of block SSE |
| Row-aware carry nudge | `apply_dc_carry`, `dc_decay`, `sse_budget`, `carry_cumulative` | Phase 4.8 (on by default) | exactly bounded per-block budget, residual funneled downstream |

For maximal coherent-residual drainage enable thermal (with tilt) + the
pin and keep the carry nudge on (the default); for a bit-exact
"pure-pipeline + free lunch" result, leave `pin_dc=True` with
`therm_passes=0` and disable `apply_dc_carry`.

### 3. Python API

```python
import numpy as np
from hpc_quantize_v2 import (
    HPCConfig,
    Q2KQuantizer,
    read_imatrix,
)
from hpc_quantize_v2.gguf import write_quantized_gguf
# 1. Quantize an in-memory tensor (workers=0 auto-uses all CPU cores)
cfg = HPCConfig(
    n_cand_d=6,
    n_cand_m=6,
    use_weight_analogs=True,
    factor_coupled_q2k=True,
    factor_joint_subscales=True,
    use_spectral=True,
    therm_passes=0,
    pin_dc=True,          # Q2_K free lunch: exact dc=0 re-solve of (d, m)
    apply_dc_carry=True,  # row-aware DC carry drain (Phase 4.8)
    workers=0,
)
quantizer = Q2KQuantizer(cfg)
weights = np.random.default_rng(0).normal(0, 0.02, size=(64, 256)).astype(np.float32)
result = quantizer.quantize(weights, row_width=256)
print("RMSE:", result.stats.rmse)
reconstructed = result.dequantize()
# 2. Convert a source GGUF + iMatrix into a quantized Q2_K GGUF
imat = read_imatrix("imatrix.gguf")
stats = write_quantized_gguf(
    "Raon.gguf",
    "Raon-Q2K.gguf",
    quantizer,
    imat=imat,
    quantize_all=True,
)
print(stats)
```