README.md

metadata

license: mit

GUI screencap

Developer Note

This requires a CUDA GPU, expect a 50GB model to take around 3-4 hours to quantize on a 3060 12GB.

I prefer to llama-quantize to 8_0, then to generate a imatrix from that 8_0, and finally to quantize the 8_0 file to Q2K.

Lattice Repair

This is NOT yet included in the quantizers automated pipeline but DOES function.

It is invoked by doing the following:

python3 lattice_repair.py
--source BF16GGUF.gguf
--quant QUANTGGUF.gguf
--output OUTPUT.gguf
--force
--thresh 1.0
--fix-frames
--dc-lambda 5.0
--ulp-radius 8
--refit-batch 512
--catalog ./lr_moved.csv Or wherever you want to point the weight dumps.

Output looks like this:

blk.11.attn_output.weight [Q2_K] outliers 60829/31457280 moved 26504 (blocks 23737/122880) frame-limited 34410 in 28363 blocks, frames fixed 13822 | rmse 4.2478e-03->4.1842e-03 max|e| 5.834e-01->3.314e-01 mean|DC| 1.380e-03->4.602e-03

Numbers to pay attention to: rmse 4.2478e-03->4.1842e-03 max|e| 5.834e-01->3.314e-01.

RMSE ends up cut a little, Max Error is absolutely shattered and cut in half recovering critical function, at the cost of DC.

DC is my heuristic to assemble the LLM with a initial pass, then Lattice Repair polishes things off.

Q2K and Q3K don't really have a lot of space to move, so the weights are mostly recalculated with fix-frame to bring them closer to the source GGUF.

HPC-Quantize V2

Holographic Phase Contraction, Geometry-Preserving Weight Analogs, and Algebraic Coupled Bilinear Factorization for Ultra-Low-Bit (Q2_K) LLM Quantization

HPC-Quantize V2 is a standalone Python/NumPy ultra-low-bit quantization engine and GGUF re-quantizer (with a real-time telemetry desktop GUI in gui.py) built for 2-bit class (Q2_K, 2.625 bpw) compression of large language models.

Core thesis: At 2 bits per weight (4 levels $q \in {0,1,2,3}$ and 4-bit sub-block scales $Ls, Lm \in {0..15}$), quantization is not an independent rounding problem and should not rely on brute-force grid scans. It is a coupled bilinear factorization and global sequence-optimization problem best solved by extracting discrete lattice coordinates from geometry-preserving weight analogs, projecting them onto the true weights via closed-form orthogonalized sufficient statistics, and optimizing cross-block state trajectories with a 6-state sequential Sieve and 36-state Viterbi dynamic program.

HPC-Quantize V2 (hpc_quantize_v2/ + gui.py) has zero C or ctypes dependencies, reads and writes stock llama.cpp-compatible .gguf models and .gguf/.dat importance matrices (iMatrix), scales across all CPU cores with bit-for-bit determinism, and requires no custom inference kernels.

---

Key Highlights in V2

1. Geometry-Preserving Weight Analogs (hpc_quantize_v2/analog.py): Constructs a portfolio of surrogate weight blocks $\tilde{x}$ that preserve exact sub-block first and second importance-weighted moments ($\mu_j, \sigma_j^2$), rank order, unit sub-block direction ($\cos\theta = 1$), total block energy, and the $2\times 2$ D6/Vesica fold Gram matrix while regularizing heavy-tailed outliers toward the 4-level Q2_K lattice.
2. Coupled Orthogonal-Basis Bilinear Q2_K Factorization (hpc_quantize_v2/factored.py): Replaces independent 1-D scale/offset heuristics with an orthogonalized sub-block decomposition $\hat{x}{j,k} = s_j(q{j,k} - \bar{q}j) + (s_j\bar{q}j - t_j)$, coupled offset compensation $t_j^{\text{coupled}} = \max(0, (d \cdot Ls_j \cdot S{wq,j} - S{x,j})/S_{w,j})$, rank-1 algebraic 1-D code refinement ($S_{vv} - S_{lv}^2/S_{l2}$), and a single-pass vectorized 36-seed candidate lattice evaluator (81x faster than per-seed loops).
3. Algebraic Coupled Phase-4 Block Assembly (hpc_quantize_v2/pipeline.py): Replaces legacy brute-force $16\times 16$ sub-block scans, $9\times 9$ weight-level ULP loops, and 256-pass greedy code searches with closed-form sufficient-statistic projections (1,200–1,475 blocks/sec in Phase 4, achieving +2.64% lower objective error than brute-force Phase 4 in 1/9th the time).
4. Bit-Identical Multi-Core Execution (--workers 0): Distributes block-independent candidate and analog solves across all CPU cores via zero-copy fork shared memory while keeping cross-block Sieve, 36-state Viterbi, and row-aware DC carry strictly sequential — producing 100% bit-for-bit identical GGUF output regardless of worker count.
5. Real-Time Telemetry GUI (gui.py) & Direct GGUF-to-GGUF Pipeline (hpc_quantize_v2/gguf.py): Ingests F32, F16, BF16, Q8_0, Q4_0, and Q2_K source GGUFs plus llama.cpp iMatrix files, displays live intra-tensor stage/block progress, dual Tensor ETA and block-weighted Whole-Model ETA, live RMSE/RMSE/σ, Sieve temperature, and floor hits, and emits a stock llama.cpp Q2_K GGUF.
6. The Coherent-Residual Drain & the Q2_K Free Lunch (spectral.py, anneal.py, q2k.py): A four-stage drain of the coherent error channel — in-objective DC/vesica shaping, an opt-in thermal λ-tilt drain, a row-aware sequential DC carry nudge, and the closed-form Zero-DC Pin (Phase 4.7): a pure $O(n)$ re-solve of the superblock scale pair on the exact Σe = 0 line that drops the block-DC residual by ~94% while slightly lowering RMSE — for free, with zero SSE budget.

---

Why Standard Q2_K Quantization Breaks Down

A Q2_K superblock encodes 256 weights in 84 bytes (16 sub-blocks of 16 weights each):

$${\widehat{x}}_{16j+k}=d\cdot L{s}_j\cdot q_{16j+k}-d_{\min }\cdot L{m}_j,q\in \{0,1,2,3\},L{s}_j,L{m}_j\in \{0,\dots ,15\},d,d_{\min }\in {\mathbb{F}}_{16}\mathrm{.}\backslash hat\{x\}\_\{16j\; +\; k\}\; =\; d\; \backslash cdot\; Ls\_j\; \backslash cdot\; q\_\{16j\; +\; k\}\; -\; d\_\{\backslash min\}\; \backslash cdot\; Lm\_j,\; \backslash qquad\; q\; \backslash in\; \backslash \{0,1,2,3\backslash \},\; \backslash quad\; Ls\_j,\; Lm\_j\; \backslash in\; \backslash \{0,\backslash ldots,15\backslash \},\; \backslash quad\; d,\; d\_\{\backslash min\}\; \backslash in\; \backslash mathbb\{F\}\_\{16\}.$$

When factoring raw LLM weights directly into this bilinear form, three structural failure modes occur:

1. Intra-sub-block outlier collapse: A single outlier weight in a 16-weight sub-block stretches the span $x_{\max} - x_{\min}$, forcing 13–15 of the 16 weights into a single bin ($q=1$ or $q=2$) and reducing effective precision below 1 bit.
2. Inter-sub-block scale crushing: A single high-dispersion sub-block inflates the shared superblock scale $d$, forcing $Ls_j \in {1, 2}$ across the remaining 15 sub-blocks.
3. Scale-offset cross-term neglect: Because $q_{j,k} \in {0,1,2,3}$ is non-negative, its importance-weighted mean $\bar{q}j = \frac{\sum_k w{j,k} q_{j,k}}{\sum_k w_{j,k}}$ is strictly positive. Rounding the scale $s_j \to d \cdot Ls_j$ shifts the sub-block mean by $(d \cdot Ls_j - s_j)\bar{q}j$. Factoring $s_j \approx d \cdot Ls_j$ and $t_j \approx d{\min} \cdot Lm_j$ independently leaves this mean shift uncompensated.

---

Mathematical Architecture of HPC-Quantize V2

1. Geometry-Preserving Weight Analogs (analog.py)

To prevent outliers from collapsing the discrete code assignment $q \in {0,1,2,3}$ while respecting the true linear-layer geometry, V2 constructs a portfolio of geometry-preserving analogs $\tilde{x}$ of each eligible weight block before extracting candidate lattice coordinates:

A. Sub-block Iso-Moment Quantile-Companded Analog (subblock_isomoment_analog)

For each 16-weight sub-block $j$ with importance weights $w_{j,k}$:

$${\mu }_j=\frac{\sum _k{w}_{j,k}{x}_{j,k}}{\sum _k{w}_{j,k}},{\sigma }_j^2=\frac{\sum _k{w}_{j,k}(x_{j,k}-{\mu }_j{)}^2}{\sum _k{w}_{j,k}},z_{j,k}=\frac{x_{j,k}-{\mu }_j}{{\sigma }_j}\mathrm{.}\backslash mu\_j\; =\; \backslash frac\{\backslash sum\_k\; w\_\{j,k\}\; x\_\{j,k\}\}\{\backslash sum\_k\; w\_\{j,k\}\},\; \backslash qquad\; \backslash sigma\_j^2\; =\; \backslash frac\{\backslash sum\_k\; w\_\{j,k\}(x\_\{j,k\}\; -\; \backslash mu\_j)^2\}\{\backslash sum\_k\; w\_\{j,k\}\},\; \backslash qquad\; z\_\{j,k\}\; =\; \backslash frac\{x\_\{j,k\}\; -\; \backslash mu\_j\}\{\backslash sigma\_j\}.$$

Standardized coordinates are mapped through an importance-weighted hyperbolic transport blended with ideal 4-level quantiles $q^{\text{ideal}}_r \in [-1.35, +1.35]$:

$${\tilde{z}}_{j,k}=(1-\alpha )\tau \tanh \text{\hspace{0.17em}⁣}\left(\frac{(w_{j,k}\mathrm{/}{\bar{w}}_j{)}^{\beta }{z}_{j,k}}{\tau }\right)+\alpha q_{\text{rank}(z_{j,k})}^{\text{ideal}},\backslash tilde\{z\}\_\{j,k\}\; =\; (1\; -\; \backslash alpha)\backslash ,\backslash tau\; \backslash tanh\backslash !\backslash left(\backslash frac\{(w\_\{j,k\}/\backslash bar\{w\}\_j)^\backslash beta\; z\_\{j,k\}\}\{\backslash tau\}\backslash right)\; +\; \backslash alpha\backslash ,\; q^\{\backslash text\{ideal\}\}\_\{\backslash text\{rank\}(z\_\{j,k\})\},$$

and then orthogonally projected back onto the exact sub-block sphere:

$${\tilde{x}}_{j,k}={\mu }_j+{\sigma }_j\frac{{\tilde{z}}_{j,k}-{\bar{z}}_j}{\sqrt{\frac{1}{W_j}\sum _k{w}_{j,k}({\tilde{z}}_{j,k}-{\bar{z}}_j{)}^2}}\mathrm{.}\backslash tilde\{x\}\_\{j,k\}\; =\; \backslash mu\_j\; +\; \backslash sigma\_j\; \backslash frac\{\backslash tilde\{z\}\_\{j,k\}\; -\; \backslash bar\{z\}\_j\}\{\backslash sqrt\{\backslash frac\{1\}\{W\_j\}\backslash sum\_k\; w\_\{j,k\}(\backslash tilde\{z\}\_\{j,k\}\; -\; \backslash bar\{z\}\_j)^2\}\}.$$

This preserves the exact sub-block weighted mean $\mu_j$, weighted variance $\sigma_j^2$, and rank ordering while equalizing spacing across the 4 quantization levels.

B. D6 / Vesica Gram-Preserving Fold Analog (vesica_gram_analog)

Decomposes the 256-weight superblock into symmetric (vesica) and antisymmetric (wave) fold modes:

$$v_p=\frac{1}{2}(x_p+x_{p+128}),u_p=\frac{1}{2}(x_p-x_{p+128}),p\in \{0,\dots ,127\}\mathrm{.}v\_p\; =\; \backslash frac\{1\}\{2\}(x\_p\; +\; x\_\{p+128\}),\; \backslash qquad\; u\_p\; =\; \backslash frac\{1\}\{2\}(x\_p\; -\; x\_\{p+128\}),\; \backslash qquad\; p\; \backslash in\; \backslash \{0,\backslash ldots,127\backslash \}.$$

Let $G \in \mathbb{R}^{2\times 2}$ be the importance-weighted covariance (Gram) matrix of $(v, u)$. V2 whitens $(v, u)$ via $G^{-1/2}$, applies smooth hyperbolic saturation along the principal axes, re-orthogonalizes, and colors back via $G^{1/2}$ so that:

$${\mathrm{Gram}}_w(\tilde{v},\tilde{u})={\mathrm{Gram}}_w(v,u),{\mathbb{E}}_w[\tilde{v}]={\mathbb{E}}_w[v],{\mathbb{E}}_w[\tilde{u}]={\mathbb{E}}_w[u]\mathrm{.}\backslash operatorname\{Gram\}\_w(\backslash tilde\{v\},\; \backslash tilde\{u\})\; =\; \backslash operatorname\{Gram\}\_w(v,\; u),\; \backslash qquad\; \backslash mathbb\{E\}\_w[\backslash tilde\{v\}]\; =\; \backslash mathbb\{E\}\_w[v],\; \backslash qquad\; \backslash mathbb\{E\}\_w[\backslash tilde\{u\}]\; =\; \backslash mathbb\{E\}\_w[u].$$

C. Cross-Sub-block Scale-Lattice Analog (subscale_lattice_analog)

Tempers extreme sub-block standard deviations $\sigma_j \mapsto \sigma_j^\gamma \sigma_{\text{block}}^{1-\gamma}$ so all 16 sub-blocks fit cleanly into the shared 4-bit ${0..15}$ scale lattice while preserving every sub-block's unit direction $\hat{u}_j$ ($\cos\theta = 1$), mean $\mu_j$, and total superblock weighted energy.

---

2. Coupled Orthogonal-Basis Bilinear Q2_K Factorization (factored.py)

Given candidate discrete codes $q_{j,k} \in {0,1,2,3}$ (from raw starts and the analog portfolio), V2 projects the continuous and 4-bit scale parameters directly onto the true weights $x$:

1. Orthogonalized Sub-block Basis: Rewriting $\hat{x}{j,k} = s_j(q{j,k} - \bar{q}j) + (s_j\bar{q}j - t_j)$ diagonalizes the $2\times 2$ sub-block normal equations. The true sensitivity of sub-block $j$ to scale error $(s_j - d \cdot Ls_j)^2$ is the centered variance weight: $$ V{qq,j} = \sum{k=0}^{15} w_{j,k}(q_{j,k} - \bar{q}j)^2 = S{qq,j} - S_{wq,j}\bar{q}_j. $$
2. Monotone Rank-1 1-D Factorization (factor_1d): Factors $s_j \approx d \cdot Ls_j$ under weights $V_{qq,j}$ using multi-start Lloyd iterations followed by an exact vectorized rank-1 coordinate sweep maximizing $S_{lv}^2 / S_{l2}$ (since $\sum_j w_j(v_j - d^* L_j)^2 = S_{vv} - S_{lv}^2/S_{l2}$).
3. Coupled Offset Target: Once $(d, Ls_j)$ are chosen, the optimal offset for sub-block $j$ is shifted in closed form to absorb the scale rounding residual: $$ t_j^{\text{coupled}} = \max!\left(0,; \frac{d \cdot Ls_j \cdot S_{wq,j} - S_{x,j}}{S_{w,j}}\right) = \max!\left(0,; t_j + \bar{q}j(d \cdot Ls_j - s_j)\right), $$ which is then factored into $d{\min} \cdot Lm_j$ under weights $S_{w,j}$.

---

3. Batched Algebraic Phase-4 Assembly (pipeline.py)

Instead of brute-force $16\times 16$ sub-block scans and $9\times 9$ weight-level ULP searches, Phase 4 refines both the Viterbi-selected state and the analog-factored state across all blocks in vectorized batches using three exact algebraic identities:

1. Closed-Form Coupled Sub-scale Projection: From sufficient statistics $(V_{qq,j}, C_{qx,j}, S_{wq,j}, S_{x,j}, S_{w,j})$, the unconstrained scale minimizer is $s_j^* = \max(0, C_{qx,j}/V_{qq,j})$. Evaluating the local integer neighborhood $Ls_j \in \operatorname{round}(s_j^*/d) + {-3,\ldots,+3}$ paired with its closed-form coupled offset $Lm_j \in \operatorname{round}(t_j^*(Ls_j)/m) + {-1,0,+1}$ replaces $16\times 16 = 256$ brute-force pairs with $21$ algebraic states.
2. Vesica + DC Augmented $2\times 2$ Cramér Solve & $O(1)$ Scalar Quadratic ULP Polish: With $A_i = Ls_{i\gg 4} q_i$ and $B_i = Lm_{i\gg 4}$, the full extended objective (weighted SSE + Vesica fold pairs + DC carry) is an exact quadratic form in $(d, m)$: $$ J(d, m) = \text{const} + \tilde{S}{aa} d^2 - 2\tilde{S}{ab} d m + \tilde{S}{bb} m^2 - 2\tilde{S}{xa} d + 2\tilde{S}{xb} m, $$ where $(\tilde{S}{aa}, \tilde{S}{ab}, \tilde{S}{bb}, \tilde{S}{xa}, \tilde{S}{xb})$ include the Vesica pair sums $(A_p + A_{p+128}, B_p + B_{p+128}, x_p + x_{p+128})$ and the row DC sum $(\sum_i A_i, \sum_i B_i, \sum_i x_i + \text{carry})$. The continuous optimum $(d^*, m^*)$ is solved by $2\times 2$ Cramér's rule and the $\mathbb{F}_{16}$ ULP neighborhood is polished in $O(1)$ scalar arithmetic without touching the 256 weights.
3. Closed-Form 1-D Quadratic Vesica/DC Code Projection: Each weight's optimal discrete step $\Delta q_k^*$ under the combined weighted SSE + Vesica + DC objective is computed directly from the 1-D quadratic minimizer: $$ \Delta q_k^* = \operatorname{round}!\left(\frac{d_{\text{sub},k}\left(w_k e_k + \frac{\lambda_{\text{vw}}}{N}(e_k + e_{k\oplus 128}) + \frac{\lambda_{\text{dc}}}{N} D\right)}{\left(w_k + \frac{\lambda_{\text{vw}} + \lambda_{\text{dc}}}{N}\right) d_{\text{sub},k}^2}\right). $$

---

4. The Coherent-Residual Drain: DC & Vesica Error Handling (spectral.py, anneal.py, q2k.py)

A. Residual anatomy: why "SSE-optimal" is not enough

For a finished Q2_K superblock, the per-element error is

$$e_i=x_i-(d\cdot L{s}_{\lfloor i\mathrm{/}16\rfloor }{q}_i-d_{\min }\cdot L{m}_{\lfloor i\mathrm{/}16\rfloor }),i\in \{0,\dots ,255\}\mathrm{.}e\_i\; =\; x\_i\; -\; \backslash big(d\; \backslash cdot\; Ls\_\{\backslash lfloor\; i/16\; \backslash rfloor\}\backslash ,\; q\_i\; -\; d\_\{\backslash min\}\; \backslash cdot\; Lm\_\{\backslash lfloor\; i/16\; \backslash rfloor\}\backslash big),\; \backslash qquad\; i\; \backslash in\; \backslash \{0,\backslash ldots,255\backslash \}.$$

SSE-optimal per-element quantizers are unbiased only on average over huge ensembles. At 2 bits per weight, with the (d, d_min) pair stored in fp16, the residual of a single superblock has two coherent components with scale far above the white-noise floor:

1. Block DC $D = \sum_{i=0}^{255} e_i$ — the systematic per-superblock offset error.
2. Vesica pairs $v_p = e_p + e_{p+128}$, $p \in {0,\ldots,127}$ — the first fold level of the dyadic/Hadamard error pyramid, i.e. the dominant low-frequency coherent residual shared by the symmetric halves of a superblock.

Measured on this stack: the unpinned pipeline leaves block-DC RMS $\approx 2.7\times 10^{-2}$ against a total RMSE of $\approx 4.1\times 10^{-3}$ — the coherent channel is ~7× louder than the entire random error, and because it is coherent it accumulates across layers in inference instead of averaging out. V2 therefore treats draining $D$ and $v_p$ as a first-class problem, with four independent, composable mechanisms:

B. Mechanism 1 — In-objective spectral shaping (Phases 1–4, always on)

The block objective augments importance-weighted SSE with the DC/vesica fold penalty

$$E=\sum _i{w}_i{e}_i^2+\frac{{\lambda }_{\text{vw}}}{n}\sum _{p=0}^{127}{v}_p^2+\frac{{\lambda }_{\text{dc}}}{n}(D+\text{carry}{)}^2,E\; =\; \backslash sum\_i\; w\_i\; e\_i^2\; \backslash ;+\backslash ;\; \backslash frac\{\backslash lambda\_\{\backslash text\{vw\}\}\}\{n\}\; \backslash sum\_\{p=0\}^\{127\}\; v\_p^2\; \backslash ;+\backslash ;\; \backslash frac\{\backslash lambda\_\{\backslash text\{dc\}\}\}\{n\}\; \backslash ,(D\; +\; \backslash text\{carry\})^2\; ,$$

(dc_lambda, vw_lambda; the full vectorized fold pyramid generalizes this to every dyadic level). All three algebraic Phase-4 solvers — the coupled sub-scale projection, the $2\times 2$ Cramér scale/offset solve, and the 1-D quadratic code descent — minimize this extended form exactly, so the lattice solution space itself is biased toward low coherent residual before any separate drain pass runs.

C. Mechanism 2 — Thermal λ-tilt drain (Phase 4.5, opt-in, --thermal + --thermal-tilt)

With therm_passes > 0, the exact heat-bath Gibbs chain sweeps the code lattice with the extended energy. Every accepted move is exact on the quantization lattice, so multiplying the spectral lambdas by the zero-residual tilt (therm_lambda_tilt ≥ 1) makes the chain deliberately spend weighted SSE to buy coherent-residual drainage — it walks $(\text{SSE}, D, v_p)$ along the exact Pareto frontier of the block. Endpoint behavior at large tilts: the coherent residual is driven to the lattice floor and the remaining error is (almost) spectrally white — measured on this model's ssm_out: DC RMS −97% at tilt $10^3$ for ~8× SSE. The pass is monotone per block (extended energy never regresses) and seeded/deterministic.

D. Mechanism 3 — Row-aware sequential DC carry nudge (Phase 4.8, on by default)

_apply_dc_nudge sweeps superblocks strictly left-to-right (row topology of the weight matrix, row_width blocks per row). Each block's codes are re-projected by dc_nudge_codes — a bounded-SSE-budget greedy descent that cancels the incoming DC carry dc_decay · rolling in exchange for as little objective cost as the sse_budget cap allows; whatever DC the block cannot cheaply cancel becomes part of the rolling carry handed to the next superblock (reset at row boundaries; carry_cumulative selects reset-to-current vs accumulate). Net effect: the residual DC of the whole tensor gets funneled into whichever blocks have the cheapest 2-bit codes to absorb it, instead of staying distributed uniformly. Cost is exactly bounded by construction; the sweep is deterministic and single-pass.

E. Mechanism 4 — The Zero-DC Pin, a.k.a. the "Q2_K Free Lunch" (Phase 4.7, opt-in, pin_dc)

This is a pure closed-form bonus: with codes, $Ls$, $Lm$ frozen at their final refined state, the remaining freedom $(d, m)$ (with $m \equiv d_{\min}$, the fp16 minimum field) is re-solved subject to exact zero DC. Two degrees of freedom, two constraints ($\min \sum e_i^2$, $\sum e_i = 0$) → an exact solution, no lattice search, no budget.

Write $a_i = Ls_{\lfloor i/16\rfloor} c_i$ and $b_i = Lm_{\lfloor i/16\rfloor}$ so $e_i = x_i - d,a_i + m,b_i$, and let $S_x = \sum_i x_i$, $S_a = \sum_i a_i$, $S_b = \sum_i b_i$. The constraint $\sum_i e_i = 0$ pins $m$ onto a line parameterized by $d$:

$$m(d)=\frac{d{S}_a-S_x}{S_b}\Rightarrow e_i(d)=u_i-d{v}_i,u_i=x_i-b_i\frac{S_x}{S_b},v_i=a_i-b_i\frac{S_a}{S_b}\mathrm{.}m(d)\; =\; \backslash frac\{d\backslash ,S\_a\; -\; S\_x\}\{S\_b\}\; \backslash quad\backslash Rightarrow\backslash quad\; e\_i(d)\; =\; u\_i\; -\; d\backslash ,v\_i,\; \backslash quad\; u\_i\; =\; x\_i\; -\; b\_i\; \backslash tfrac\{S\_x\}\{S\_b\},\backslash ;\; v\_i\; =\; a\_i\; -\; b\_i\; \backslash tfrac\{S\_a\}\{S\_b\}.$$

Along that line the block energy

$$\sum _i{e}_i^2(d)=\mathrm{\parallel }u{\mathrm{\parallel }}^2-2d⟨u,v⟩+d^2\mathrm{\parallel }v{\mathrm{\parallel }}^2\backslash sum\_i\; e\_i^2(d)\; =\; \backslash |u\backslash |^2\; \backslash ;-\backslash ;\; 2d\backslash ,\backslash langle\; u,\; v\backslash rangle\; \backslash ;+\backslash ;\; d^2\; \backslash |v\backslash |^2$$

is a convex 1-D quadratic with exact minimizer

$$d^{\ast }=\frac{⟨u,v⟩}{⟨v,v⟩},m^{\ast }=\frac{d^{\ast }{S}_a-S_x}{S_b}\mathrm{.}d^*\; =\; \backslash frac\{\backslash langle\; u,\; v\backslash rangle\}\{\backslash langle\; v,\; v\backslash rangle\},\; \backslash qquad\; m^*\; =\; \backslash frac\{d^*\; S\_a\; -\; S\_x\}\{S\_b\}.$$

Geometric meaning: $v$ is the component of the code vector $a$ orthogonal to the all-ones direction (the DC direction), and $u$ is the analogous component of the data — so $d^*$ is the exact OLS scale of orthogonally-projected data against orthogonally-projected codes. The pin is self-dominating: moving from the input $(d, m)$ to $(d^*, m^*)$ travels along the zero-DC line, so it is at worst the zero move — SSE is non-increasing, $|D|$ is non-increasing, only the fp16 storage rounding of the (d, m) fields limits the final DC. Blocks where the translation degree of freedom degenerates ($S_b = 0$ or $\langle v, v\rangle = 0$) keep their input parameters; the map is fp16-idempotent (re-pinning an already-pinned state is a no-op).

Its practical effect (live blk.0.ssm_out.weight, 8,192 superblocks, RTX 3060 path):

Configuration	RMSE	Block-DC RMS	mean $\lvert D \rvert$
Base pipeline (Phases 1–4 + carry nudge)	$4.068\times 10^{-3}$	$2.72\times 10^{-2}$	$1.96\times 10^{-2}$
Base + Zero-DC Pin (Phase 4.7)	$4.045\times 10^{-3}$	$1.70\times 10^{-3}$	$1.36\times 10^{-3}$

i.e. DC RMS −93.8%, RMSE −0.6%, at <10 ms for the whole tensor (a handful of $O(n_B \cdot 256)$ reductions). The weighted iMatrix-objective does rise slightly (~5% on wSSE for the pinned sample) — that is the point: it is an exchange, trading part of the importance-weighted objective for a near-zero coherent channel, and it composes safely with the drain stages of Mechanisms 1–3 (recommended order: thermal λ-tilt drain → pin → row-aware carry nudge, which is exactly the pipeline's Stage order 4.5 → 4.7 → 4.8).

---

5. End-to-End HPC Pipeline

Source Weights + Optional iMatrix
        │
        ▼
Kurtosis-Gated Clamp & Batched GGML Seeds (make_qkx2_batch / make_qp_batch)
        │
        ▼
Vectorized Superblock WLS + Multi-Core Analog-Guided Factored Candidate Field
        │
        ▼
6-State Sequential Sieve (Bounded Neighbor Parity Back-Action + Log-Slack)
        │
        ▼
36-State Viterbi Global Sequence Optimization (Free-Energy Bins + Manhattan Prior)
        │
        ▼
Batched Algebraic Phase-4 Coupled Assembly + Quadratic ULP Polish + Closed-Form Code Projection
        │
        ▼
Opt-In Thermal Drain (Exact Heat-Bath Sweep, Zero-Residual λ-Tilt — Phase 4.5)
        │
        ▼
Zero-DC Pin / "Q2_K Free Lunch" (Closed-Form dc=0 Re-Solve of (d, m) — Phase 4.7)
        │
        ▼
Row-Aware Sequential DC Residual Nudge & Monotone Local-Safety Floors (Phase 4.8)
        │
        ▼
Standard Q2_K GGUF Output

---

Measured Benchmarks

Evaluated across 48 synthetic blocks (4 distributions) and a full row (47 superblocks = 12,032 weights) of blk.0.ffn_down.weight from Raon.gguf weighted by imatrix.gguf:

Benchmark Suite	Mean Objective Reduction	Median Reduction	Min Reduction	Worse Cases
Gaussian ($\mathcal{N}(0, 0.02^2)$)	+8.85%	+8.45%	+3.17%	0 / 12
Heavy-Tailed ($\text{Student-}t_3$)	+10.25%	+8.30%	+5.04%	0 / 12
Offset-Sparse (50% density + DC shift)	+14.07%	+14.83%	+4.16%	0 / 12
Sparse (10% density)	+35.40%	+32.78%	+17.01%	0 / 12
Real Raon.gguf (blk.0.ffn_down.weight + iMatrix)	+15.96%	—	—	0 / 47

Reproduce at any time with:

python3 -m hpc_quantize_v2 --bench --bench-per-kind 12

---

Quickstart

Requirements

Only Python 3.10+ and NumPy (tkinter is included with standard Python installations for the GUI). No C compiler or shared libraries are needed.

pip install numpy

1. Desktop GUI with Live Telemetry (gui.py)

Launch the graphical interface to browse a source .gguf, attach an .imatrix/.gguf importance file, inspect eligible tensors, tune analog/HPC parameters, set CPU worker count, copy the equivalent CLI command, and monitor live quantization with dual progress bars (Model Progress and Tensor Superblocks) and dual ETAs (Tensor ETA and Model ETA):

python3 gui.py

Or pre-populate paths on launch:

python3 gui.py --gguf Raon.gguf --imatrix imatrix.gguf --output Raon-Q2K.gguf

2. Command-Line Interface (hpc_quantize_v2)

Quantize a full GGUF model to a Q2_K GGUF with an iMatrix (using all CPU cores by default):

python3 -m hpc_quantize_v2 \
  --gguf Raon.gguf \
  --all-q2k \
  --imatrix-gguf imatrix.gguf \
  --output Raon-Q2K.gguf \
  --pin-dc \
  --workers 0 \
  --verbose

Inspect tensors in a GGUF file:

python3 -m hpc_quantize_v2 --gguf Raon.gguf --list-tensors

Smoke-test a single tensor into a valid GGUF (copies remaining tensors verbatim):

python3 -m hpc_quantize_v2 \
  --gguf Raon.gguf \
  --tensor blk.0.ssm_beta.weight \
  --imatrix-gguf imatrix.gguf \
  --output Raon-smoke.gguf

Quantize a NumPy .npy weight array to .npz:

python3 -m hpc_quantize_v2 --input weights.npy --output weights_q2k.npz --verbose

Optional thermal refinery (opt-in, seeded, deterministic per --seed):

python3 -m hpc_quantize_v2 --gguf model.gguf --all-q2k --thermal 4 --verbose

--thermal N runs N heat-bath annealing passes over each block's code lattice after Phase 4 (Phase 4.5), trading a tiny amount of weighted SSE for large reductions in the rolling row-wise DC residual (measured on real GGUF tensors: extended objective ~1.3% better, DC penalty ~40% lower at ~4% time cost per 4 passes). 0 (default) keeps the deterministic pipeline output bit-for-bit. The full knob set (therm_passes, therm_sweeps, therm_t0, therm_t_decay, therm_seed) lives on HPCConfig; the pass is monotone per block and never regresses a block's extended energy.

The refinery also covers the format quantizers: with therm_passes > 0 the finished Q4_0 / Q1_0 / Q3_K blocks are re-swept on their own 16/2/8-level lattices (signed fp16 scales included) with extended-nonincreasing acceptance under the sse_budget cap — on CPU and CUDA alike, and with the same bit-exact seeded behavior.

Zero-residual ("0-error") anneal (therm_lambda_tilt, GUI "Zero-Residual Tilt", CLI --thermal-tilt): a multiplier (≥1, 1=off) applied to dc_lambda/vw_lambda while annealing. Because every Gibbs move is exact on the quantization lattice, tilting the spectral lambdas makes the chain spend weighted SSE to drain the coherent residual — row DC and vesica pairs — toward the lattice floor (measured on this model's ssm_out: DC RMS −97% at tilt 1e3 for ~8× SSE). The endpoint is a weight matrix whose remaining error is (almost) spectrally white: no residual systematic error left to accumulate across layers.

Q2_K Free Lunch — Zero-DC Pin (pin_dc, GUI "Pin DC residual to 0 (Q2_K free lunch)", CLI --pin-dc, API pin_dc=True): after Phase 4 (and thermal, when enabled) the pin re-solves each superblock's (d, d_min) pair in closed form on the exact Σe = 0 line — codes and 4-bit scales frozen — so the block-DC residual collapses to the fp16 storage floor (measured: DC RMS −94%, RMSE −0.6%, block SSE non-increasing, <10 ms per tensor). It spends no SSE budget (the pin can only lower unweighted block SSE) and runs identically on the CPU and CUDA paths.

Zero-DC Pin for the format quantizers (Q4_0 / Q3_K): those formats carry a single scale per block and no additive degree of freedom, so the exact closed-form re-solve is degenerate there. Instead the pin performs a deterministic code-flip exchange: each round picks, per block, the flip (direction + position) with the best ΔSSE / |ΔDC| ratio and applies it until the block has spent at most pin_budget of its starting SSE or the residual reaches the floor of half a flip quantum. Measured on real tensors (blk.0, CUDA path):

Format	Config	RMSE	Block-DC RMS
Q3_K (blk.0.ffn_down)	base	$1.554\times10^{-3}$	$1.96\times10^{-2}$
Q3_K + pin 5%	pin_budget=0.05	$1.560\times10^{-3}$ (+0.4%)	$3.55\times10^{-3}$ (−82%)
Q4_0 (blk.0.attn_qkv)	base	$1.223\times10^{-3}$	$5.82\times10^{-3}$
Q4_0 + pin 5%	pin_budget=0.05	$1.231\times10^{-3}$ (+0.7%)	$3.36\times10^{-3}$ (−42%)
Q4_0 + pin 10%	pin_budget=0.10	$1.239\times10^{-3}$ (+1.3%)	$2.18\times10^{-3}$ (−63%)

pin_budget (default 0.05, 0 disables the spend) bounds the per-block SSE the exchange may spend; the Q2_K closed-form pin never consults it. The pin is bit-deterministic, applies after thermal refinement, runs fully on-GPU on the CUDA paths (pin_dc_codes_uniform_cuda in spectral.py) and on NumPy on the CPU paths — Q3_K's residual lands near the DC floor because its signed sc unit grid has DC slack on both sides, while Q4_0's unsigned [0, 15] nibble grid is bounded by the code-16 edge and settles higher.

The same single invocation covers Q2_K and the format quantizers — --device cuda (or --device auto, which picks CUDA when available) routes the format pipelines to the CUDA path and the pin runs on-GPU automatically, no extra flags needed:

# Pure Q3_K, full CUDA pipeline (thermal + on-GPU zero-DC pin):
python3 -m hpc_quantize_v2 \
  --gguf model.gguf \
  --format q3_k \
  --device cuda \
  --imatrix-gguf imatrix.gguf \
  --thermal 4 \
  --pin-dc \
  --output model-Q3K.gguf \
  --verbose

# Pure Q4_0 (pin included via --pin-dc):
python3 -m hpc_quantize_v2 --gguf model.gguf --format q4_0 --device cuda --pin-dc --output model-Q4_0.gguf

In the GUI, tick Pin DC residual to 0 and pick the CUDA device from the device selector — the checkbox enables the pin for whichever format each tensor is quantized to (closed-form re-solve for Q2_K superblocks, code-flip exchange for Q4_0/Q3_K blocks). pin_budget itself is an HPCConfig field (0.05 default) for API users, GUI/CLI defaults keep the sensible order of magnitude.

DC/vesica drain knob map (and where they act):

Knob	Config field	Stage	Cost model
Spectral shaping	dc_lambda, vw_lambda (λ_dc=λ_vw=1.0 default)	Phases 1–4 (in-objective)	arises inside the weighted objective itself
Thermal λ-tilt drain	therm_passes, therm_lambda_tilt	Phase 4.5 (opt-in)	spends SSE under sse_budget, seeded & deterministic
Zero-DC Pin	pin_dc, pin_budget	Phase 4.7 (opt-in)	Q2_K: exact & free (closed-form, SSE non-increasing) · Q4_0/Q3_K: code-flip exchange bounded by pin_budget of block SSE
Row-aware carry nudge	apply_dc_carry, dc_decay, sse_budget, carry_cumulative	Phase 4.8 (on by default)	exactly bounded per-block budget, residual funneled downstream

For maximal coherent-residual drainage enable thermal (with tilt) + the pin and keep the carry nudge on (the default); for a bit-exact "pure-pipeline + free lunch" result, leave pin_dc=True with therm_passes=0 and disable apply_dc_carry.

3. Python API

import numpy as np
from hpc_quantize_v2 import (
    HPCConfig,
    Q2KQuantizer,
    read_imatrix,
)
from hpc_quantize_v2.gguf import write_quantized_gguf

# 1. Quantize an in-memory tensor (workers=0 auto-uses all CPU cores)
cfg = HPCConfig(
    n_cand_d=6,
    n_cand_m=6,
    use_weight_analogs=True,
    factor_coupled_q2k=True,
    factor_joint_subscales=True,
    use_spectral=True,
    therm_passes=0,
    pin_dc=True,          # Q2_K free lunch: exact dc=0 re-solve of (d, m)
    apply_dc_carry=True,  # row-aware DC carry drain (Phase 4.8)
    workers=0,
)
quantizer = Q2KQuantizer(cfg)

weights = np.random.default_rng(0).normal(0, 0.02, size=(64, 256)).astype(np.float32)
result = quantizer.quantize(weights, row_width=256)
print("RMSE:", result.stats.rmse)
reconstructed = result.dequantize()

# 2. Convert a source GGUF + iMatrix into a quantized Q2_K GGUF
imat = read_imatrix("imatrix.gguf")
stats = write_quantized_gguf(
    "Raon.gguf",
    "Raon-Q2K.gguf",
    quantizer,
    imat=imat,
    quantize_all=True,
)
print(stats)