File size: 2,130 Bytes
9118991 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | reference:
project: "FlatQuant: Flatness Matters for LLM Quantization"
role: "Primary implementation cited as reference [35] by LoopQ"
repository: "https://github.com/ruikangliu/FlatQuant"
commit: "9d88ffcb7d2c6bda59fb5c44dad36adc101aadb1"
commit_date: "2025-11-25T20:18:56+08:00"
license: MIT
inspected_on: "2026-09-06"
files:
- path: flatquant/trans_utils.py
sha256: bbd12d6effa9b46d21875abd85ec8cfb91075bb0ed6151fa663c60411715bb93
relevance: "SVD/direct single and Kronecker-decomposed transform parameterizations"
- path: flatquant/flat_utils.py
sha256: 892bae4c29cd91e27e07034b75c2ed69a731eded35e833c91fb204a9ff9a0284
relevance: "Kronecker matmul and diagonal-scale folding into normalization"
- path: flatquant/flat_linear.py
sha256: e5977a76d076ad6fc0b2e737bdcc9e4f0fe62d1682b3df15613c0d948a8c904b
relevance: "Activation-transform and inverse-transpose weight-folding contract"
- path: flatquant/train_utils.py
sha256: 52bd6a24a403a15be8a5d53413485ff49892796b642b4d46463140d5e5cb0104
relevance: "AdamW parameter groups, cosine schedule, and layer-wise reconstruction calibration"
- path: flatquant/model_tools/llama_utils.py
sha256: 4d5770f9f45bfc14b6d22060db8870989dfbd341b3f41ee90daab49c55665765
relevance: "QKV/O/Up-Gate/Down module placement and reparameterization"
verified_contract:
kronecker_matrix: "P = kron(P_left, P_right)"
activation_side: "X_transformed = X @ P"
weight_side: "W_folded = W @ P^{-T}"
invariant: "(X @ P) @ (W @ P^{-T})^T = X @ W^T"
diagonal_order: "When enabled, activation diagonal scaling precedes Kronecker transformation."
module_groups:
- "Attention input transform consumed by Q, K, and V projections"
- "Attention output transform consumed by O projection"
- "MLP input transform consumed by Gate and Up projections"
- "MLP activation-output transform consumed by Down projection"
not_vendored: >-
No FlatQuant source code is copied into LoopQ. This manifest pins the primary
reference; the local minimal module independently expresses only the folding
invariant required by LoopQ LQ3.
|