Update README.md
Browse files
README.md
CHANGED
|
@@ -2,6 +2,17 @@
|
|
| 2 |
license: mit
|
| 3 |
---
|
| 4 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
# HPC-Quantize
|
| 6 |
|
| 7 |
### Holographic Phase Contraction for Ultra-Low-Bit LLM Quantization
|
|
|
|
| 2 |
license: mit
|
| 3 |
---
|
| 4 |
|
| 5 |
+
# Disclaimer
|
| 6 |
+
|
| 7 |
+
With recent changes, the runtime of a quantization operation has gone through the roof, but models quantized this way benefit from a massive increase in model coherency.
|
| 8 |
+
|
| 9 |
+
I am not going to revert these changes, but will attempt to streamline this further.
|
| 10 |
+
|
| 11 |
+
To use this properly you must dump a .hxh file and enable --hessian and --analog-imatrix, otherwise it will fall back to the typical RMSE path which is still vastly superior to everything else.
|
| 12 |
+
|
| 13 |
+
It is worth the cost of using --hessian and --analog-imatrix, though.
|
| 14 |
+
|
| 15 |
+
|
| 16 |
# HPC-Quantize
|
| 17 |
|
| 18 |
### Holographic Phase Contraction for Ultra-Low-Bit LLM Quantization
|