Update README.md
Browse files
README.md
CHANGED
|
@@ -4,7 +4,9 @@ license: mit
|
|
| 4 |
|
| 5 |
# Developer Note
|
| 6 |
|
| 7 |
-
This requires a CUDA GPU, expect a 50GB model to take around 3-4 hours to quantize.
|
|
|
|
|
|
|
| 8 |
|
| 9 |
If all goes as expected, it should be a substantial improvement over V1, which alone outperformed GCQ-RSQ; it is not even a contest of quality but time taken to quantize.
|
| 10 |
|
|
|
|
| 4 |
|
| 5 |
# Developer Note
|
| 6 |
|
| 7 |
+
This requires a CUDA GPU, expect a 50GB model to take around 3-4 hours to quantize ona 3060 12GB.
|
| 8 |
+
|
| 9 |
+
I prefer to llama-quantize to 8_0, generate a imatrix from 8_0 and then quantize the 8_0 fileto Q2K.
|
| 10 |
|
| 11 |
If all goes as expected, it should be a substantial improvement over V1, which alone outperformed GCQ-RSQ; it is not even a contest of quality but time taken to quantize.
|
| 12 |
|