Hugging Face's logo

CompressedGemma
/
HPC-Quantize-V2

Model card Files
xet
 Community
CompressedGemma commited on
Commit
a0450dc
·
verified ·
1 Parent(s): 11c45f9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -6,7 +6,7 @@ license: mit
6
 
7
  This requires a CUDA GPU, expect a 50GB model to take around 3-4 hours to quantize ona 3060 12GB.
8
 
9
- I prefer to llama-quantize to 8_0, generate a imatrix from 8_0 and then quantize the 8_0 fileto Q2K.
10
 
11
  If all goes as expected, it should be a substantial improvement over V1, which alone outperformed GCQ-RSQ; it is not even a contest of quality but time taken to quantize.
12
 
 
6
 
7
  This requires a CUDA GPU, expect a 50GB model to take around 3-4 hours to quantize ona 3060 12GB.
8
 
9
+ I prefer to llama-quantize to 8_0, generate a imatrix from 8_0 and then quantize the 8_0 file to Q2K.
10
 
11
  If all goes as expected, it should be a substantial improvement over V1, which alone outperformed GCQ-RSQ; it is not even a contest of quality but time taken to quantize.
12