Offering GGUF quantizations to save conversion hassle

#2
by aviel052 - opened

Hi @Zeldeo ,

I recently performed a quantization on the Qwen 2B model to make it much easier to deploy and avoid compatibility issues with 16-bit files (such as qi_k_m.gguf formats).

I wanted to check with you: would you prefer that I send the quantized files over for you to host directly under your repository/organization, or would you rather I upload them on my end as community contributions and tag/link your original repository?

Either option works completely fine for me! I just want to make the quantized versions readily available to save people the hassle of converting the 16-bit files themselves.

Let me know what you prefer. Thanks!

Sign up or log in to comment