Zen-5-Coder GGUF Quantizations
This repository provides GGUF quantizations of Zen-5-Coder 80B for llama.cpp and compatible runtimes.
The original model was released by Zen LM in Hugging Face Transformers format. These files were converted to GGUF and quantized locally using llama.cpp.
Available Quantizations
Quantization Intended Use
IQ3_M Lowest memory usage
Q3_K_M Balanced low-memory option
Q4_K_M Recommended default
Q5_K_M Higher quality
Q6_K Near-lossless experience
Q8_0 Maximum quality quantization
Conversion Process
HF Transformers → GGUF FP16 → llama.cpp Quantization
Conversion performed using:
llama.cpp
convert_hf_to_gguf.py
llama-quantize
Tested Hardware
NVIDIA RTX 3060 12GB
NVIDIA Tesla P40 24GB
64GB RAM
Debian Linux
Credits
Original model: Zen LM
GGUF conversion and quantization: VibeManGeo
If these files save you several hours of conversion work, consider supporting the original model creators.
Zen-5-Coder GGUF Quantizations
Code-specialized member of the [Zen5](https://zenlm.org) family. 80B-parameter sparse MoE tuned for repo-scale code understanding, agentic refactoring, and tool-use coding loops.
This repository provides GGUF quantizations of Zen-5-Coder 80B for llama.cpp and compatible runtimes.
The original model was released by Zen LM in Hugging Face Transformers format. These files were converted to GGUF and quantized locally using llama.cpp.
Available Quantizations
Quantization Intended Use
IQ3_M Lowest memory usage
Q3_K_M Balanced low-memory option
Q4_K_M Recommended default
Q5_K_M Higher quality
Q6_K Near-lossless experience
Q8_0 Maximum quality quantization
Conversion Process
HF Transformers → GGUF FP16 → llama.cpp Quantization
Conversion performed using:
llama.cpp
convert_hf_to_gguf.py
llama-quantize
Tested Hardware
NVIDIA RTX 3060 12GB
NVIDIA Tesla P40 24GB
64GB RAM
Debian Linux
Credits
Original model: Zen LM
GGUF conversion and quantization: VibeManGeo…
