# Q4_K Quant of Deepseek-R1 for the MLA fork pull request
## Requires this custom build of llama.cpp:
https://github.com/ggerganov/llama.cpp/pull/11446
There's a Q3_K_M version here: [daydream-org/DeepSeek-R1-GGUF-11446](https://huggingface.co/daydream-org/DeepSeek-R1-GGUF-11446)
Created using the script below by [evshiron](https://huggingface.co/evshiron):
```python
export WORK_DIR=$(pwd)
python3 -m venv venv
source venv/bin/activate
pip3 install -U "huggingface_hub[cli]"
# the fp8 checkpoints are around 700GB
mkdir checkpoints
huggingface-cli download --resume-download --local-dir checkpoints/DeepSeek-R1 deepseek-ai/DeepSeek-R1
# my fork of llama.cpp including pr #11446 and some changes to allow converting fp8 hf to bf16 gguf directly using triton(-cpu) without the need of intermediate checkpoints
git clone https://github.com/evshiron/llama.cpp --recursive
pushd llama.cpp
pip3 install -r requirements/requirements-convert_hf_to_gguf.txt
cmake -B build
cmake --build build --conf…
