Instructions to use unsloth/DeepSeek-R1-Zero-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use unsloth/DeepSeek-R1-Zero-GGUF with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("unsloth/DeepSeek-R1-Zero-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use unsloth/DeepSeek-R1-Zero-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M
Use Docker
docker model run hf.co/unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use unsloth/DeepSeek-R1-Zero-GGUF with Ollama:
ollama run hf.co/unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use unsloth/DeepSeek-R1-Zero-GGUF with Docker Model Runner:
docker model run hf.co/unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M
- Lemonade
How to use unsloth/DeepSeek-R1-Zero-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull unsloth/DeepSeek-R1-Zero-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.DeepSeek-R1-Zero-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Commit History
-
Update README.md
Verified signature7a2f0f7
-
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00016-of-00016.gguf with huggingface_hub
Verified signature4036508 -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00015-of-00016.gguf with huggingface_hub
Verified signatured7ca830 -
Upload DeepSeek-R1-Zero-Q4_K_M/DeepSeek-R1-Zero-Q4_K_M-00009-of-00009.gguf with huggingface_hub
Verified signature5db1233 -
Update README.md
Verified signature1a5b839 -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00014-of-00016.gguf with huggingface_hub
Verified signature250d329 -
Upload DeepSeek-R1-Zero-Q4_K_M/DeepSeek-R1-Zero-Q4_K_M-00008-of-00009.gguf with huggingface_hub
Verified signature8227bef -
Upload DeepSeek-R1-Zero-Q4_K_M/DeepSeek-R1-Zero-Q4_K_M-00007-of-00009.gguf with huggingface_hub
Verified signature7c6bc2c -
Upload DeepSeek-R1-Zero-Q4_K_M/DeepSeek-R1-Zero-Q4_K_M-00006-of-00009.gguf with huggingface_hub
Verified signaturede99acd -
Upload DeepSeek-R1-Zero-Q4_K_M/DeepSeek-R1-Zero-Q4_K_M-00005-of-00009.gguf with huggingface_hub
Verified signaturefa73e2c -
Upload DeepSeek-R1-Zero-Q4_K_M/DeepSeek-R1-Zero-Q4_K_M-00004-of-00009.gguf with huggingface_hub
Verified signature9925ea9 -
Upload DeepSeek-R1-Zero-Q4_K_M/DeepSeek-R1-Zero-Q4_K_M-00003-of-00009.gguf with huggingface_hub
Verified signaturecfbf466 -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00013-of-00016.gguf with huggingface_hub
Verified signaturee3ab1e8 -
Upload DeepSeek-R1-Zero-Q4_K_M/DeepSeek-R1-Zero-Q4_K_M-00002-of-00009.gguf with huggingface_hub
Verified signaturedfa2567 -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00012-of-00016.gguf with huggingface_hub
Verified signatured689cc8 -
Upload DeepSeek-R1-Zero-Q4_K_M/DeepSeek-R1-Zero-Q4_K_M-00001-of-00009.gguf with huggingface_hub
Verified signature6e929da -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00011-of-00016.gguf with huggingface_hub
Verified signature2316e72 -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00010-of-00016.gguf with huggingface_hub
Verified signature9561216 -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00009-of-00016.gguf with huggingface_hub
Verified signature3c223bb -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00008-of-00016.gguf with huggingface_hub
Verified signature2d46bd2 -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00007-of-00016.gguf with huggingface_hub
Verified signature44536d1 -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00006-of-00016.gguf with huggingface_hub
Verified signature9e5098d -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00005-of-00016.gguf with huggingface_hub
Verified signature1978b76 -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00004-of-00016.gguf with huggingface_hub
Verified signature28e0c0e -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00003-of-00016.gguf with huggingface_hub
Verified signature97e2ddb -
Upload DeepSeek-R1-Zero-Q3_K_M/DeepSeek-R1-Zero-Q3_K_M-00007-of-00007.gguf with huggingface_hub
Verified signature59f3770 -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00002-of-00016.gguf with huggingface_hub
Verified signatureaace6bc -
Upload DeepSeek-R1-Zero-Q3_K_M/DeepSeek-R1-Zero-Q3_K_M-00006-of-00007.gguf with huggingface_hub
Verified signaturebad8e52 -
Upload DeepSeek-R1-Zero-Q8_0/DeepSeek-R1-Zero-BF16-256x20B-Q8_0-00001-of-00016.gguf with huggingface_hub
Verified signatureb62fd17 -
Upload DeepSeek-R1-Zero-Q3_K_M/DeepSeek-R1-Zero-Q3_K_M-00005-of-00007.gguf with huggingface_hub
Verified signaturedd6ae93 -
Upload DeepSeek-R1-Zero-Q3_K_M/DeepSeek-R1-Zero-Q3_K_M-00004-of-00007.gguf with huggingface_hub
Verified signature0a5b66d -
Upload DeepSeek-R1-Zero-Q3_K_M/DeepSeek-R1-Zero-Q3_K_M-00003-of-00007.gguf with huggingface_hub
Verified signatureeeaa064 -
Upload DeepSeek-R1-Zero-Q3_K_M/DeepSeek-R1-Zero-Q3_K_M-00002-of-00007.gguf with huggingface_hub
Verified signature9c39ca7 -
Upload DeepSeek-R1-Zero-Q3_K_M/DeepSeek-R1-Zero-Q3_K_M-00001-of-00007.gguf with huggingface_hub
Verified signature3f9d06e -
Update README.md
Verified signatured9552a1 -
Upload DeepSeek-R1-Zero-Q2_K_XS/DeepSeek-R1-Zero-Q2_K_XS-00005-of-00005.gguf with huggingface_hub
Verified signature6ddad4f -
Upload DeepSeek-R1-Zero-Q2_K_XS/DeepSeek-R1-Zero-Q2_K_XS-00004-of-00005.gguf with huggingface_hub
Verified signaturee61cc79 -
Upload DeepSeek-R1-Zero-Q2_K_XS/DeepSeek-R1-Zero-Q2_K_XS-00003-of-00005.gguf with huggingface_hub
Verified signature93949bf -
Update README.md
Verified signature1470315 -
Upload DeepSeek-R1-Zero-Q2_K_XS/DeepSeek-R1-Zero-Q2_K_XS-00002-of-00005.gguf with huggingface_hub
Verified signature40ce8d1 -
Upload DeepSeek-R1-Zero-Q2_K_XS/DeepSeek-R1-Zero-Q2_K_XS-00001-of-00005.gguf with huggingface_hub
Verified signature76e5af0 -
Upload DeepSeek-R1-Zero-Q2_K_L/DeepSeek-R1-Zero-Q2_K_L-00005-of-00005.gguf with huggingface_hub
Verified signature29f025e -
Upload DeepSeek-R1-Zero-Q2_K_L/DeepSeek-R1-Zero-Q2_K_L-00004-of-00005.gguf with huggingface_hub
Verified signaturedb962f4 -
Upload DeepSeek-R1-Zero-Q2_K_L/DeepSeek-R1-Zero-Q2_K_L-00003-of-00005.gguf with huggingface_hub
Verified signature6090d2c -
Upload DeepSeek-R1-Zero-Q2_K_L/DeepSeek-R1-Zero-Q2_K_L-00002-of-00005.gguf with huggingface_hub
Verified signature0ea103f -
Upload DeepSeek-R1-Zero-Q2_K_L/DeepSeek-R1-Zero-Q2_K_L-00001-of-00005.gguf with huggingface_hub
Verified signature5d2214e -
Create README.md
Verified signaturebd91163 -
initial commit
Verified signatureed1f5ee