README.md

---
license: apache-2.0
tags:
- gguf
- llama.cpp
- quantized
- deepseek
- stheno
---

# DeepSeek Sunfall Merged - GGUF Quantized Models

This repository contains multiple **quantized GGUF variants** of the merged DeepSeek + Sunfall model, compatible with `llama.cpp`.

## 🧠 Available Quantized Formats

| Format      | File Name                                        | Description                     |
|-------------|--------------------------------------------------|---------------------------------|
| Q3_K_M      | `deepseek_sunfall_merged_Model.Q3_K_M.gguf`      | Smallest size, fastest inference |
| Q4_K_M      | `deepseek_sunfall_merged_Model.Q4_K_M.gguf`      | Balanced speed & performance   |
| Q5_K_M      | `deepseek_sunfall_merged_Model.Q5_K_M.gguf`      | Better quality, slower         |
| Q6_K        | `deepseek_sunfall_merged_Model.Q6_K.gguf`        | Near full precision            |
| Q8_0        | `deepseek_sunfall_merged_Model.Q8_0.gguf`        | Almost no compression loss     |

## 🔧 Usage (Python)

Install `llama-cpp-python`:

```bash
pip install llama-cpp-python
from llama_cpp import Llama

model = Llama(model_path="deepseek_sunfall_merged_Model.Q4_K_M.gguf")  # or Q3_K_M, etc.
output = model("Tell me a story about stars.")
print(output)```