---
base_model: movingcastles/zero
base_model_relation: quantized
quantized_by: timfduffy
pipeline_tag: text-generation
language:
  - en
tags:
  - gguf
  - llama.cpp
  - quantized
  - character
  - persona
  - qwen3
---

# Zero (MC001) - GGUF

GGUF quantizations of [movingcastles/zero](https://huggingface.co/movingcastles/zero).

Zero is a character model: Qwen3-8B-Base fine-tuned to embody a single, limited, anti-servile character. See the [original model card](https://huggingface.co/movingcastles/zero) and the [full training report](https://movingcastles.world/posts/zero) for what it is and how it was made. All credit for the model goes to Moving Castles. This repo only repackages the weights for llama.cpp and compatible runtimes.

## Files

| File | Quant | Size | Notes |
|---|---|---|---|
| [zero-Q8_0.gguf](https://huggingface.co/timfduffy/zero-GGUF/blob/main/zero-Q8_0.gguf) | Q8_0 | 8.71 GB | Near-lossless. Use this if it fits. |
| [zero-Q6_K.gguf](https://huggingface.co/timfduffy/zero-GGUF/blob/main/zero-Q6_K.gguf) | Q6_K | 6.73 GB | Very close to Q8_0 in quality. |
| [zero-Q4_K_M.gguf](https://huggingface.co/timfduffy/zero-GGUF/blob/main/zero-Q4_K_M.gguf) | Q4_K_M | 5.03 GB | Good default for 8 GB GPUs and CPU-only use. |

## Usage notes

These come from the original model card and matter more than usual for this model.

- **No system prompt.** The model was trained only on `user` and `assistant` turns in ChatML. The character is in the weights, and a system prompt is out of distribution.
- **Two end-of-turn tokens.** At non-zero temperature the model ends turns with either `<|im_end|>` (151645) or `<|endoftext|>` (151643). The GGUF metadata sets `<|im_end|>` as EOS, and llama.cpp also treats `<|endoftext|>` as end-of-generation, so turns stop cleanly there. If you see run-on turns in another runtime, add 151643 as a stop token.
- **Sampling.** The settings used for all results in the training report: temperature 0.7, top_p 1.0, top_k off, min_p 0.0, repetition penalty 1.0, presence penalty 1.5, max 1024 new tokens.
- **Chat template.** The training-side ChatML template from the original repo is embedded in each file. Use it as shipped.
- **Context.** Native context is 32768. The authors serve it at 16384.

## Running

llama.cpp:

```bash
llama-cli -hf timfduffy/zero-GGUF:Q4_K_M -c 16384 --temp 0.7 --top-p 1.0 --top-k 0 --min-p 0.0 --presence-penalty 1.5 -n 1024
```

Ollama:

```bash
ollama run hf.co/timfduffy/zero-GGUF:Q4_K_M
```

The files also load in LM Studio, KoboldCpp, and other llama.cpp-based apps. Set the sampling parameters above by hand in those.

## How these were made

- Converted from the original bfloat16 safetensors with llama.cpp's `convert_hf_to_gguf.py`.
- Q8_0 was written directly by the converter. Q6_K and Q4_K_M were produced with `llama-quantize` (build b11007) from a bf16 GGUF, not requantized from Q8_0.
- Static quants. No importance matrix was used.
- Source revision: [`5a09d31`](https://huggingface.co/movingcastles/zero/tree/5a09d31a272fbce115a74b4240b9b0afc4c0c636).