| --- |
| base_model: movingcastles/zero |
| base_model_relation: quantized |
| quantized_by: timfduffy |
| pipeline_tag: text-generation |
| language: |
| - en |
| tags: |
| - gguf |
| - llama.cpp |
| - quantized |
| - character |
| - persona |
| - qwen3 |
| --- |
| |
| # Zero (MC001) - GGUF |
|
|
| GGUF quantizations of [movingcastles/zero](https://huggingface.co/movingcastles/zero). |
|
|
| Zero is a character model: Qwen3-8B-Base fine-tuned to embody a single, limited, anti-servile character. See the [original model card](https://huggingface.co/movingcastles/zero) and the [full training report](https://movingcastles.world/posts/zero) for what it is and how it was made. All credit for the model goes to Moving Castles. This repo only repackages the weights for llama.cpp and compatible runtimes. |
|
|
| ## Files |
|
|
| | File | Quant | Size | Notes | |
| |---|---|---|---| |
| | [zero-Q8_0.gguf](https://huggingface.co/timfduffy/zero-GGUF/blob/main/zero-Q8_0.gguf) | Q8_0 | 8.71 GB | Near-lossless. Use this if it fits. | |
| | [zero-Q6_K.gguf](https://huggingface.co/timfduffy/zero-GGUF/blob/main/zero-Q6_K.gguf) | Q6_K | 6.73 GB | Very close to Q8_0 in quality. | |
| | [zero-Q4_K_M.gguf](https://huggingface.co/timfduffy/zero-GGUF/blob/main/zero-Q4_K_M.gguf) | Q4_K_M | 5.03 GB | Good default for 8 GB GPUs and CPU-only use. | |
| |
| ## Usage notes |
| |
| These come from the original model card and matter more than usual for this model. |
| |
| - **No system prompt.** The model was trained only on `user` and `assistant` turns in ChatML. The character is in the weights, and a system prompt is out of distribution. |
| - **Two end-of-turn tokens.** At non-zero temperature the model ends turns with either `<|im_end|>` (151645) or `<|endoftext|>` (151643). The GGUF metadata sets `<|im_end|>` as EOS, and llama.cpp also treats `<|endoftext|>` as end-of-generation, so turns stop cleanly there. If you see run-on turns in another runtime, add 151643 as a stop token. |
| - **Sampling.** The settings used for all results in the training report: temperature 0.7, top_p 1.0, top_k off, min_p 0.0, repetition penalty 1.0, presence penalty 1.5, max 1024 new tokens. |
| - **Chat template.** The training-side ChatML template from the original repo is embedded in each file. Use it as shipped. |
| - **Context.** Native context is 32768. The authors serve it at 16384. |
|
|
| ## Running |
|
|
| llama.cpp: |
|
|
| ```bash |
| llama-cli -hf timfduffy/zero-GGUF:Q4_K_M -c 16384 --temp 0.7 --top-p 1.0 --top-k 0 --min-p 0.0 --presence-penalty 1.5 -n 1024 |
| ``` |
|
|
| Ollama: |
|
|
| ```bash |
| ollama run hf.co/timfduffy/zero-GGUF:Q4_K_M |
| ``` |
|
|
| The files also load in LM Studio, KoboldCpp, and other llama.cpp-based apps. Set the sampling parameters above by hand in those. |
|
|
| ## How these were made |
|
|
| - Converted from the original bfloat16 safetensors with llama.cpp's `convert_hf_to_gguf.py`. |
| - Q8_0 was written directly by the converter. Q6_K and Q4_K_M were produced with `llama-quantize` (build b11007) from a bf16 GGUF, not requantized from Q8_0. |
| - Static quants. No importance matrix was used. |
| - Source revision: [`5a09d31`](https://huggingface.co/movingcastles/zero/tree/5a09d31a272fbce115a74b4240b9b0afc4c0c636). |
