You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

RastNus-Base-12B-GGUF

RastNus-Base-12B-GGUF

ڕاستنووس — Kurdish Grammar & Spelling Correction

Built by PawanKrd · GGUF builds of RastNus-Base-12B · Apache 2.0

License Language GGUF Org


Overview

This repository contains GGUF builds of RastNus-Base-12B, a 12-billion-parameter model for grammar, spelling, and punctuation correction in Central Kurdish (Sorani / کوردیی ناوەندی), for use with llama.cpp and compatible runtimes (LM Studio, Ollama, llama-cpp-python, …).

All quantizations below 16-bit were made with an importance matrix computed on in-domain Kurdish correction data, which preserves quality at low bit widths. The chat template with the RastNus control switches is embedded in every file.

Model Family

Model Params bf16 FP8 NVFP4 GGUF
RastNus-Large – Not publicly available - Try it now – – –
RastNus-Medium – Under development – – –
RastNus-Base 12B bf16 FP8 NVFP4 ✅ You are here
RastNus-Small 5.1B bf16 FP8 NVFP4 GGUF
RastNus-Tiny 1.03B bf16 FP8 NVFP4 GGUF

Scores for every model and format are in the Benchmarks section below.


Files

File Bits Size Notes
RastNus-Base-12B-BF16.gguf 16-bit 23.83 GB Lossless reference
RastNus-Base-12B-Q8_0.gguf 8-bit 12.67 GB Near-lossless — recommended for quality
RastNus-Base-12B-Q6_K.gguf 6-bit 9.79 GB Near-lossless
RastNus-Base-12B-Q4_K_M.gguf 4-bit 7.38 GB Recommended for size/quality balance
RastNus-Base-12B-Q2_K.gguf 2-bit 4.83 GB ⚠️ Not recommended — no better than leaving text uncorrected

All quantizations below 16-bit use an importance matrix computed on in-domain Kurdish correction data (imatrix.gguf). See Benchmarks for the scores of every file.


Benchmarks

Every RastNus model and format was evaluated on the same internal held-out test set of 329 Central Kurdish correction examples, none of which were seen during training by any RastNus model, using greedy decoding. CER / WER = character / word error rate against the reference correction (lower is better). For reference, returning the input unchanged scores 6.34% CER. Differences of about ±0.2% CER are within noise.

vLLM — bf16, FP8, NVFP4

Model Format Size CER ↓ WER ↓ Exact match ↑ Throughput ↑
RastNus-Base-12B bf16 23.9 GB 3.68% 11.63% 20.7% 653 tok/s
RastNus-Base-12B-FP8 FP8 13.1 GB 3.75% 11.64% 21.3% 1,009 tok/s
RastNus-Base-12B-NVFP4 NVFP4 11.7 GB 3.54% 11.80% 17.3% 1,071 tok/s
RastNus-Small-5B bf16 10.2 GB 3.67% 10.76% 17.6% 2,783 tok/s
RastNus-Small-5B-FP8 FP8 8.4 GB 3.62% 10.73% 17.6% 3,521 tok/s
RastNus-Small-5B-NVFP4 NVFP4 8.0 GB 3.72% 11.29% 17.9% 3,783 tok/s
RastNus-Tiny-1B bf16 2.1 GB 4.77% 13.08% 19.5% 4,777 tok/s
RastNus-Tiny-1B-FP8 FP8 1.6 GB 4.81% 13.00% 19.5% 5,148 tok/s
RastNus-Tiny-1B-NVFP4 NVFP4 1.5 GB 4.56% 12.67% 18.5% 5,891 tok/s

Throughput = generated tokens per second for the whole test set as a batch on a single NVIDIA RTX PRO 5000 Blackwell (vLLM 0.33).

llama.cpp — GGUF

Model Quant Size CER ↓ WER ↓ Exact match ↑ Same output as BF16 ↑
RastNus-Base-12B-GGUF BF16 23.83 GB 3.69% 11.65% 20.7% 100%
RastNus-Base-12B-GGUF Q8_0 ¹ 12.67 GB 4.36% 12.13% 21.3% 86.0%
RastNus-Base-12B-GGUF Q6_K ¹ 9.79 GB 4.43% 12.33% 20.4% 78.7%
RastNus-Base-12B-GGUF Q4_K_M 7.38 GB 3.43% 11.01% 20.4% 62.9%
RastNus-Base-12B-GGUF Q2_K ⚠️ 4.83 GB 8.20% 19.19% 13.4% 22.2%
RastNus-Small-5B-GGUF BF16 9.27 GB 3.68% 10.82% 17.3% 100%
RastNus-Small-5B-GGUF Q8_0 4.95 GB 3.63% 10.76% 17.3% 90.6%
RastNus-Small-5B-GGUF Q6_K 3.83 GB 3.64% 10.77% 17.3% 86.3%
RastNus-Small-5B-GGUF Q4_K_M 3.42 GB 3.67% 10.87% 18.2% 68.7%
RastNus-Small-5B-GGUF Q2_K ⚠️ 2.98 GB 5.33% 14.80% 14.0% 32.2%
RastNus-Tiny-1B-GGUF BF16 2.06 GB 4.53% 12.69% 18.5% 100%
RastNus-Tiny-1B-GGUF Q8_0 1.11 GB 4.56% 12.70% 18.8% 91.8%
RastNus-Tiny-1B-GGUF Q6_K 0.86 GB 4.51% 12.61% 18.2% 88.4%
RastNus-Tiny-1B-GGUF Q4_K_M 0.70 GB 4.58% 12.58% 18.2% 72.9%
RastNus-Tiny-1B-GGUF Q2_K ⚠️ 0.55 GB 6.57% 15.58% 15.5% 44.4%

Measured with llama.cpp (CUDA) on the same GPU. Same output as BF16 = share of examples where the quantized file produces exactly the same text as the lossless BF16 GGUF — the most direct measure of quantization loss. ⚠️ = not recommended (no better than leaving the text uncorrected). ¹ = one test input containing many blank lines makes the model loop on newlines until the length limit; without that single example these files score at the BF16 level (≈3.7% CER). Small differences between the vLLM and llama.cpp bf16 scores come from numerical differences between the two engines, not from the weights.


Example

Input

خویندنەوەی کتیب زور گرنگە بو مروف ، چونکە بیروهۆش دەکاتەوە ئەو کەسە ی کتێب دەخوێنەوە؛ ئاستی رۆشنبیری یان بەرز ترە لەوانی تر

Output (Q4_K_M)

خوێندنەوەی کتێب زۆر گرنگە بۆ مرۆڤ، چونکە بیر و هۆش دەکاتەوە. ئەو کەسەی کتێب دەخوێنێتەوە، ئاستی ڕۆشنبیرییان بەرزترە لەوانی تر.

Prompt Format & Control Tokens

Token Effect
<|punct:on|> / <|punct:off|> Add & normalize punctuation, or leave punctuation untouched
<|pure:on|> / <|pure:off|> Replace non-Kurdish loanwords with pure Kurdish equivalents, or keep original vocabulary
<bos><|task:kurdish_correction|><|punct:on|><|pure:off|><|input|>
{your Kurdish text}
<|output|>
{corrected text}<eos>

The embedded chat template builds this for you. Defaults are punctuationEnabled=True and kurdishPureMode=False.


Usage

llama.cpp server (OpenAI-compatible)

llama-server -hf PawanKrd/RastNus-Base-12B-GGUF:Q4_K_M --jinja -ngl 99
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="none")

resp = client.chat.completions.create(
    model="RastNus-Base-12B",
    messages=[{"role": "user", "content": "خویندنەوەی کتیب زور گرنگە بو مروف"}],
    extra_body={"chat_template_kwargs": {
        "punctuationEnabled": True,
        "kurdishPureMode": False,
    }},
    temperature=0.0,
)
print(resp.choices[0].message.content)

Note: --jinja is required so that chat_template_kwargs (the punctuation / pure-Kurdish switches) are applied.

Ollama

ollama run hf.co/PawanKrd/RastNus-Base-12B-GGUF:Q4_K_M

Type or paste Kurdish text and the corrected text comes back. The repository ships an Ollama template and params, so the RastNus prompt format and greedy decoding are applied automatically.

Modes. By default punctuation correction is on and pure-Kurdish mode is off. To change the mode, send a system message:

System message Punctuation Pure Kurdish
(none) on off
pure:on on on
punct:off off off
punct:off pure:on off on
curl http://localhost:11434/api/chat -d '{
  "model": "hf.co/PawanKrd/RastNus-Base-12B-GGUF:Q4_K_M",
  "stream": false,
  "messages": [
    {"role": "system", "content": "pure:on"},
    {"role": "user", "content": "خویندنەوەی کتیب زور گرنگە بو مروف"}
  ]
}'

In the interactive CLI, use /set system pure:on. The same system messages work through Ollama's OpenAI-compatible endpoint (/v1/chat/completions).

Important: run the hf.co/... model directly. Don't wrap it in a custom model with ollama create / a Modelfile — Ollama then attaches its built-in Gemma 4 chat format, which RastNus doesn't use, and the output breaks.

Raw completion (any GGUF runtime)

If your runtime does not support chat-template arguments, send the prompt directly (without <bos> — llama.cpp adds it automatically):

curl http://localhost:8080/completion -d '{
  "prompt": "<|task:kurdish_correction|><|punct:on|><|pure:off|><|input|>\nخویندنەوەی کتیب زور گرنگە بو مروف\n<|output|>\n",
  "temperature": 0, "n_predict": 512
}'

Recommended Generation Settings

  • Greedy decoding (temperature=0) — correction is a deterministic task
  • These defaults are built in: every GGUF file carries greedy sampling metadata (temperature 0, top_k 1) and the repository includes Ollama template and params files, so no extra settings are needed
  • Max new tokens ≈ 1.5 × input token count
  • For long documents, split at sentence/paragraph boundaries (≈1,500–2,000 characters per chunk), correct each chunk, and rejoin

Intended Use & Limitations

Intended uses: local, offline proofreading of Sorani text on consumer GPUs, laptops and servers.

These GGUF files are text-only; the image/audio inputs of the original checkpoint are not included.

Limitations:

  • Targets Central Kurdish (Sorani) in Arabic script; not designed for Kurmanji (Latin script), Zazaki, or other varieties.
  • A correction model, not a chat assistant.
  • Can over-correct rare proper nouns, dialect-specific spellings, or intentional stylistic choices.
  • Human review is recommended for high-stakes publishing.

About PawanKrd

Pawan.Krd (PKRD, LLC) builds AI and cloud infrastructure — LLMs, ASR, TTS, and related APIs — with a focus on reliability and open models. RastNus is part of that work: first-class writing tools for Kurdish, released openly.


License & Attribution

Released under the Apache License, Version 2.0. See LICENSE and NOTICE.

RastNus-Base-12B-GGUF contains quantized conversions of RastNus-Base-12B, itself a derivative work of Gemma 4 12B by Google LLC / Google DeepMind (Apache 2.0). Gemma is a trademark of Google LLC, used here only to describe the origin of the base model. This project is not affiliated with, endorsed by, or sponsored by Google.

This model is provided "AS IS", without warranties or conditions of any kind.

Citation

@misc{rastnus2026,
  title   = {RastNus-Base-12B: A Central Kurdish (Sorani) Grammar and Spelling Correction Model},
  author  = {PawanKrd},
  year    = {2026},
  url     = {https://huggingface.co/PawanKrd/RastNus-Base-12B},
  note    = {Derivative of Google Gemma 4 12B, Apache License 2.0}
}
Downloads last month
105
GGUF
Model size
12B params
Architecture
gemma4
Hardware compatibility
Log In to add your hardware

2-bit

4-bit

6-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for PawanKrd/RastNus-Base-12B-GGUF

Quantized
(1)
this model

Collection including PawanKrd/RastNus-Base-12B-GGUF