RastNus-Small-5B-NVFP4

ڕاستنووس — Kurdish Grammar & Spelling Correction

Built by PawanKrd · Based on Google Gemma 4 E2B · Apache 2.0

---

Overview

This repository contains the NVFP4-quantized version of RastNus-Small-5B for fast serving with vLLM.

RastNus (ڕاستنووس, "correct writing") is a 5-billion-parameter (~2B effective) language model specialized in grammar, spelling, and punctuation correction for Central Kurdish (Sorani / کوردیی ناوەندی).

Kurdish is one of the most under-served languages in modern NLP. Everyday written Sorani commonly contains orthographic inconsistencies — missing diacritic letters (ی/ێ, و/ۆ, ر/ڕ), spacing errors, informal punctuation, and mixed loanword usage. RastNus takes raw Sorani text and returns a clean, standardized, correctly written version, while preserving the author's meaning and voice.

RastNus is developed and released by PawanKrd as part of our commitment to bringing state-of-the-art AI to the Kurdish language and its speakers.

Key Capabilities

• Spelling correction — fixes common orthographic errors and non-standard letter usage (e.g. کتیب → کتێب, زور → زۆر, مروف → مرۆڤ)
• Grammar correction — repairs agreement, morphology, and word-form errors
• Punctuation control — optionally adds and normalizes punctuation, or leaves punctuation untouched, via a dedicated control token
• Pure Kurdish mode — optionally replaces non-Kurdish loanwords with pure Kurdish equivalents, via a dedicated control token
• Meaning-preserving — corrects without paraphrasing, summarizing, or adding content

Model Family

Model	Params	bf16	FP8	NVFP4	GGUF
RastNus-Large	–	Not publicly available - Try it now	–	–	–
RastNus-Medium	–	Under development	–	–	–
RastNus-Base	12B	bf16	FP8	NVFP4	GGUF
RastNus-Small	5.1B	bf16	FP8	✅ You are here	GGUF
RastNus-Tiny	1.03B	bf16	FP8	NVFP4	GGUF

Scores for every model and format are in the Benchmarks section below.

---

Benchmarks

Every RastNus model and format was evaluated on the same internal held-out test set of 329 Central Kurdish correction examples, none of which were seen during training by any RastNus model, using greedy decoding. CER / WER = character / word error rate against the reference correction (lower is better). For reference, returning the input unchanged scores 6.34% CER. Differences of about ±0.2% CER are within noise.

vLLM — bf16, FP8, NVFP4

Model	Format	Size	CER ↓	WER ↓	Exact match ↑	Throughput ↑
RastNus-Base-12B	bf16	23.9 GB	3.68%	11.63%	20.7%	653 tok/s
RastNus-Base-12B-FP8	FP8	13.1 GB	3.75%	11.64%	21.3%	1,009 tok/s
RastNus-Base-12B-NVFP4	NVFP4	11.7 GB	3.54%	11.80%	17.3%	1,071 tok/s
RastNus-Small-5B	bf16	10.2 GB	3.67%	10.76%	17.6%	2,783 tok/s
RastNus-Small-5B-FP8	FP8	8.4 GB	3.62%	10.73%	17.6%	3,521 tok/s
RastNus-Small-5B-NVFP4	NVFP4	8.0 GB	3.72%	11.29%	17.9%	3,783 tok/s
RastNus-Tiny-1B	bf16	2.1 GB	4.77%	13.08%	19.5%	4,777 tok/s
RastNus-Tiny-1B-FP8	FP8	1.6 GB	4.81%	13.00%	19.5%	5,148 tok/s
RastNus-Tiny-1B-NVFP4	NVFP4	1.5 GB	4.56%	12.67%	18.5%	5,891 tok/s

Throughput = generated tokens per second for the whole test set as a batch on a single NVIDIA RTX PRO 5000 Blackwell (vLLM 0.33).

llama.cpp — GGUF

Model	Quant	Size	CER ↓	WER ↓	Exact match ↑	Same output as BF16 ↑
RastNus-Base-12B-GGUF	BF16	23.83 GB	3.69%	11.65%	20.7%	100%
RastNus-Base-12B-GGUF	Q8_0 ¹	12.67 GB	4.36%	12.13%	21.3%	86.0%
RastNus-Base-12B-GGUF	Q6_K ¹	9.79 GB	4.43%	12.33%	20.4%	78.7%
RastNus-Base-12B-GGUF	Q4_K_M	7.38 GB	3.43%	11.01%	20.4%	62.9%
RastNus-Base-12B-GGUF	Q2_K ⚠️	4.83 GB	8.20%	19.19%	13.4%	22.2%
RastNus-Small-5B-GGUF	BF16	9.27 GB	3.68%	10.82%	17.3%	100%
RastNus-Small-5B-GGUF	Q8_0	4.95 GB	3.63%	10.76%	17.3%	90.6%
RastNus-Small-5B-GGUF	Q6_K	3.83 GB	3.64%	10.77%	17.3%	86.3%
RastNus-Small-5B-GGUF	Q4_K_M	3.42 GB	3.67%	10.87%	18.2%	68.7%
RastNus-Small-5B-GGUF	Q2_K ⚠️	2.98 GB	5.33%	14.80%	14.0%	32.2%
RastNus-Tiny-1B-GGUF	BF16	2.06 GB	4.53%	12.69%	18.5%	100%
RastNus-Tiny-1B-GGUF	Q8_0	1.11 GB	4.56%	12.70%	18.8%	91.8%
RastNus-Tiny-1B-GGUF	Q6_K	0.86 GB	4.51%	12.61%	18.2%	88.4%
RastNus-Tiny-1B-GGUF	Q4_K_M	0.70 GB	4.58%	12.58%	18.2%	72.9%
RastNus-Tiny-1B-GGUF	Q2_K ⚠️	0.55 GB	6.57%	15.58%	15.5%	44.4%

Measured with llama.cpp (CUDA) on the same GPU. Same output as BF16 = share of examples where the quantized file produces exactly the same text as the lossless BF16 GGUF — the most direct measure of quantization loss. ⚠️ = not recommended (no better than leaving the text uncorrected). ¹ = one test input containing many blank lines makes the model loop on newlines until the length limit; without that single example these files score at the BF16 level (≈3.7% CER). Small differences between the vLLM and llama.cpp bf16 scores come from numerical differences between the two engines, not from the weights.

---

Example

Input

خویندنەوەی کتیب زور گرنگە بو مروف ، چونکە بیروهۆش دەکاتەوە ئەو کەسە ی کتێب دەخوێنەوە؛ ئاستی رۆشنبیری یان بەرز ترە لەوانی تر

Output

خوێندنەوەی کتێب زۆر گرنگە بۆ مرۆڤ، چونکە بیروهۆش دەکاتەوە. ئەو کەسەی کتێب دەخوێنێتەوە، ئاستی ڕۆشنبیرییان بەرزترە لەوانی تر.

---

Model Details

Developed by	PawanKrd
Model type	Causal decoder-only transformer (Gemma 4 architecture)
Base model	Google Gemma 4 E2B
Parameters	5.1B total (2B effective; includes per-layer embeddings)
Language	Central Kurdish / Sorani (ckb), Arabic script
Task	Text correction (grammar, spelling, punctuation)
Precision	NVFP4 (MLP), bf16 (attention, embeddings)
Context length	Long-context (inherited from Gemma 4); recommended working length up to ~8K tokens per request
License	Apache 2.0

Prompt Format & Control Tokens

RastNus uses a task-specific prompt format with dedicated special tokens (added to the tokenizer vocabulary). The behavior of the model is controlled by two switches:

Token	Effect
<|punct:on|> / <|punct:off|>	Add & normalize punctuation, or leave punctuation untouched
<|pure:on|> / <|pure:off|>	Replace non-Kurdish loanwords with pure Kurdish equivalents, or keep original vocabulary

The full sequence layout:

<bos><|task:kurdish_correction|><|punct:on|><|pure:off|><|input|>
{your Kurdish text}
<|output|>
{corrected text}<eos>

You do not need to build this manually — the repository ships with a chat template that handles it (see below). Defaults are punctuationEnabled=True and kurdishPureMode=False.

Quantization

NVFP4 (4-bit floating point) weights and activations for the MLP layers, produced with llm-compressor and calibrated on 512 in-domain Kurdish correction examples. Attention layers, embeddings (including the large per-layer embedding tables), lm_head and the vision/audio towers are kept in bf16.

Hardware: NVIDIA Blackwell GPUs (e.g. RTX 50xx, RTX PRO 6000/5000, B200).

See Benchmarks for the quality and speed of every format.

---

Usage

Transformers

Requires pip install compressed-tensors (the quantized weights are loaded automatically).

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "PawanKrd/RastNus-Small-5B-NVFP4"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)

text = "خویندنەوەی کتیب زور گرنگە بو مروف"

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": text}],
    tokenize=False,
    add_generation_prompt=True,
    punctuationEnabled=True,   # add/normalize punctuation
    kurdishPureMode=False,     # keep loanwords as-is
)

inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Note: pass add_special_tokens=False when tokenizing — the chat template already emits <bos>.

vLLM (OpenAI-compatible server)

vllm serve PawanKrd/RastNus-Small-5B-NVFP4

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")

resp = client.chat.completions.create(
    model="PawanKrd/RastNus-Small-5B-NVFP4",
    messages=[{"role": "user", "content": "خویندنەوەی کتیب زور گرنگە بو مروف"}],
    extra_body={"chat_template_kwargs": {
        "punctuationEnabled": True,
        "kurdishPureMode": True,   # enable pure-Kurdish word replacement
    }},
    temperature=0.0,
)
print(resp.choices[0].message.content)

Recommended Generation Settings

• Greedy decoding (do_sample=False / temperature=0) — correction is a deterministic task
• These defaults are built in: generation_config.json sets greedy decoding, so model.generate() needs no extra sampling arguments
• max_new_tokens ≈ 1.5 × input token count (the corrected output is roughly the same length as the input)
• For documents longer than ~4,000–5,000 characters, split at sentence/paragraph boundaries, correct each chunk, and rejoin

---

Intended Use & Limitations

Intended uses: proofreading and standardization of Sorani text in publishing, education, journalism, document processing, writing assistants, and any pipeline that needs clean, orthographically consistent Kurdish.

Limitations:

• The model targets Central Kurdish (Sorani) written in Arabic script. It is not designed for Kurmanji (Latin script), Zazaki, or other Kurdish varieties.
• It is a correction model, not a general chat assistant — it expects the task prompt format above and will not follow open-ended instructions.
• Like all language models, it can occasionally miss errors or over-correct rare proper nouns, dialect-specific spellings, or intentional stylistic choices. Human review is recommended for high-stakes publishing.
• Pure-Kurdish mode applies vocabulary substitutions that may not match every publication's style guide; review its output for terminology-sensitive content.

---

About PawanKrd

Pawan.Krd (PKRD, LLC) builds AI and cloud infrastructure — LLMs, ASR, TTS, and related APIs — with a focus on reliability and open models. RastNus is part of that work: first-class writing tools for Kurdish, released openly.

• 🌐 Website: https://pawan.krd
• 🤗 Hugging Face: https://huggingface.co/PawanKrd

---

License & Attribution

This model is released under the Apache License, Version 2.0. See the LICENSE file for the full text and the NOTICE file for attribution details.

RastNus-Small-5B-NVFP4 is a quantized version of RastNus-Small-5B, a derivative work of Gemma 4 E2B by Google LLC / Google DeepMind, which is also licensed under Apache 2.0. In accordance with the license:

• The original Apache 2.0 license text is included in this repository.
• The NOTICE file carries the required attribution to Google and a prominent statement that the model weights, tokenizer, and chat template have been modified by PawanKrd.
• Gemma is a trademark of Google LLC, used here only to describe the origin of the base model. This project is not affiliated with, endorsed by, or sponsored by Google.

This model is provided "AS IS", without warranties or conditions of any kind. See Sections 7 and 8 of the Apache License for the disclaimer of warranty and limitation of liability.

Citation

@misc{rastnussmall2026,
  title   = {RastNus-Small-5B: A Central Kurdish (Sorani) Grammar and Spelling Correction Model},
  author  = {PawanKrd},
  year    = {2026},
  url     = {https://huggingface.co/PawanKrd/RastNus-Small-5B},
  note    = {Derivative of Google Gemma 4 E2B, Apache License 2.0}
}