Instructions to use PawanKrd/RastNus-Tiny-1B-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PawanKrd/RastNus-Tiny-1B-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PawanKrd/RastNus-Tiny-1B-FP8")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("PawanKrd/RastNus-Tiny-1B-FP8")
model = AutoModelForCausalLM.from_pretrained("PawanKrd/RastNus-Tiny-1B-FP8", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use PawanKrd/RastNus-Tiny-1B-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PawanKrd/RastNus-Tiny-1B-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PawanKrd/RastNus-Tiny-1B-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/PawanKrd/RastNus-Tiny-1B-FP8

SGLang

How to use PawanKrd/RastNus-Tiny-1B-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PawanKrd/RastNus-Tiny-1B-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PawanKrd/RastNus-Tiny-1B-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PawanKrd/RastNus-Tiny-1B-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PawanKrd/RastNus-Tiny-1B-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use PawanKrd/RastNus-Tiny-1B-FP8 with Docker Model Runner:
```
docker model run hf.co/PawanKrd/RastNus-Tiny-1B-FP8
```

You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

RastNus-Tiny-1B-FP8

ڕاستنووس — Kurdish Grammar & Spelling Correction

Built by PawanKrd · Derived from Google Gemma 4 E2B · Apache 2.0

Overview

This repository contains the FP8-quantized version of RastNus-Tiny-1B for fast serving with vLLM.

RastNus-Tiny (ڕاستنووس, "correct writing") is a 1-billion-parameter model for grammar, spelling, and punctuation correction in Central Kurdish (Sorani / کوردیی ناوەندی).

It is the smallest member of the RastNus family, built for fast, cheap, high-throughput correction: on a single GPU with vLLM it generates ~6,000 tokens/s in batch, and it fits comfortably on small GPUs and edge hardware (~2 GB of weights in bf16).

RastNus-Tiny was created by fine-tuning Gemma 4 E2B on Kurdish correction, then structurally pruning it to ~1B parameters and recovering quality with knowledge distillation from the fine-tuned teacher.

Key Capabilities

Spelling correction — fixes common orthographic errors and non-standard letter usage (e.g. کتیب → کتێب, زور → زۆر, مروف → مرۆڤ)
Grammar correction — repairs agreement, morphology, and word-form errors
Punctuation control — optionally adds and normalizes punctuation, or leaves punctuation untouched, via a dedicated control token
Pure Kurdish mode — optionally replaces non-Kurdish loanwords with pure Kurdish equivalents, via a dedicated control token
Meaning-preserving — corrects without paraphrasing, summarizing, or adding content

Model Family

Model	Params	bf16	FP8	NVFP4	GGUF
RastNus-Large	–	Not publicly available - Try it now	–	–	–
RastNus-Medium	–	Under development	–	–	–
RastNus-Base	12B	bf16	FP8	NVFP4	GGUF
RastNus-Small	5.1B	bf16	FP8	NVFP4	GGUF
RastNus-Tiny	1.03B	bf16	✅ You are here	NVFP4	GGUF

Scores for every model and format are in the Benchmarks section below.

Benchmarks

Every RastNus model and format was evaluated on the same internal held-out test set of 329 Central Kurdish correction examples, none of which were seen during training by any RastNus model, using greedy decoding. CER / WER = character / word error rate against the reference correction (lower is better). For reference, returning the input unchanged scores 6.34% CER. Differences of about ±0.2% CER are within noise.

vLLM — bf16, FP8, NVFP4

Model	Format	Size	CER ↓	WER ↓	Exact match ↑	Throughput ↑
RastNus-Base-12B	bf16	23.9 GB	3.68%	11.63%	20.7%	653 tok/s
RastNus-Base-12B-FP8	FP8	13.1 GB	3.75%	11.64%	21.3%	1,009 tok/s
RastNus-Base-12B-NVFP4	NVFP4	11.7 GB	3.54%	11.80%	17.3%	1,071 tok/s
RastNus-Small-5B	bf16	10.2 GB	3.67%	10.76%	17.6%	2,783 tok/s
RastNus-Small-5B-FP8	FP8	8.4 GB	3.62%	10.73%	17.6%	3,521 tok/s
RastNus-Small-5B-NVFP4	NVFP4	8.0 GB	3.72%	11.29%	17.9%	3,783 tok/s
RastNus-Tiny-1B	bf16	2.1 GB	4.77%	13.08%	19.5%	4,777 tok/s
RastNus-Tiny-1B-FP8	FP8	1.6 GB	4.81%	13.00%	19.5%	5,148 tok/s
RastNus-Tiny-1B-NVFP4	NVFP4	1.5 GB	4.56%	12.67%	18.5%	5,891 tok/s

Throughput = generated tokens per second for the whole test set as a batch on a single NVIDIA RTX PRO 5000 Blackwell (vLLM 0.33).

llama.cpp — GGUF

Model	Quant	Size	CER ↓	WER ↓	Exact match ↑	Same output as BF16 ↑
RastNus-Base-12B-GGUF	`BF16`	23.83 GB	3.69%	11.65%	20.7%	100%
RastNus-Base-12B-GGUF	`Q8_0` ¹	12.67 GB	4.36%	12.13%	21.3%	86.0%
RastNus-Base-12B-GGUF	`Q6_K` ¹	9.79 GB	4.43%	12.33%	20.4%	78.7%
RastNus-Base-12B-GGUF	`Q4_K_M`	7.38 GB	3.43%	11.01%	20.4%	62.9%
RastNus-Base-12B-GGUF	`Q2_K` ⚠️	4.83 GB	8.20%	19.19%	13.4%	22.2%
RastNus-Small-5B-GGUF	`BF16`	9.27 GB	3.68%	10.82%	17.3%	100%
RastNus-Small-5B-GGUF	`Q8_0`	4.95 GB	3.63%	10.76%	17.3%	90.6%
RastNus-Small-5B-GGUF	`Q6_K`	3.83 GB	3.64%	10.77%	17.3%	86.3%
RastNus-Small-5B-GGUF	`Q4_K_M`	3.42 GB	3.67%	10.87%	18.2%	68.7%
RastNus-Small-5B-GGUF	`Q2_K` ⚠️	2.98 GB	5.33%	14.80%	14.0%	32.2%
RastNus-Tiny-1B-GGUF	`BF16`	2.06 GB	4.53%	12.69%	18.5%	100%
RastNus-Tiny-1B-GGUF	`Q8_0`	1.11 GB	4.56%	12.70%	18.8%	91.8%
RastNus-Tiny-1B-GGUF	`Q6_K`	0.86 GB	4.51%	12.61%	18.2%	88.4%
RastNus-Tiny-1B-GGUF	`Q4_K_M`	0.70 GB	4.58%	12.58%	18.2%	72.9%
RastNus-Tiny-1B-GGUF	`Q2_K` ⚠️	0.55 GB	6.57%	15.58%	15.5%	44.4%

Measured with llama.cpp (CUDA) on the same GPU. Same output as BF16 = share of examples where the quantized file produces exactly the same text as the lossless BF16 GGUF — the most direct measure of quantization loss. ⚠️ = not recommended (no better than leaving the text uncorrected). ¹ = one test input containing many blank lines makes the model loop on newlines until the length limit; without that single example these files score at the BF16 level (≈3.7% CER). Small differences between the vLLM and llama.cpp bf16 scores come from numerical differences between the two engines, not from the weights.

Example

Input

من دوێنی چووم بو بازار و چەند شتێکم کری بەڵام زور گران بوو

Output

من دوێنێ چووم بۆ بازاڕ و چەند شتێکم کڕی، بەڵام زۆر گران بوو.

Model Details


Developed by	PawanKrd
Model type	Causal decoder-only transformer (Gemma 4 architecture, text-only)
Base model	Google Gemma 4 E2B
Parameters	~1.03B
Language	Central Kurdish / Sorani (`ckb`), Arabic script
Task	Text correction (grammar, spelling, punctuation)
Precision	FP8 (MLP), bf16 (attention, embeddings)
Context length	Recommended working length up to ~8K tokens per request
License	Apache 2.0

Prompt Format & Control Tokens

RastNus uses a task-specific prompt format with dedicated special tokens. The behavior of the model is controlled by two switches:

Token	Effect
`<\|punct:on\|>` / `<\|punct:off\|>`	Add & normalize punctuation, or leave punctuation untouched
`<\|pure:on\|>` / `<\|pure:off\|>`	Replace non-Kurdish loanwords with pure Kurdish equivalents, or keep original vocabulary

The full sequence layout:

<bos><|task:kurdish_correction|><|punct:on|><|pure:off|><|input|>
{your Kurdish text}
<|output|>
{corrected text}<eos>

You do not need to build this manually — the repository ships with a chat template that handles it (see below). Defaults are punctuationEnabled=True and kurdishPureMode=False.

Quantization

FP8 weights (per-channel) and FP8 activations (dynamic, per-token) for the MLP layers, produced with llm-compressor (FP8_DYNAMIC, no calibration needed). Attention layers, embeddings (including the per-layer embedding tables) and lm_head are kept in bf16 — at this model size, quantizing attention caused occasional runaway generations.

Hardware: NVIDIA GPUs with FP8 support (Ada Lovelace, Hopper, Blackwell — e.g. RTX 40xx, L40S, H100, B200).

See Benchmarks for the quality and speed of every format.

Usage

Transformers

Requires pip install compressed-tensors (the quantized weights are loaded automatically).

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "PawanKrd/RastNus-Tiny-1B-FP8"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)

text = "خویندنەوەی کتیب زور گرنگە بو مروف"

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": text}],
    tokenize=False,
    add_generation_prompt=True,
    punctuationEnabled=True,   # add/normalize punctuation
    kurdishPureMode=False,     # keep loanwords as-is
)

inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Note: pass add_special_tokens=False when tokenizing — the chat template already emits <bos>.

vLLM (OpenAI-compatible server)

vllm serve PawanKrd/RastNus-Tiny-1B-FP8

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")

resp = client.chat.completions.create(
    model="PawanKrd/RastNus-Tiny-1B-FP8",
    messages=[{"role": "user", "content": "خویندنەوەی کتیب زور گرنگە بو مروف"}],
    extra_body={"chat_template_kwargs": {
        "punctuationEnabled": True,
        "kurdishPureMode": True,   # enable pure-Kurdish word replacement
    }},
    temperature=0.0,
)
print(resp.choices[0].message.content)

Recommended Generation Settings

Greedy decoding (do_sample=False / temperature=0) — correction is a deterministic task
These defaults are built in: generation_config.json sets greedy decoding, so model.generate() needs no extra sampling arguments
max_new_tokens ≈ 1.5 × input token count (the corrected output is roughly the same length as the input)
For long documents, split at sentence/paragraph boundaries (≈1,500–2,000 characters per chunk works best for this size), correct each chunk, and rejoin

Intended Use & Limitations

Intended uses: fast proofreading and standardization of Sorani text in writing assistants, document pipelines, search/indexing preprocessing, and on-device or low-cost deployments.

Limitations:

The model targets Central Kurdish (Sorani) written in Arabic script. It is not designed for Kurmanji (Latin script), Zazaki, or other Kurdish varieties.
It is a correction model, not a general chat assistant — it expects the task prompt format above and will not follow open-ended instructions.
Being a small model, it misses more errors than RastNus-Base-12B and can occasionally produce repeated text on very long inputs; chunking long documents and setting max_new_tokens as recommended avoids this.
It can over-correct rare proper nouns, dialect-specific spellings, or intentional stylistic choices. Human review is recommended for high-stakes publishing.
Pure-Kurdish mode applies vocabulary substitutions that may not match every publication's style guide.

About PawanKrd

Pawan.Krd (PKRD, LLC) builds AI and cloud infrastructure — LLMs, ASR, TTS, and related APIs — with a focus on reliability and open models. RastNus is part of that work: first-class writing tools for Kurdish, released openly.

🌐 Website: https://pawan.krd
🤗 Hugging Face: https://huggingface.co/PawanKrd

License & Attribution

This model is released under the Apache License, Version 2.0. See the LICENSE file for the full text and the NOTICE file for attribution details.

RastNus-Tiny-1B-FP8 is a quantized version of RastNus-Tiny-1B, a derivative work of Gemma 4 E2B by Google LLC / Google DeepMind, which is also licensed under Apache 2.0. In accordance with the license:

The original Apache 2.0 license text is included in this repository.
The NOTICE file carries the required attribution to Google and a prominent statement that the model weights, tokenizer, and chat template have been modified by PawanKrd (fine-tuned, pruned, distilled, vocabulary trimmed).
Gemma is a trademark of Google LLC, used here only to describe the origin of the base model. This project is not affiliated with, endorsed by, or sponsored by Google.

This model is provided "AS IS", without warranties or conditions of any kind. See Sections 7 and 8 of the Apache License for the disclaimer of warranty and limitation of liability.

Citation

@misc{rastnustiny2026,
  title   = {RastNus-Tiny-1B: A Compact Central Kurdish (Sorani) Grammar and Spelling Correction Model},
  author  = {PawanKrd},
  year    = {2026},
  url     = {https://huggingface.co/PawanKrd/RastNus-Tiny-1B},
  note    = {Pruned and distilled derivative of Google Gemma 4 E2B, Apache License 2.0}
}