Jugurtha-2-9B

An 8.95B-parameter generative foundation model for Kabyle (Taqbaylit, kab, Latin script) with native chain-of-thought reasoning in Taqbaylit. It represents the second-generation scale of the Jugurtha family, advancing state-of-the-art Amazigh language capabilities across conversational assistance, translation, and structured multi-step reasoning.

It achieves 80.81% on SIB-200 topic classification (becoming the first Amazigh model to cross the 80% mark), 49.45 chrF++ / 27.86 BLEU on FLORES+ kab→eng, 30.80 chrF++ / 7.62 BLEU on eng→kab, and natively writes canonical Berber Latin orthography with an orthography gap of 0.00.

Highlights

• Dual-mode reasoning: standard conversational mode and transparent step-by-step reasoning in Kabyle (<think>...</think>) — switchable per request with enable_thinking
• Generational scaling: 8.95B-parameter hybrid architecture delivering unprecedented depth, syntactic cohesion, and cross-lingual translation quality
• Native Kabyle generation: lifts kab→eng to 49.45 chrF++ / 27.86 BLEU and eng→kab to 30.80 chrF++ / 7.62 BLEU, with the orthography gap closing completely to 0.00
• Multilingual retention: held-out English BPC of 0.993 (perplexity 17.99) and French BPC of 1.258 (perplexity 24.76) — zero catastrophic forgetting of global knowledge
• Baked sovereign system template: sovereign identity, linguistic purity, and anti-hallucination guardrails are built directly into the tokenizer chat template
• No trust_remote_code: standard transformers loading, BF16, device_map="auto"

Results

Evaluated on an NVIDIA H100 NVL MIG slice (lm-eval 0.4.13). Translation scored 5-shot against the orthographically corrected FLORES+ kab_Latn reference (agbalu/KabBench).

Kabyle & multilingual

Benchmark / Task	Metric	Score
SIB-200 (kab_Latn, 7-shot)	Accuracy (chance 14.29%)	80.81%
FLORES+ kab → eng	5-shot chrF++ / BLEU	49.45 / 27.86
FLORES+ eng → kab	5-shot chrF++ / BLEU	30.80 / 7.62
FLORES+ Orthography Gap	chrF++ (normalised − raw)	0.00
Held-out Kabyle	Bits per character (2,000 docs)	1.587
Held-out English	Bits per character (832 docs)	0.993
Held-out French	Bits per character (2,691 docs)	1.258

Three things worth reading carefully.

Generational scale brings massive translation and comprehension gains. Jugurtha-2-9B lifts kab→eng from Jugurtha-4B's 46.17 chrF++ / 23.79 BLEU to 49.45 chrF++ / 27.86 BLEU (+4.07 BLEU), and topic classification on SIB-200 from 73.74% to 80.81% (+7.07%). The additional parameter capacity allows nuanced syntactic handling of complex relative clauses and particle dependencies that smaller models compress.

The orthography gap remains absolute zero. Scored raw and then passed through the versioned AƔBALU normaliser, Jugurtha-2-9B's results are identical (49.4467 vs 49.4467 and 30.8004 vs 30.8004). The model natively produces standard Kabyle Latin — ɣ ɛ ḥ ḍ ṣ ṭ ẓ ṛ č ǧ ţ — without phonemic degradation or glyph corruption.

Multilingual knowledge is preserved intact. Held-out English BPC is 0.993 (perplexity 17.99) and French BPC is 1.258 (perplexity 24.76). Continued pretraining and post-training on Kabyle did not degrade the underlying model's fluency across major world languages.

Intended use

• Primary: Kabyle conversational assistance, reading comprehension, question answering, summarisation, cultural explanations, and bidirectional Kabyle ↔ English / French translation.
• Thinking mode: Set enable_thinking=True for transparent step-by-step reasoning in Kabyle inside <think>...</think> blocks. Set enable_thinking=False (default) for direct, fast conversational generation.
• Not suitable for: Safety-critical medical diagnosis, pharmacological guidance, or binding legal counsel.

Usage

transformers and torch in BF16. The architecture is standard causal LM, so no trust_remote_code is required.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL = "agbalu/Jugurtha-2-9B"

tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(
    MODEL,
    torch_dtype=torch.bfloat16,
    device_map="auto",
).eval()

messages = [
    {"role": "user", "content": "Segzi-yi-d s wegzul anta i d tamurt n Yiqbayliyen d wamek i d-tezga."},
]

# Standard mode (thinking OFF — default)
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=512, do_sample=False)

reply = tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(reply)

For step-by-step Kabyle chain-of-thought, set enable_thinking=True and sample with temperature=0.6, top_p=0.95. The reasoning trace appears between <think> and </think>.

vLLM

vllm serve agbalu/Jugurtha-2-9B --max-model-len 32768 --dtype bfloat16

Architecture

Hybrid causal language model with tied word embeddings and 32,768-token native context.

Parameters	8,953,803,264 (text tower) / 9,409,813,744 (total)
Layers	Hybrid recurrent DeltaNet linear-attention + full-attention
Context Window	32,768 tokens
Vocabulary	248,320 tokens (tied input/output embeddings)
Format	BF16, 775 tensors, 4 safetensor shards

Training

Data

Jugurtha-2-9B was trained on three streams:

1. Kabyle pretraining corpus: Cleaned Kabyle web and document prose, filtered with Dihya-5M and deduplicated, human-authored parallel bitext from the AƔBALU corpus, and curated multilingual replay to preserve general-purpose capability.
2. Curated instruction set: High-density Kabyle dialogues covering native conversation, translation, question answering, and reasoning — including verified Kabyle chain-of-thought traces for think-mode instances.
3. Decontamination: Filtered against FLORES+ kab_Latn and SIB-200 with an n-gram and sentence boundary checker. Measured overlap: 0 sentences, verified with positive controls.

Recipe

Hardware	Single NVIDIA H100 NVL MIG (3g.47gb)
Stage 1 — CPT	Full-parameter continued pretraining on the 9B text tower
Stage 2 — Merge	Task-vector arithmetic into a 9B instruction-tuned backbone ($\alpha = 0.85$)
Stage 3 — SFT	Supervised fine-tuning on curated Kabyle dialogues with thinking traces

Limitations

• Informal keyboards & Arabizi: Trained on canonical Kabyle Latin (ɣ ɛ ḥ ḍ ṣ ṭ ẓ ṛ č ǧ ţ). Noisy Arabizi (digits 3, 7, 9) or unstandardised spelling inflates token fertility. Preprocess informal input with Boulifa-48M before prompting.
• Reasoning vs. standard mode: Think mode aligns the reasoning process to Kabyle, which trades some raw latency. Use standard mode when latency matters.
• Corpus factual bounds: Knowledge in low-resource cultural domains reflects the verified corpus; unverified claims from public web text are filtered but may surface in niche topics.
• Medical & legal: Not certified for clinical diagnosis, drug prescribing, or legal advice.

Files

File	Contents
model.safetensors-00001-of-00004.safetensors	Shard 1 of 4 (BF16 weights)
model.safetensors-00002-of-00004.safetensors	Shard 2 of 4 (BF16 weights)
model.safetensors-00003-of-00004.safetensors	Shard 3 of 4 (BF16 weights)
model.safetensors-00004-of-00004.safetensors	Shard 4 of 4 (BF16 weights, 775 tensors total)
config.json	Architecture configuration
generation_config.json	Default sampling and EOS generation parameters
chat_template.jinja	Native dual-mode chat template with baked sovereign prompt
tokenizer.json, tokenizer_config.json	Tokenizer (248,320 vocabulary)

The name

Jugurtha (c. 160–104 BCE) was the grandson of King Masinissa of Numidia. When Rome sought to partition and subjugate Numidia after Masinissa's death, Jugurtha waged a seven-year war to preserve Numidian sovereignty — and was never taken in battle.

Within the AƔBALU model family, Masinissa-31M and Jugurtha-2-9B stand at the two ends of the stack: the grandfather who unified disparate tribes into one nation, and the grandson who refused to surrender its independence.

The naming is homage. It implies no endorsement, and neither Jugurtha nor his descendants are affiliated with this work.

Citation

@software{agbalu_jugurtha_2026,
  title  = {Jugurtha-2-9B: A Sovereign Kabyle Generative Foundation Model},
  author = {Ainouche, Abderahmane and {AƔBALU Project}},
  year   = {2026},
  url    = {https://huggingface.co/agbalu/Jugurtha-2-9B},
}

Licence

Apache-2.0 for the weights and configurations. That grant does not relicense the text the weights were trained on, and a third of that text has no licence anyone could resolve. The composition is published so you can make your own judgement:

redistribution	share
unclear	34.9%
permissive	32.0%
share-alike	31.0%
non-commercial	2.0%

unclear is not a permissive category — it is the absence of a resolvable licence. A permissive-only rebuild is possible from the 32.0% permissive sentences. If you need one, open an issue.