Pusula 9B

Pusula 9B: a Turkish chat assistant
9B parameters · Turkish-first, English retained · tool calling · safetensors + GGUF · Apache-2.0


Overview

Pusula 9B is a Turkish general-purpose chat model. It was fine-tuned from an Apache-2.0 licensed open 9B base model. The goal is fluent, well-structured Turkish answers at the base model's level of knowledge and reasoning.

Pusula answers in natural Turkish, uses Markdown where it helps, and understands the Turkish context. On calculation and logic questions it works through the solution first and states the result at the end. Previous versions announced an answer up front and sometimes contradicted it while calculating; this version avoids that pattern.

Languages Turkish (tr), English (en)
Parameters ~9B
Context 32K tested (longer supported by the architecture)
Formats safetensors (BF16), GGUF (Q4_K_M, BF16)
Training data serhatkagans/turkish-chat-sft-v3 + English replay + tool calling
License Apache-2.0

Files

File Size Use
model-0000X-of-00004.safetensors ~18 GB Transformers / vLLM (BF16)
Turkce-9B-v6_06-Q4_K_M.gguf ~5.6 GB llama.cpp, LM Studio, Ollama; fits an 8 GB GPU
Turkce-9B-v6_06-BF16.gguf ~17.9 GB Full-precision GGUF, for making other quantizations

Quick start

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

m = "serhatkagans/Pusula-9B"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m, torch_dtype="auto", device_map="auto")

msg = [{"role": "user", "content": "Kahvaltıda ne yesem, hafif bir şey öner."}]
x = tok.apply_chat_template(msg, add_generation_prompt=True, enable_thinking=False, return_tensors="pt").to(model.device)
out = model.generate(x, max_new_tokens=512, do_sample=True, temperature=0.7, top_p=0.8, top_k=20)
print(tok.decode(out[0][x.shape[1]:], skip_special_tokens=True))

llama.cpp

llama-server -m Turkce-9B-v6_06-Q4_K_M.gguf -ngl 99 -c 32768 --jinja

Recommended sampling, with thinking off: temperature 0.7, top_p 0.8, top_k 20, presence_penalty 1.5.

Training

  • Method: QLoRA on a 4-bit base for 1 epoch, 765 steps.
    • Settings: r = 32, learning rate 7e-5, effective batch size 16, max length 4,096 tokens.
    • The LoRA was merged into the 16-bit base at 0.6 scale. A full-scale (1.0) merge gave worse chat quality in blind comparisons.
  • Data (12,234 samples):
    • ~9,970 Turkish dialogues from turkish-chat-sft-v3, across 12 categories. In this version:
      • 1,182 logic/math answers were rewritten in "solution first, result last" form.
      • Every rewritten answer was checked individually by a strict reviewer.
      • 51 samples with incorrect answers were removed.
    • 1,751 English samples, added so the model keeps its English ability.
    • 512 tool-calling samples, in Turkish and English.

Evaluation

All models were evaluated with identical settings. "Pusula v3" is the previous release.

Standard benchmarks

Benchmark Base model Pusula v3 Pusula 9B
MMLU-Pro (14 subjects × 20 questions, 5-shot; ±2.8) 63.2 63.6 63.9
Tool calling (20 tasks) 19/20 18/20 18/20

Turkish reference-answer accuracy (263 new questions, none of them in the training data)

All three models' answers to each question were given to one external judge in a single call. The judge did not know which model wrote which answer, and the order was shuffled. Scoring: correct 1, partially correct 0.5, wrong 0.

Base model Pusula v3 Pusula 9B
Overall score % 50.2 46.8 51.5
Self-contradicting answers 22 46 11
Logic 68.6 48.6 64.3
Math 90.2 85.4 93.9
Turkey (history, geography, culture) 22.6 27.4 28.3
Turkish grammar & spelling 27.4 34.9 31.1
General knowledge 67.4 57.0 66.3
Honesty (false premises, made-up entities) 40.8 35.5 38.2
  • Pusula 9B vs. v3: +4.8 points, a statistically significant difference (95% CI +1.1 to +8.6).
  • Pusula 9B vs. base model: +1.3 points, not statistically significant (95% CI −2.9 to +5.3). Accuracy is on par with the base model.

Blind chat preference (test200: 200 Turkish prompts, 13 categories)

Two external judges compared the answers. Each pair was judged in both orders.

Matchup Score
Pusula 9B vs. base model 122.0 – 58.7
Pusula 9B vs. Pusula v3 90.0 – 81.0 (the two judges disagreed)

What it is good at

The win rates below come from the blind comparison against the base model on test200; ties count as half a win. Small categories such as honesty (4 prompts) are noisy.

Category Pusula 9B preferred Verdict
Turkey-specific topics (culture, daily life, institutions) 81% ✅ Strong: natural, context-aware Turkish
Writing & editing (emails, petitions, messages) 81% ✅ Strong
Everyday questions & small talk 79% ✅ Strong
General knowledge explanations 77% ✅ Strong
Coding help 76% ✅ Strong
Health & lifestyle (general information) 76% ✅ Strong
Math (word problems, step-by-step solutions) 66% ✅ Good; 93.9% accuracy on reference questions
Education / school topics 62% 👍 Better than base
Business, career & personal finance 62% 👍 Better than base
Turkish grammar & spelling 49% ➖ Same as base
Logic puzzles 48% ➖ Same as base; slightly lower accuracy
Creative writing (stories, poems) 22% ⚠️ Weaker than base

In short:

  • Use Pusula for: everyday Turkish chat, explanations, writing and editing texts, coding help and step-by-step math.
  • Use with care for: precise facts (especially Turkish history and geography), grammar rules and creative writing.

Intended uses

  • Turkish chat assistant for everyday questions, writing, explanations, coding help and math.
  • Local or on-premise deployment through GGUF (llama.cpp, LM Studio, Ollama).
  • A base for further Turkish fine-tuning or retrieval-augmented (RAG) systems.

Out of scope: Pusula is not a reliable source of facts. Do not use it on its own for medical, legal or financial decisions.

Limitations, biases and risks

  • Factual knowledge is not improved.
    • On Turkey-specific fact questions (provinces, dates, people, geography), accuracy is only about 25–30%. Fine-tuning improved style, not knowledge.
    • The model can state wrong facts confidently. Pair it with search or RAG when facts matter.
  • Logic puzzles: slightly behind the base model.
  • No internet access: the model does not know about recent events.
  • Thinking mode: when enabled, the reasoning text is mostly in English.
  • No separate safety tuning: the model inherits the base model's safety behavior and biases.

License

Released under the Apache License 2.0. This model is a modified (fine-tuned) derivative of an Apache-2.0 licensed base model. You may use, modify and redistribute it, including commercially, provided the license text and attribution notices are retained.

Citation

@misc{pusula_9b,
  title        = {Pusula 9B: A Turkish Chat Assistant},
  author       = {Serhat Kağan Şahin},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/serhatkagans/Pusula-9B}}
}

🇹🇷 Türkçe özet

Pusula 9B, Türkçe genel sohbet için ince ayar yapılmış 9 milyar parametreli bir dil modelidir. Apache-2.0 lisanslı açık bir temel modelden türetilmiştir ve turkish-chat-sft-v3 veri setiyle eğitilmiştir.

Hesap ve mantık sorularında önce çözümü gösterir, sonucu en sonda verir. Önceki sürüme göre kendi kendiyle çelişen cevap sayısı dörtte birine indi (46'dan 11'e). MMLU-Pro'da 63,9 puan aldı. 263 yeni Türkçe soruda doğrulukta temel modelle aynı düzeyde. Kör sohbet karşılaştırmasında ise iki dış hakem temel modele açıkça tercih etti (122'ye 58,7).

Güçlü olduğu alanlar: Türkiye ile ilgili gündelik konular, yazışma ve metin düzenleme, günlük sohbet, genel kültür açıklamaları, kod yardımı, sağlık ve yaşam hakkında genel bilgi, adım adım matematik çözümü. Bu alanlarda kör karşılaştırmada temel model yerine %66-81 oranında tercih edildi.

Temel modelle aynı düzeyde olduğu alanlar: Dil bilgisi ve mantık bulmacaları.

Daha zayıf olduğu alan: Yaratıcı yazarlık (hikâye, şiir).

Sınırlar: Türkiye'ye özgü olgu sorularında doğruluk %25-30 civarında. Önemli bilgileri doğrulayın; sağlık, hukuk ve finans konularında profesyonel tavsiye yerine geçmez.

GGUF (Q4_K_M, 5,6 GB) dosyası llama.cpp, LM Studio ve Ollama ile 8 GB ekran kartında çalışır.

Lisans: Apache-2.0.

Downloads last month
-
Safetensors
Model size
9B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train serhatkagans/Pusula-9B

Evaluation results

  • accuracy on MMLU-Pro (14 subjects × 20 questions, 5-shot)
    test set self-reported
    63.900
  • judged accuracy on Turkish reference-answer set (263 questions, blind external judge)
    self-reported
    51.500
  • accuracy on Tool calling (20 tasks)
    self-reported
    90.000