Kayra-1 — experimental Turkish language model

Kayra-1 is a small (~100M parameter) experimental Turkish instruction-tuned language model. It is a research prototype, not a production-ready assistant. The model has a custom architecture and ships its own tokenizer and modeling code.

Model details

Item	Value
Architecture	Kayra (custom KayraForCausalLM)
Parameters	Approximately 100M (reported)
Layers / hidden size	10 / 640 (from config.json)
Vocabulary	32,000
Maximum positions	512
Language	Turkish
Training hardware/time	T4 GPU / approximately 11 hours (original project notes)

Quick start

Install PyTorch and Transformers, then load both the model and tokenizer from this repository:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sixfingerdev/kayra-1"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
)
model.eval()

prompt = "### Soru: Türkiye'nin başkenti neresidir?\n\n### Cevap:"
inputs = tokenizer(prompt, return_tensors="pt")
inputs.pop("token_type_ids", None)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=80,
        do_sample=False,
        repetition_penalty=1.2,
    )

new_tokens = output_ids[0, inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

Security note: trust_remote_code=True executes custom Python code from this repository. Review configuration_kayra.py and modeling_kayra.py before loading an untrusted revision.

Evaluation status

The earlier project card reported a validation loss of 2.14 and perplexity of approximately 8.5, compared with 3.75 and 42.5 for an earlier checkpoint. These are author-reported training observations, not a published independent benchmark; the evaluation split and exact reproduction procedure are not provided.

Previous percentage-based “factuality” claims have been removed because a reproducible test set and scoring method were not documented.

Known limitations

• The 512-token position limit restricts longer conversations.
• The tokenizer may split Turkish words awkwardly.
• Generated answers may be repetitive, incoherent, or factually wrong.
• Small-scale research training does not demonstrate broad reasoning capability.
• No production-level safety or reliability evaluation has been published.

Prompt format

### Soru: {soru}

### Cevap:

Training data and license

The repository metadata lists Wikipedia, mC4, and SixFinger Turkish QA data as sources. The card's mit value describes this repository's declared license; upstream datasets may carry their own licenses and attribution requirements. Verify them before redistribution or commercial use.

This project is provided for research and experimentation.