Instructions to use eademir/TurkLM-scratch-0.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use eademir/TurkLM-scratch-0.5B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="eademir/TurkLM-scratch-0.5B")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("eademir/TurkLM-scratch-0.5B") model = AutoModelForCausalLM.from_pretrained("eademir/TurkLM-scratch-0.5B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use eademir/TurkLM-scratch-0.5B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "eademir/TurkLM-scratch-0.5B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "eademir/TurkLM-scratch-0.5B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/eademir/TurkLM-scratch-0.5B
- SGLang
How to use eademir/TurkLM-scratch-0.5B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "eademir/TurkLM-scratch-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "eademir/TurkLM-scratch-0.5B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "eademir/TurkLM-scratch-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "eademir/TurkLM-scratch-0.5B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use eademir/TurkLM-scratch-0.5B with Docker Model Runner:
docker model run hf.co/eademir/TurkLM-scratch-0.5B
TurkLM-scratch-0.5B
A 473M-parameter Turkish causal language model, trained fully from scratch (random initialization, no weight transfer from any existing checkpoint) on
Turkish-only data, using the Qwen3 architecture and the same 32,000-token Turkish BPE tokenizer as TurkLM-scratch-0.3B.
Evaluation
| Model | Params | BPC ↓ | XCOPA-TR ↑ | Fertility ↓ | Speed (tok/s) ↑ |
|---|---|---|---|---|---|
| TurkLM-scratch-0.3B | 316M | 1.0108 | 56.0% | 1.80 | 64.9 |
| TurkLM-scratch-0.5B | 473M | 0.9956 | 54.0% | 1.80 | 53.5 |
| Qwen3-0.6B | 596M | 1.6026 | 54.2% | 2.69 | 51.4 |
| Qwen3-1.7B | 1,721M | 1.3467 | 51.0% | 2.69 | 37.3 |
| Turkcell-LLM-7b-v1 | 7,376M | 0.9528 | 56.2% | 2.17 | 15.4 |
- BPC = bits per character on 200 Turkish Wikipedia articles created after the training data was collected (page id ≥ 4,500,000
of the 2026-11-01 dump, roughly February 2026 onward; 958,229 characters). Lower is better, and unlike perplexity it is comparable across
tokenizers. The articles are shipped in
eval/data/fresh_wiki.json. (An older 200-article test set lies inside the training stream, so it is not used here.) - XCOPA-TR = 500 two-choice commonsense questions, random = 50%. All base models are close to chance, so this column does not separate them.
- Fertility = tokens per Turkish word (lower means more text per token). Speed = tokens per second generating 100 tokens, bfloat16, on one Apple-silicon Mac, all models measured in the same session; compare only the ratios, not the absolute numbers. Because TurkLM needs fewer tokens per word, its speed in Turkish words per second is higher than the token column suggests (TurkLM-0.3B 36, TurkLM-0.5B 30, Qwen3-0.6B 19, Qwen3-1.7B 14, Turkcell-7B 7).
- On this test the TurkLM models beat Qwen3-0.6B and Qwen3-1.7B on BPC. Turkcell-LLM-7b-v1, a 7B model that is 15 times larger, is better, by 4% over the 0.5B model.
- 200 articles is a small test; differences below about 0.02 BPC should not be over-read.
All numbers can be reproduced with the scripts and data in eval/.
Model details
| Parameters | 473M (473.2M measured) |
| Architecture | Qwen3-style decoder: GQA, SwiGLU, RoPE, RMSNorm, tied embeddings |
| Hidden size | 1,024 |
| Layers | 28 |
| Attention heads | 16 (query) / 8 (KV, grouped-query attention) |
| FFN intermediate size | 3,072 |
| Head dimension | 128 |
| RoPE base (θ) | 10,000 |
| Max position embeddings | 40,960 (trained only on 1,024-token sequences) |
| Vocabulary | 32,000 (Turkish-only BPE, ByteLevel, NFC-normalized) |
| Precision | bfloat16 |
Note on the RoPE base. This model's RoPE base is 10,000 (the 0.3B model uses 1,000,000). The training script for this model builds its config without
setting rope_theta, so the library's default applied. We checked which value the trained weights expect: on 60 fresh articles the model gets 1.034 bits per
character with 10,000 and 1.277 with 1,000,000. Keep rope_theta as saved in config.json; changing it makes the model clearly worse. Because of this, the two TurkLM models
are not a perfectly controlled size comparison.
Training
142,900 optimizer steps in several phases (batch size changed during training): about the first 40,800 steps at 32,768 tokens/step, then 131,072 tokens/step until step 140,000, and 196,608 tokens/step for the last 2,900 steps, about 14.9 billion tokens in total. The token count is the step count times each phase's batch size, reconstructed from the configuration changes, not a logged count (uncertainty about ±1B). Data: Turkish Wikipedia (30%) and Turkish CulturaX (70%); the interleaving settings differed slightly between phases. This is the final checkpoint; earlier checkpoints are not released (an earlier 100,000-step checkpoint scored 0.9931 on the same BPC test, i.e. the same within noise).
Tokenizer
ByteLevel BPE, 32,000 tokens, trained from scratch on 500,000 Turkish documents with NFC normalization.
Intended use
A research base model for Turkish text modelling, tokenization studies, and as a starting point for fine-tuning. It is not instruction-tuned, has had no safety training, and can produce incorrect or biased text. It was trained on far less data than multilingual models of the same size, so its world knowledge is limited: do not rely on it for factual question answering without retrieval or fine-tuning.
Known limitations
- Base model only: no instruction tuning or RLHF.
- Single training run, no seed variation.
- Evaluated mainly on Wikipedia-style text; behavior on conversational or domain-specific Turkish is untested.
Training data and licenses
Turkish Wikipedia (wikimedia/wikipedia, 20231101.tr; CC BY-SA 3.0 and GFDL) and the Turkish subset of CulturaX (uonlp/CulturaX; its terms follow
those of mC4 and OSCAR), mixed roughly 30% / 70% by tokens. The weights are released under Apache-2.0; if data licenses matter for your use, check the terms
of these datasets yourself.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("eademir/TurkLM-scratch-0.5B")
model = AutoModelForCausalLM.from_pretrained("eademir/TurkLM-scratch-0.5B", dtype="bfloat16")
inputs = tok("Türkiye'nin başkenti", return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=50, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True))
(On Apple GPUs use attn_implementation="eager": the default attention kernel on macOS 27 / torch 2.12.1 was not causal in our tests.)
- Downloads last month
- 3