Instructions to use eademir/TurkLM-scratch-0.3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use eademir/TurkLM-scratch-0.3B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="eademir/TurkLM-scratch-0.3B")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("eademir/TurkLM-scratch-0.3B") model = AutoModelForCausalLM.from_pretrained("eademir/TurkLM-scratch-0.3B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use eademir/TurkLM-scratch-0.3B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "eademir/TurkLM-scratch-0.3B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "eademir/TurkLM-scratch-0.3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/eademir/TurkLM-scratch-0.3B
- SGLang
How to use eademir/TurkLM-scratch-0.3B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "eademir/TurkLM-scratch-0.3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "eademir/TurkLM-scratch-0.3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "eademir/TurkLM-scratch-0.3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "eademir/TurkLM-scratch-0.3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use eademir/TurkLM-scratch-0.3B with Docker Model Runner:
docker model run hf.co/eademir/TurkLM-scratch-0.3B
TurkLM-scratch-0.3B
A 316M-parameter Turkish causal language model, trained fully from scratch (random initialization, no weight transfer from any existing checkpoint) on Turkish-only data, using the Qwen3 architecture and a 32,000-token Turkish BPE tokenizer trained specifically for this model.
Evaluation
| Model | Params | BPC ↓ | XCOPA-TR ↑ | Fertility ↓ | Speed (tok/s) ↑ |
|---|---|---|---|---|---|
| TurkLM-scratch-0.3B | 316M | 1.0108 | 56.0% | 1.80 | 64.9 |
| TurkLM-scratch-0.5B | 473M | 0.9956 | 54.0% | 1.80 | 53.5 |
| Qwen3-0.6B | 596M | 1.6026 | 54.2% | 2.69 | 51.4 |
| Qwen3-1.7B | 1,721M | 1.3467 | 51.0% | 2.69 | 37.3 |
| Turkcell-LLM-7b-v1 | 7,376M | 0.9528 | 56.2% | 2.17 | 15.4 |
- BPC = bits per character on 200 Turkish Wikipedia articles created after the training data was collected (page id ≥ 4,500,000
of the 2026-11-01 dump, roughly February 2026 onward; 958,229 characters). Lower is better, and unlike perplexity it is comparable across
tokenizers. The articles are shipped in
eval/data/fresh_wiki.json. (An older 200-article test set lies inside the training stream, so it is not used here.) - XCOPA-TR = 500 two-choice commonsense questions, random = 50%. All base models are close to chance, so this column does not separate them.
- Fertility = tokens per Turkish word (lower means more text per token). Speed = tokens per second generating 100 tokens, bfloat16, on one Apple-silicon Mac, all models measured in the same session; compare only the ratios, not the absolute numbers. Because TurkLM needs fewer tokens per word, its speed in Turkish words per second is higher than the token column suggests (TurkLM-0.3B 36, TurkLM-0.5B 30, Qwen3-0.6B 19, Qwen3-1.7B 14, Turkcell-7B 7).
- On this test the TurkLM models beat Qwen3-0.6B and Qwen3-1.7B on BPC. Turkcell-LLM-7b-v1, a 7B model that is 23 times larger, is better than this model, by about 6%.
- 200 articles is a small test; differences below about 0.02 BPC should not be over-read.
All numbers can be reproduced with the scripts and data in eval/.
Model details
| Parameters | 316M (315.9M measured) |
| Architecture | Qwen3-style decoder: GQA, SwiGLU, RoPE, RMSNorm, tied embeddings |
| Hidden size | 1,024 |
| Layers | 24 |
| Attention heads | 16 (query) / 8 (KV, grouped-query attention) |
| FFN intermediate size | 2,816 |
| Head dimension | 64 |
| RoPE base (θ) | 1,000,000 |
| Max position embeddings | 8,192 (trained only on 1,024-token sequences) |
| Vocabulary | 32,000 (Turkish-only BPE, ByteLevel, NFC-normalized) |
| Precision | bfloat16 |
Training
100,000 steps at a constant effective batch of 64 sequences × 1,024 tokens (65,536 tokens/step), about 6.5 billion tokens (a logged count), on Turkish Wikipedia (30%) and the Turkish subset of CulturaX (70%).
Tokenizer
ByteLevel BPE, 32,000 tokens, trained from scratch on 500,000 documents (70% Turkish Wikipedia, 30% a Turkish synthetic corpus) with NFC normalization. The vocabulary comes only from Turkish text, which gives much lower fertility on Turkish than general multilingual tokenizers.
Intended use
A research base model for Turkish text modelling, tokenization studies, and as a starting point for fine-tuning. It is not instruction-tuned, has had no safety training, and can produce incorrect or biased text. It was trained on far less data than multilingual models of the same size, so its world knowledge is limited: do not rely on it for factual question answering without retrieval or fine-tuning.
Known limitations
- Base model only: no instruction tuning or RLHF.
- Single training run, no seed variation.
- Evaluated mainly on Wikipedia-style text; behavior on conversational or domain-specific Turkish is untested.
Training data and licenses
Turkish Wikipedia (wikimedia/wikipedia, 20231101.tr; CC BY-SA 3.0 and GFDL) and the Turkish subset of CulturaX (uonlp/CulturaX; its terms follow
those of mC4 and OSCAR), mixed roughly 30% / 70% by tokens. The weights are released under Apache-2.0; if data licenses matter for your use, check the terms
of these datasets yourself.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("eademir/TurkLM-scratch-0.3B")
model = AutoModelForCausalLM.from_pretrained("eademir/TurkLM-scratch-0.3B", dtype="bfloat16")
inputs = tok("Türkiye'nin başkenti", return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=50, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True))
(On Apple GPUs use attn_implementation="eager": the default attention kernel on macOS 27 / torch 2.12.1 was not causal in our tests.)
- Downloads last month
- 3