Instructions to use edededdy/dilmac-en-tr-28m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use edededdy/dilmac-en-tr-28m with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="edededdy/dilmac-en-tr-28m", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("edededdy/dilmac-en-tr-28m", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
DilmaΓ§ 28M: English β Turkish
DilmaΓ§ (dil-maΓ§) is an old Turkic word for an interpreter. This is a small EnglishβTurkish translation model: 28M parameters, 112 MB in fp32 (56 MB in bf16). It scores 54.8 chrF++ on FLORES-200, runs fully offline, and is small enough for a laptop CPU.
The -28m suffix gives the size. Larger siblings, if they are made, will follow the same naming.
| Direction | English β Turkish (one way) |
| Parameters | 28.1M (18.9M excluding embeddings) |
| Architecture | Encoder-decoder Transformer, 8 encoder layers + 2 decoder layers |
| Vocabulary | 24,000-token BPE, shared by English and Turkish |
| Input length | Sentence-level, up to 128 source tokens |
| License | Apache-2.0 |
Quick start
pip install torch transformers
from transformers import AutoModel, AutoTokenizer
repo = "edededdy/dilmac-en-tr-28m"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True)
model.translate("Where is the nearest train station?", tok)
# 'En yakΔ±n tren istasyonu nerede?'
model.translate([
"Could you send me the report by Friday, please?",
"You look nice in the suit.",
], tok)
# ['Raporu Cuma\'ya kadar gΓΆnderir misiniz lΓΌtfen?', 'KostΓΌmΓΌn iΓ§inde gΓΌzel gΓΆrΓΌnΓΌyorsun.']
The model uses its own architecture, so loading it needs trust_remote_code=True. The code
it runs is modeling_dilmac.py and configuration_dilmac.py in this repository, plain PyTorch
of about 300 lines. Translation goes through model.translate(). It does not use
generate() or pipeline().
Faster inference: bf16 gives the same quality (see the results below) at half the size, and is much faster on CPU:
import torch
model = model.to(torch.bfloat16) # CPU
model = model.to("cuda", torch.bfloat16) # GPU
Decoding options
translate(texts, tokenizer, num_beams=4, no_repeat_ngram_size=4, max_length_ratio=1.5, length_penalty=1.0, batch_size=32)
| Option | Default | Meaning |
|---|---|---|
num_beams |
4 | Beam width; 1 = greedy (about 3Γ faster, about 1.5 chrF++ lower) |
no_repeat_ngram_size |
4 | Never repeat a 4-token sequence (prevents loops) |
max_length_ratio |
1.5 | Stop after 1.5 Γ source length + 5 tokens |
length_penalty |
1.0 | Beam score = log p / ((5 + len) / 6) ^ length_penalty |
batch_size |
32 | Sentences per batch (sorted by length internally) |
The defaults are the best settings we found. A beam of 5 with length penalty 0.6 was no better.
Results
sacrebleu scores on the full test sets: chrF++
(word_order=2) and BLEU (13a tokenization). Higher is better for both. For Turkish, chrF++ is
the more meaningful of the two: it gives credit for partly correct word forms, while BLEU
counts any wrong suffix as a miss.
| Test set | Sentences | Greedy chrF++ / BLEU | Beam 4 chrF++ / BLEU | Beam 4 (bf16) chrF++ / BLEU |
|---|---|---|---|---|
| FLORES-200 devtest | 1,012 | 53.29 / 24.38 | 54.83 / 26.32 | 54.75 / 26.10 |
| WMT17 newstest2017 | 3,006 | 47.86 / 18.59 | 49.37 / 20.17 | 49.36 / 20.16 |
| NTREX-128 | 1,997 | 44.05 / 15.58 | 45.48 / 17.12 | 45.60 / 17.10 |
Speed: all speeds were measured on a laptop, with EnglishβTurkish batches of 64 sentences.
| Device | Greedy | Beam 4 |
|---|---|---|
| AMD Radeon 890M integrated GPU, fp32 | ~145 sent/s | ~46 sent/s |
| AMD Radeon 890M integrated GPU, bf16 | ~70 sent/s | |
| Laptop CPU, bf16 | ~30 sent/s |
Examples
These are beam-search outputs (default settings), including some mistakes.
| English | DilmaΓ§ 28M |
|---|---|
| Where is the nearest train station? | En yakΔ±n tren istasyonu nerede? |
| Could you send me the report by Friday, please? | Raporu Cuma'ya kadar gΓΆnderir misiniz lΓΌtfen? |
| Dr. Ehud Ur, professor of medicine at Dalhousie University in Halifax, Nova Scotia and chair of the clinical and scientific division of the Canadian Diabetes Association cautioned that the research is still in its early days. | Dr. Ehud Ur, Halifax, Nova Scotia'daki Dalhousie Γniversitesi'nde tΔ±p profesΓΆrΓΌ ve Kanada Diyabet DerneΔi'nin klinik ve bilimsel bΓΆlΓΌm baΕkanΔ±, araΕtΔ±rmanΔ±n hala ilk gΓΌnlerinde olduΔuna dikkat Γ§ekti. |
| A 28-year-old chef who had recently moved to San Francisco was found dead in the stairwell of a local mall this week. | YakΔ±n zamanda San Francisco'ya taΕΔ±nan 28 yaΕΔ±ndaki bir Εef, bu hafta yerel bir alΔ±ΕveriΕ merkezinde ΓΆlΓΌ bulundu. (drops "in the stairwell") |
| I would be glad to talk to you about the new workers' brigade! Or anything else! | Yeni iΕΓ§ilerin tugayΔ± hakkΔ±nda konuΕmaktan mutluluk duyarΔ±m! (drops "Or anything else!") |
| 28-Year-Old Chef Found Dead at San Francisco Mall | San Francisco Mall AlΔ±ΕveriΕ Merkezi'de ΓlΓΌ Bulundu San Francisco Mall Mall'da ΓlΓΌ 28 YaΕlΔ± Chef Bulun (headline: garbled) |
Model details
Pre-norm Transformer encoder-decoder:
| Width / heads | d_model 384, 6 attention heads (head size 64) |
| Feed-forward | SwiGLU, hidden size 1024 |
| Layers | 8 encoder, 2 decoder (a deep encoder and a shallow decoder, for fast decoding) |
| Normalization | RMSNorm (pre-norm), plus a final norm in the encoder and in the decoder |
| Positions | Rotary embeddings (RoPE) in self-attention; none in cross-attention |
| Embeddings | One matrix shared by source, target and output projection |
| Biases | None |
Special tokens: <pad> = 0, <s> = 1 (decoder start), </s> = 2, <sep> = 3 (unused by this model).
Training
- Data: about 70M EnglishβTurkish sentence pairs from publicly available parallel corpora. Before training, the pairs were filtered by length ratio, script and language checks, sentence completeness, and template and placeholder text, then deduplicated.
- Tokens: a single pass over 2.82B tokens (source + target), 274k steps.
- Batches: about 12k tokens per batch, with source and target capped at 128 tokens.
- Optimizer: AdamW (Ξ² = 0.9, 0.98). Peak learning rate 1e-3 with 400 warmup steps, then cosine decay over tokens to 10% of the peak.
- Regularization: label smoothing 0.1, no dropout.
- Precision: bf16 autocast.
- Released weights: the average of the last 5 checkpoints, about 30 minutes of training apart.
- Hardware: one AMD Radeon 890M integrated laptop GPU (ROCm on Windows), for 58.6 hours.
- Selection: the shape was chosen in a size-matched architecture race between deep-encoder, balanced, decoder-only and wide variants.
Benchmark overlap: sentences matching FLORES-200 and NTREX-128 were removed from most of the training data. A few hundred very short, generic lines that also appear in those test sets remain (e.g. one-line greetings).
Intended use
- Getting the gist of English text in Turkish: articles, documents, messages.
- Translation on devices where a large model or a cloud API isn't an option.
- Drafts for a human translator to post-edit.
- A small, fast baseline for EnglishβTurkish MT research and distillation experiments.
Limitations
- English β Turkish only. It will not translate Turkish into English.
- Headlines and Title-Case text are a clear weak spot. Output may be garbled or repetitive (see the examples).
- Sentence level only. It has no document context, so the formal siz and informal sen, pronouns and terminology may be inconsistent across sentences. Split long texts into sentences.
- Long, complex sentences sometimes lose a clause.
- Rare names, technical terms and numbers are less reliable than everyday vocabulary.
- Inputs longer than 128 tokens (about 100 English words) are truncated.
- It is much smaller than commercial systems and large language models, and less accurate on hard text. Do not use its output unreviewed where errors matter: legal, medical, safety or publication.
- Like any model trained on web and media text, it can reproduce biases from its training data, for example gendered word choices in contexts the English leaves unspecified.
Reproducing the scores
import json, sacrebleu, torch
from transformers import AutoModel, AutoTokenizer
repo = "edededdy/dilmac-en-tr-28m"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True).to("cuda")
src, ref = ... # lists of English sources and Turkish references
hyp = model.translate(src, tok, batch_size=64)
print(sacrebleu.corpus_chrf(hyp, [ref], word_order=2).score, sacrebleu.corpus_bleu(hyp, [ref]).score)
License
Apache-2.0, for the weights and the code.
- Downloads last month
- 29
Space using edededdy/dilmac-en-tr-28m 1
Collection including edededdy/dilmac-en-tr-28m
Evaluation results
- chrF++ on FLORES-200 (devtest)self-reported54.830
- BLEU on FLORES-200 (devtest)self-reported26.320
- chrF++ on NTREX-128test set self-reported45.480
- BLEU on NTREX-128test set self-reported17.120
- chrF++ on WMT17 newstest2017test set self-reported49.370
- BLEU on WMT17 newstest2017test set self-reported20.170