SabiYarn 3B tokenizer

Byte-level BPE tokenizer with a 131,072-token vocabulary (2^17) for English, French, Arabic and African languages. This repository holds the tokenizer only; no SabiYarn 3B model weights exist yet.

It is the successor of the 65,536-token tokenizer of SabiYarn 0.5B, trained on a language-balanced sample of the project's pretraining corpus.

Properties

  • Vocabulary 131,072 = 180 special tokens + 256 byte symbols + 130,636 merges. Because ids go above 65,535, tokenized corpora need uint32 (4 bytes per token), not uint16.
  • Special tokens are unchanged. The 180 special tokens (language, task, chat-role, reasoning, tool-call and audio tokens, and the quoted JSON keys) keep exactly the ids 0-179 they have in SabiYarn 0.5B, with the same flags, so data, chat template and tool tokens written for the 0.5B tokenizer carry over. The chat template (chat_template.jinja) is identical.
  • NFC normalizer. Decomposed (NFD) and composed (NFC) spellings of a word get identical ids. Text that is already NFC round-trips exactly (decode(encode(text)) == text); other input decodes to its NFC form.
  • No OOV. All 256 byte symbols are in the vocabulary.
  • Checked on 20,910 evaluation texts (73 languages): NFC text round-tripped exactly in 20,910 of 20,910, and NFD versions decoded to the NFC string in 20,910 of 20,910.

How it was trained

  • Corpus. 4.91 GB of text (14.35 million documents, 71 languages, 1.57 billion tokens under the 0.5B tokenizer) sampled at random from the project's deduplicated, language-verified pretraining corpus.
  • Balance. English 8%, French 4% and Arabic 3% of the budget; the remaining 85% allocated UniMax-style with one epoch per language (languages with little data taken whole, the larger ones capped at roughly 25-30 million tokens each, about 55-115 MB of text).
  • Decontamination. Documents containing a FLORES+ devtest sentence, or equal to a FineWeb-2 evaluation document used below, were removed from the training sample (50,649 documents).
  • Training. Hugging Face tokenizers BpeTrainer, min_frequency 2, NFC normalizer, the same pre-tokenizer as SabiYarn 0.5B (whitespace and punctuation split followed by byte-level BPE), the 180 special tokens pinned to ids 0-179. Training took 17 minutes.
  • Effect on the pretraining mix. On fresh held-out text sampled from the same corpus, weighted by the language mix of the 40-billion-token training bin, text needs 14.8% fewer tokens than with the 0.5B tokenizer (English 4% fewer, French 30%, Zulu 17%, Wolof 19%, Kikuyu 38%, Kabiye 45%; Amharic unchanged).

Tokenizer comparison across supported languages

Tokens per word for this tokenizer against other tokenizers on the same text, lower is better. Measured 2026-10-20.

How it was measured

  • Metric. Tokens per whitespace-separated word, every text NFC-normalised first and no special tokens added. Premium vs English is a language's token count divided by the English count for the same parallel sentences (FLORES+ only). Unknown tokens is the share of output tokens equal to the tokenizer's UNK id; tokenizers with a high share lose characters, which lowers their counts.
  • Languages. The 86 language tags in the special tokens plus Arabic. 73 had accessible text and are evaluated; no usable text was found for ijo, ful, ibb, anw, orx, igl, snq, nup, kln, lch, son, kon, vai, mey. Core languages (n=25) are the languages named on the SabiYarn 0.5B card; Other tags are the rest.
  • Text. F = FLORES+ devtest (1,012 parallel sentences per language; 51 languages including English and Arabic). W = FineWeb-2 web text for languages FLORES+ lacks (22 languages; 25 to 600 documents of up to 2,000 characters each).
  • Reused values. The values for every tokenizer except this one come from the SabiYarn 0.5B model card measurement of 2026-10-17 and were not re-measured. As a control, re-running the 0.5B tokenizer on the regenerated texts reproduced its card values to within 0.005 on all 73 languages. The "Best on # langs" and "Within 10% of best" columns were recomputed with this tokenizer included.
  • Caveats. Whitespace words are not comparable across languages, so compare tokenizers within a row. The W texts come from the same family of sources as part of the training corpus, so this tokenizer's advantage there may be somewhat inflated; on the held-out F languages alone the mean is 1.70 (this tokenizer), 1.73 (Serengeti), 1.82 (NLLB-200), 2.07 (SabiYarn 0.5B), 2.37 (Gemma 3).

Summary (mean tokens per word)

Tokenizer Vocab Core langs (n=25) Other tags (n=48) All (n=73) Premium vs English (FLORES+, n=50) Best on # langs Within 10% of best Unknown tokens English
SabiYarn 3B (131k) 131,072 1.61 1.76 1.71 1.39 42 61 0.0% 1.27
Serengeti 250,004 1.63 1.79 1.74 1.41 22 56 2.6% 1.28
SabiYarn 0.5B 65,536 1.78 2.46 2.23 1.66 0 8 0.0% 1.31
AfriTeVa v2 150,100 1.83 2.17 2.05 1.63 2 20 5.3% 1.31
NLLB-200 256,204 1.83 1.89 1.87 1.35 2 33 0.8% 1.40
AfriBERTa 70,005 2.22 2.52 2.42 1.54 1 6 0.0% 1.64
AfroXLMR 250,002 2.29 2.33 2.31 1.73 0 2 1.5% 1.40
Gemma 3 262,145 2.37 2.42 2.40 1.99 0 3 0.0% 1.24
Gemma 4 262,144 2.37 2.42 2.40 1.99 0 3 0.0% 1.24
Llama 4 201,135 2.44 2.56 2.52 2.06 0 3 0.0% 1.24
Morena 1.5B 65,536 2.53 2.60 2.57 1.84 2 8 0.0% 1.41
BLOOM 250,680 2.55 2.42 2.47 1.99 1 5 0.0% 1.25
GPT-4o / GPT-5 (o200k) 200,019 2.61 2.49 2.53 2.07 2 4 0.0% 1.23
gpt-oss (o200k_harmony) 201,088 2.61 2.49 2.53 2.07 2 4 0.0% 1.23
Qwen3 151,669 2.85 2.64 2.71 2.23 0 1 0.0% 1.26
Aya Expanse 255,029 2.96 2.64 2.75 2.27 1 4 0.0% 1.23
DeepSeek-V3 128,815 3.06 2.78 2.87 2.35 1 2 0.0% 1.23
InkubaLM 0.4B* 61,788 3.18 2.58 2.79 2.04 0 0 0.0% 1.40
Mistral Nemo 131,072 3.25 2.85 2.99 2.38 0 1 0.0% 1.27
Llama 3.2 128,256 3.28 2.89 3.02 2.48 0 2 0.0% 1.24
GPT-4 (cl100k) 100,277 3.41 2.95 3.11 2.55 0 1 0.0% 1.24
Llama 2 32,000 3.70 3.00 3.24 2.31 0 0 0.0% 1.42

Tokenizers and sources

Tokenizer Vocab Loaded from
SabiYarn 3B (131k) 131,072 BeardedMonster/sabiyarn-3B
SabiYarn 0.5B 65,536 AletheiaAING/sabiyarn-0.5B
Morena 1.5B 65,536 vamboai/morena-1.5b-instruct
InkubaLM 0.4B* 61,788 OxxoCodes/InkubaLM-Instruct-test (copy of lelapa/InkubaLM-0.4B tokenizer)
Gemma 3 262,145 unsloth/gemma-3-1b-it
Gemma 4 262,144 google/gemma-4-E4B-it
GPT-4 (cl100k) 100,277 tiktoken cl100k_base
GPT-4o / GPT-5 (o200k) 200,019 tiktoken o200k_base
gpt-oss (o200k_harmony) 201,088 tiktoken o200k_harmony
Llama 2 32,000 NousResearch/Llama-2-7b-hf
Llama 3.2 128,256 unsloth/Llama-3.2-1B
Llama 4 201,135 unsloth/Llama-4-Scout-17B-16E-Instruct (mirror)
Qwen3 151,669 Qwen/Qwen3-0.6B
AfriBERTa 70,005 castorini/afriberta_base
AfroXLMR 250,002 Davlan/afro-xlmr-large
Serengeti 250,004 UBC-NLP/serengeti
NLLB-200 256,204 facebook/nllb-200-distilled-600M
AfriTeVa v2 150,100 castorini/afriteva_v2_base
BLOOM 250,680 bigscience/bloom-560m
Aya Expanse 255,029 CohereLabs/aya-expanse-8b
Mistral Nemo 131,072 unsloth/Mistral-Nemo-Instruct-2407
DeepSeek-V3 128,815 deepseek-ai/DeepSeek-V3

* InkubaLM's official repository is gated, so its tokenizer was loaded from a public fine-tune copy (OxxoCodes/InkubaLM-Instruct-test); its vocabulary size (61,788) matches the published figure. Llama 4 comes from a mirror because the official repository is gated. GPT-5 uses o200k_base through tiktoken (encoding_for_model("gpt-5")), and gpt-oss uses o200k_harmony. Gemma 3 and Gemma 4 give identical counts in this test.

Reading the results. On the 25 core languages the 131k tokenizer averages 1.61 tokens per word against 1.78 for the 0.5B tokenizer, and across all 73 languages 1.71 against 2.23. It also edges the specialist African tokenizer Serengeti (250k vocabulary) on all three means (1.61, 1.76 and 1.71 against 1.63, 1.79 and 1.74) and has the lowest count on 42 of 73 languages. It is lower than the 0.5B tokenizer on 72 of 73 languages. The biggest changes are French (2.08 to 1.42, now level with GPT-4o and Gemma 3), the Southern Bantu languages (Zulu 2.48 to 2.09, Xhosa 2.46 to 2.09, Shona 2.27 to 1.98), Kikuyu (2.93 to 1.75), Kabiye (3.28 to 1.83) and Tifinagh (tzm, 12.2 to 2.5). Known gaps: Amharic is worse than the 0.5B tokenizer on FLORES+ (1.91 against 1.78) and Arabic is unchanged; Serengeti and NLLB-200 remain lower on Swahili (1.50 against 1.26), Afrikaans, Somali, Malagasy, Tifinagh and Amharic.

Usage

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("BeardedMonster/sabiyarn-3B")
tok.apply_chat_template(
    [{"role": "user", "content": "Bawo ni?"}], tokenize=False
)

Per-language tokenizer comparison

Tokens per word for each tokenizer on the same text, lower is better; the best value in each row is in bold. Text is the source: F = FLORES+ devtest (1,012 parallel sentences), W = FineWeb-2 web text. Core languages (those named at the top of this card) come first, then the remaining language tags, each alphabetically. Method, caveats and sources are in the sections above.

Language Code Text SabiYarn 3B (131k) SabiYarn 0.5B Morena 1.5B InkubaLM 0.4B* Gemma 3 Gemma 4 GPT-4 (cl100k) GPT-4o / GPT-5 (o200k) gpt-oss (o200k_harmony) Llama 2 Llama 3.2 Llama 4 Qwen3 AfriBERTa AfroXLMR Serengeti NLLB-200 AfriTeVa v2 BLOOM Aya Expanse Mistral Nemo DeepSeek-V3
Afrikaans afr F 1.56 1.70 1.57 2.02 1.79 1.79 1.99 1.66 1.66 2.09 1.98 1.75 2.01 2.45 1.59 1.35 1.61 1.40 2.00 1.75 1.83 1.91
Amharic amh F 1.91 1.78 7.49 12.98 3.03 3.03 11.91 8.96 8.96 13.03 11.89 3.73 6.45 1.69 2.33 1.58 2.32 1.75 7.90 9.83 11.93 9.32
Arabic ara F 1.89 1.94 4.57 4.53 2.03 2.03 4.19 1.89 1.89 5.41 2.29 2.19 2.29 3.07 1.83 1.59 2.17 2.37 1.60 2.03 1.76 2.23
Chichewa nya F 1.67 1.90 2.35 2.54 2.57 2.57 2.88 2.21 2.21 3.18 2.84 2.52 2.90 2.69 2.48 1.90 1.79 1.68 2.47 2.67 2.81 2.85
Efik efi W 1.46 1.80 2.44 2.42 2.54 2.54 2.97 2.47 2.47 3.21 2.86 2.61 2.92 2.44 2.75 1.77 2.19 2.10 2.41 2.72 2.96 2.84
English eng F 1.27 1.31 1.41 1.40 1.24 1.24 1.24 1.23 1.23 1.42 1.24 1.24 1.26 1.64 1.40 1.28 1.40 1.31 1.25 1.23 1.27 1.23
Ewe ewe F 1.48 1.59 2.64 2.57 2.37 2.37 2.98 2.52 2.52 2.83 2.86 2.49 2.59 2.57 2.46 1.64 1.54 2.40 2.31 2.59 2.91 2.69
Fon fon F 1.44 1.63 2.70 2.58 2.28 2.28 3.19 2.48 2.48 2.82 3.14 2.48 2.58 2.54 2.51 1.53 1.59 2.40 1.89 2.54 3.10 2.72
Hausa hau F 1.32 1.35 1.39 1.57 1.86 1.86 2.15 1.66 1.66 2.33 2.13 1.86 2.13 1.37 1.69 1.28 1.43 1.31 1.92 1.98 2.10 2.09
Igbo ibo F 1.41 1.43 1.57 2.48 2.36 2.36 2.69 1.79 1.79 2.93 2.62 2.22 2.64 1.57 2.64 1.53 1.62 1.56 1.89 2.60 2.67 2.63
Kinyarwanda kin F 1.70 1.95 1.86 2.60 2.65 2.65 2.88 2.29 2.29 3.16 2.86 2.67 2.90 1.95 2.61 1.82 1.97 1.86 2.15 2.72 2.81 2.86
Nigerian Pidgin pcm W 1.40 1.45 1.51 1.56 1.44 1.44 1.52 1.42 1.42 1.69 1.50 1.45 1.53 1.58 1.53 1.38 1.45 1.51 1.45 1.46 1.52 1.51
Oromo orm F 1.97 2.16 3.11 3.11 2.95 2.95 3.22 2.58 2.58 3.53 3.20 2.98 3.24 1.89 2.79 1.92 2.22 2.20 3.06 3.01 3.15 3.21
Setswana tsn F 1.39 1.56 1.45 1.89 1.84 1.84 2.00 1.66 1.66 2.12 1.99 1.83 2.01 2.03 1.82 1.38 1.43 1.47 1.78 1.90 1.96 1.98
Shona sna F 1.98 2.27 1.94 3.03 2.93 2.93 3.34 2.63 2.63 3.59 3.25 2.92 3.36 3.22 2.88 2.17 2.15 1.95 2.85 3.04 3.18 3.25
Somali som F 1.66 1.82 2.48 2.45 2.25 2.25 2.57 1.99 1.99 2.87 2.55 2.26 2.59 1.54 1.84 1.45 1.83 1.42 2.40 2.41 2.57 2.58
Southern Sotho sot F 1.40 1.55 1.69 1.95 1.89 1.89 2.09 1.71 1.71 2.24 2.07 1.88 2.10 2.13 1.89 1.43 1.47 1.45 1.86 1.98 2.06 2.06
Swahili swh F 1.50 1.71 1.54 1.69 2.09 2.09 2.49 1.88 1.88 2.71 2.46 2.12 2.51 1.49 1.67 1.26 1.62 1.40 1.60 2.30 2.37 2.46
Tigrinya ti F 1.92 1.97 6.91 11.94 3.46 3.46 10.91 8.27 8.27 11.98 10.90 4.19 5.92 1.91 3.10 1.86 2.27 2.11 7.25 9.01 10.92 8.54
Twi (Akuapem) twi F 1.39 1.51 2.20 2.16 2.02 2.02 2.61 1.91 1.91 2.40 2.60 2.14 2.27 2.31 2.21 1.51 1.47 2.09 1.90 2.20 2.55 2.25
Urhobo urh W 1.36 1.75 2.17 2.12 2.40 2.40 2.88 2.23 2.23 3.39 2.53 2.40 2.58 2.35 2.53 1.63 2.04 2.09 2.22 2.52 2.61 2.87
Wolof wol F 1.47 1.80 2.03 1.99 1.92 1.92 2.10 1.86 1.86 2.25 2.06 1.91 2.10 2.11 1.96 1.61 1.61 1.92 1.85 1.95 2.07 2.03
Xhosa xho F 2.09 2.46 2.17 2.90 3.30 3.30 3.60 2.84 2.84 3.93 3.57 3.22 3.62 3.55 2.94 2.18 2.37 2.18 2.93 3.37 3.54 3.57
Yoruba yor F 1.57 1.63 1.87 1.95 2.56 2.56 3.16 2.27 2.27 3.32 2.91 2.65 2.94 1.76 2.77 1.54 1.84 1.65 1.70 2.77 3.02 3.02
Zulu zul F 2.09 2.48 2.10 3.00 3.37 3.37 3.78 2.91 2.91 4.11 3.74 3.34 3.80 3.72 3.00 2.17 2.38 2.06 3.03 3.49 3.68 3.72
Akan (Asante Twi) aka F 1.45 1.65 2.34 2.33 2.23 2.23 2.74 2.09 2.09 2.57 2.73 2.30 2.36 2.38 2.28 1.63 1.59 2.17 2.10 2.35 2.70 2.39
Bambara bm F 1.55 2.20 2.59 2.36 2.50 2.50 3.08 2.42 2.42 2.73 3.03 2.51 2.60 2.58 2.46 1.75 1.72 2.17 2.28 2.54 3.00 2.62
Bemba bem F 1.77 2.22 2.22 2.34 2.40 2.40 2.63 2.27 2.27 2.90 2.60 2.42 2.64 2.43 2.33 1.83 1.85 2.07 2.28 2.45 2.55 2.61
Central Atlas Tamazight tzm W 2.48 12.24 12.32 11.98 5.36 5.36 11.96 11.93 11.93 12.06 11.94 9.02 6.13 2.21 2.12 2.11 2.90 2.19 9.13 10.08 11.97 11.94
Central Kanuri knc F 2.18 2.85 2.90 2.83 2.57 2.57 3.07 2.70 2.70 3.17 3.01 2.76 2.83 2.74 2.53 2.23 2.25 2.60 2.60 2.82 2.90 2.95
Dagbani dag W 2.06 2.37 2.58 2.46 2.40 2.40 2.67 2.35 2.35 2.77 2.64 2.39 2.52 2.70 2.51 2.02 2.25 2.39 2.23 2.46 2.70 2.45
Deg mzw W 1.51 1.69 2.11 1.99 2.09 2.09 2.64 2.02 2.02 2.28 2.64 2.10 2.21 2.07 2.11 1.41 1.55 1.98 1.99 2.17 2.64 2.20
Dinka din F 1.61 2.24 2.42 2.30 2.17 2.17 2.60 2.13 2.13 2.48 2.52 2.17 2.37 2.44 2.19 1.66 1.62 2.29 2.11 2.29 2.44 2.25
Dyula dyu F 1.80 2.16 2.16 2.16 2.06 2.06 2.30 2.05 2.05 2.41 2.26 2.10 2.22 2.30 2.07 1.79 1.81 2.03 2.03 2.10 2.20 2.26
Edo bin W 1.32 1.91 2.01 1.98 2.15 2.15 2.62 2.03 2.03 3.06 2.34 2.22 2.40 2.15 2.29 1.52 1.88 1.68 2.07 2.32 2.40 2.60
Fante fat W 1.53 1.76 2.35 2.32 2.26 2.26 2.76 2.14 2.14 2.60 2.75 2.34 2.46 2.47 2.32 1.62 1.69 2.22 2.11 2.38 2.73 2.44
French fra F 1.42 2.08 1.60 1.68 1.49 1.49 1.70 1.43 1.43 1.79 1.70 1.44 1.70 2.51 1.56 1.46 1.62 1.43 1.29 1.41 1.43 1.63
Ga gaa W 1.60 2.04 2.57 2.46 2.44 2.44 3.38 2.46 2.46 2.80 3.33 2.58 2.68 2.55 2.55 1.62 1.77 2.40 2.28 2.60 3.37 2.67
Ghomala bbj W 2.26 2.62 2.85 2.71 2.56 2.56 3.14 2.51 2.51 2.89 3.08 2.59 2.79 2.83 2.52 1.90 2.20 2.79 2.55 2.81 3.05 2.78
Isoko iso W 1.57 1.75 1.86 1.87 2.14 2.14 2.51 1.95 1.95 2.83 2.27 2.16 2.30 1.97 2.20 1.48 1.78 1.62 1.90 2.24 2.30 2.47
Kabiye kbp F 1.83 3.28 4.36 3.97 3.43 3.43 5.05 4.14 4.14 4.26 5.04 4.23 3.63 3.37 3.59 2.08 1.87 3.29 3.59 4.34 4.96 4.30
Kabuverdianu kea F 1.93 2.09 2.10 2.01 1.87 1.87 2.06 1.83 1.83 2.16 2.02 1.85 2.07 2.30 1.82 1.75 1.72 1.95 1.82 1.84 1.91 1.97
Kabyle kab F 2.00 2.68 2.94 2.63 2.55 2.55 3.03 2.60 2.60 2.82 2.93 2.65 2.67 2.79 2.55 2.01 2.17 2.64 2.50 2.64 2.90 2.64
Kamba kam F 1.77 2.29 2.33 2.37 2.38 2.38 2.79 2.31 2.31 2.80 2.57 2.38 2.59 2.55 2.34 1.85 1.84 2.29 2.28 2.43 2.54 2.58
Kasem xsm W 1.45 2.71 3.10 2.84 2.48 2.48 3.27 3.00 3.00 2.99 3.26 3.06 2.51 2.50 2.45 1.67 1.80 2.49 2.74 3.11 3.27 3.11
Kikuyu kik F 1.75 2.93 2.93 2.99 2.86 2.86 3.70 2.92 2.92 3.24 3.04 2.90 3.04 3.11 2.94 1.96 1.93 2.91 2.81 2.92 3.02 3.06
Kimbundu kmb F 1.76 2.14 2.16 2.21 2.23 2.23 2.53 2.15 2.15 2.74 2.50 2.27 2.54 2.36 2.20 1.77 1.80 2.02 2.17 2.29 2.41 2.47
Kirundi run F 1.76 2.14 2.10 2.62 2.65 2.65 2.88 2.37 2.37 3.16 2.85 2.66 2.89 1.98 2.59 1.95 2.01 1.98 2.23 2.70 2.79 2.84
Lingala lin F 1.39 1.85 1.88 1.87 1.95 1.95 2.12 1.82 1.82 2.32 2.11 2.01 2.14 2.07 1.95 1.47 1.43 1.81 1.89 2.01 2.08 2.11
Lolak (llg) llg W 1.75 1.98 2.11 2.00 1.99 1.99 2.33 2.05 2.05 2.54 2.31 2.14 2.30 2.20 1.94 1.70 1.72 2.03 2.01 2.10 2.22 2.28
Lomwe ngl W 2.42 2.66 2.78 2.79 2.84 2.84 3.07 2.74 2.74 3.30 3.05 2.87 3.10 2.94 2.74 2.09 2.48 2.57 2.79 2.86 3.03 3.02
Luganda lug F 2.02 2.55 2.58 2.73 2.80 2.80 3.02 2.53 2.53 3.35 2.99 2.81 3.04 2.77 2.70 1.99 2.18 2.38 2.60 2.87 2.95 2.98
Luo luo F 1.51 1.91 1.97 1.98 1.91 1.91 2.03 1.85 1.85 2.24 2.02 1.92 2.05 2.16 1.90 1.60 1.55 1.96 1.88 1.94 2.01 2.03
Malagasy mlg F 2.01 2.53 2.60 2.61 2.43 2.43 2.65 2.09 2.09 2.86 2.63 2.40 2.67 2.82 2.08 1.64 1.84 1.66 2.45 2.52 2.56 2.55
Mizo lus F 1.74 1.88 1.93 1.87 1.72 1.72 1.90 1.72 1.72 2.04 1.85 1.76 1.91 2.19 1.79 1.68 1.42 1.92 1.78 1.77 1.84 1.87
Mooré mos F 1.62 2.20 2.48 2.43 2.18 2.18 2.49 2.20 2.20 2.62 2.36 2.24 2.27 2.42 2.18 1.71 1.67 2.26 2.16 2.28 2.33 2.35
Nigerian Fulfulde fuv F 1.59 1.76 1.95 1.94 1.99 1.99 2.25 1.92 1.92 2.39 2.22 1.99 2.16 2.03 1.99 1.61 1.67 1.90 2.01 2.10 2.15 2.22
North Ndebele nde W 1.97 2.40 1.84 2.71 3.08 3.08 3.41 2.73 2.73 3.73 3.38 3.07 3.42 3.37 2.79 1.97 2.30 2.00 2.83 3.15 3.35 3.36
Northern Sotho nso F 1.40 1.67 1.74 2.00 1.90 1.90 2.08 1.76 1.76 2.20 2.05 1.91 2.10 2.20 1.88 1.41 1.45 1.49 1.87 1.94 2.01 2.02
Nuer nus F 1.59 2.96 3.40 2.79 2.80 2.80 3.82 3.07 3.07 3.10 3.81 3.13 3.29 2.99 2.83 1.85 2.05 2.82 2.69 3.30 3.70 3.26
Okpe oke W 1.52 2.08 2.33 2.27 2.45 2.45 2.98 2.32 2.32 3.44 2.64 2.48 2.73 2.47 2.66 1.74 2.14 2.23 2.29 2.61 2.75 2.89
Sango sag F 1.33 1.62 1.61 1.58 1.61 1.61 1.80 1.56 1.56 1.93 1.78 1.65 1.79 1.75 1.61 1.34 1.35 1.58 1.57 1.67 1.73 1.76
South Ndebele nbl W 2.10 2.71 2.52 3.00 3.33 3.33 3.66 3.01 3.01 4.00 3.63 3.33 3.68 3.64 3.04 2.27 2.53 2.30 3.11 3.41 3.60 3.60
Swati ssw F 2.23 2.91 2.69 3.14 3.33 3.33 3.73 3.08 3.08 4.00 3.69 3.34 3.76 3.73 3.13 2.41 2.48 2.51 3.18 3.42 3.59 3.63
Tamasheq tmh F 2.20 2.63 2.82 2.59 2.39 2.39 2.80 2.60 2.60 2.79 2.76 2.54 2.55 2.68 2.43 2.10 2.15 2.56 2.41 2.54 2.71 2.61
Tashelhit shi W 1.95 2.46 2.58 2.46 2.21 2.21 2.58 2.34 2.34 2.66 2.52 2.32 2.35 2.52 2.21 1.95 2.02 2.33 2.27 2.35 2.49 2.46
Tiv tiv W 1.27 1.72 1.80 1.71 1.73 1.73 1.86 1.70 1.70 2.02 1.85 1.75 1.87 1.90 1.72 1.39 1.57 1.72 1.73 1.81 1.86 1.85
Tongan ton W 2.01 2.68 2.45 2.39 2.15 2.15 2.76 1.93 1.93 2.65 2.72 2.14 2.50 2.61 2.31 2.20 2.00 2.40 2.16 2.20 2.39 2.41
Tsonga tso F 1.46 1.98 1.95 1.96 2.10 2.10 2.28 1.74 1.74 2.49 2.26 2.11 2.29 2.25 2.03 1.50 1.47 1.84 2.04 2.16 2.23 2.22
Tumbuka tum F 1.66 2.14 2.38 2.54 2.54 2.54 2.86 2.31 2.31 3.13 2.84 2.54 2.84 2.62 2.41 1.81 1.79 1.78 2.46 2.65 2.77 2.82
Umbundu umb F 2.15 2.70 2.76 2.80 2.82 2.82 3.06 2.66 2.66 3.27 2.99 2.82 3.05 2.99 2.69 2.25 2.20 2.59 2.67 2.84 2.96 3.01
Vagla vag W 1.70 1.88 2.40 2.29 2.37 2.37 2.69 2.17 2.17 2.58 2.68 2.24 2.36 2.02 2.03 1.57 1.75 1.95 2.18 2.43 2.67 2.32
Venda ven W 1.47 2.14 2.15 2.22 2.22 2.22 2.45 2.01 2.01 2.70 2.41 2.27 2.40 2.36 2.16 1.59 1.84 1.97 2.21 2.29 2.44 2.40
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support