Tembra speech recognition for low-resource languages
Armenian, Georgian, Uzbek, Kazakh, Kyrgyz, Azerbaijani — languages with little public speech data. Tembra AI builds dedicated recognizers for them with one shared approach. This page compares them with the ElevenLabs Scribe v2 and Google Chirp 3 speech-to-text APIs and with open models on the public FLEURS and Common Voice test sets (November 2026).
Lowest average word error rate in every language on this page — fewer errors than both ElevenLabs Scribe v2 and Google Chirp 3 in each of them.
- Armenian: 3.15 % average WER — the lowest in the table: 44 % fewer errors than ElevenLabs Scribe v2 (5.59 %) and 51 % fewer errors than Google Chirp 3 (6.38 %); best open model: NVIDIA stt_hy FastConformer, 5.76 %. On FLEURS (ArmBench-ASR transcripts) the character error rate is below 1 % (0.61 %; ElevenLabs 1.12 %, Google 1.47 %). With the FLEURS transcripts corrected to what the readers actually said, WER is 2.18 % (ElevenLabs 3.71 %, Google 4.09 %).
- Georgian: 6.57 % average WER — the lowest in the table: 4 % fewer errors than ElevenLabs Scribe v2 (6.86 %) and 8 % fewer errors than Google Chirp 3 (7.11 %); best open model: NVIDIA stt_ka FastConformer, 7.71 %. On Common Voice 22: 4.48 %, 31 % fewer errors than ElevenLabs and 41 % fewer errors than Google.
- Uzbek: 6.37 % average WER — the lowest in the table: 50 % fewer errors than ElevenLabs Scribe v2 (12.69 %) and 28 % fewer errors than Google Chirp 3 (8.81 %); best open model: Meta MMS-1B-all, 22.01 %.
- Kazakh: 5.73 % average WER — the lowest in the table: 42 % fewer errors than ElevenLabs Scribe v2 (9.89 %) and 65 % fewer errors than Google Chirp 3 (16.24 %); best open model: Meta Omnilingual ASR CTC 1B v2, 18.28 %.
- Kyrgyz: 5.00 % average WER — the lowest in the table: 62 % fewer errors than ElevenLabs Scribe v2 (13.10 %) and 63 % fewer errors than Google Chirp 3 (13.49 %); best open model: Meta MMS-1B-all, 19.26 %.
- Azerbaijani: 6.25 % WER on FLEURS — the lowest in the table: 22 % fewer errors than ElevenLabs Scribe v2 (7.97 %) and 5 % fewer errors than Google Chirp 3 (6.55 %); best open model: hasan0v/whisper-turbo-az, 10.02 %.
Average word error rate over the public test sets
WER, %, lower is better: the mean of the FLEURS and Common Voice WER of each language (Azerbaijani: FLEURS only — its Common Voice test set, 95 clips, is too small for a reliable comparison). «Best open model» is the open model with the lowest average among the open models in this table (NVIDIA FastConformer, Meta Omnilingual ASR, Meta MMS; for Azerbaijani also hasan0v/whisper-turbo-az, an open fine-tune of Whisper large-v3 turbo); «—» — not evaluated on all test sets of the language (see the per-language tables; Whisper does not support Kyrgyz).
| language | test sets | Tembra | ElevenLabs Scribe v2 | Google Chirp 3 | best open model | OpenAI Whisper large-v3 |
|---|---|---|---|---|---|---|
| Armenian (Հայերեն) | FLEURS + CV | 3.15 | 5.59 | 6.38 | 5.76 (NVIDIA stt_hy FastConformer) | — |
| Georgian (ქართული) | FLEURS + CV | 6.57 | 6.86 | 7.11 | 7.71 (NVIDIA stt_ka FastConformer) | 67.76 |
| Uzbek (Oʻzbekcha) | FLEURS + CV | 6.37 | 12.69 | 8.81 | 22.01 (Meta MMS-1B-all) | — |
| Kazakh (Қазақша) | FLEURS + CV | 5.73 | 9.89 | 16.24 | 18.28 (Meta Omnilingual ASR CTC 1B v2) | — |
| Kyrgyz (Кыргызча) | FLEURS + CV | 5.00 | 13.10 | 13.49 | 19.26 (Meta MMS-1B-all) | — |
| Azerbaijani (Azərbaycanca) | FLEURS | 6.25 | 7.97 | 6.55 | 10.02 (hasan0v/whisper-turbo-az) | 18.94 |
Results per language
WER, %, 95 % bootstrap confidence intervals in parentheses (2,000 resamples over FLEURS sentences or Common Voice speakers/clips).
Armenian
| system | Average | FLEURS test with the ArmBench-ASR transcripts | Common Voice 26 test |
|---|---|---|---|
| Tembra | 3.15 | 2.45 (1.7–3.3) | 3.85 (3.6–4.1) |
| ElevenLabs Scribe v2 | 5.59 | 3.90 (3.0–4.9) | 7.29 (6.9–7.8) |
| NVIDIA stt_hy FastConformer | 5.76 | 6.64 (5.5–7.9) | 4.87 (4.5–5.2) |
| Google Chirp 3 | 6.38 | 4.25 (3.3–5.4) | 8.51 (8.0–9.1) |
| OpenAI Whisper large-v3 | — | 38.65 (36.5–40.8) | — |
Test sets: FLEURS test with the ArmBench-ASR transcripts, utterances without digits or Latin-script words; Common Voice 26 test.
Armenian FLEURS against what the readers actually said
FLEURS transcripts are the sentences the readers were given to read, not a record of what they said. The Armenian FLEURS transcripts on this page include the corrections published with ArmBench-ASR by Metric AI Lab (Metric-AI/fleurs-corrections). On top of them, we checked the places where Tembra, ElevenLabs Scribe v2, Google Chirp 3 or NVIDIA stt_hy FastConformer differs from the transcript: a place was corrected only when the recording clearly says something else and at least two of these systems write the same, so no correction rests on a single system, ours included; spelling variants, proper names and numbers were left as they are. Our check found 13 such places across 12 recordings (12 changed words, 1 skipped). For example, the transcript has «այս աշխարհիկ կյանքից» where the reader says «այս աշխարհի կյանքից»; «արհամարանքի» in the transcript is a typo for «արհամարհանքի»; «փորձառությունն սկսում» has the wrong article before a consonant — the reader says «փորձառությունը». The full list is in eval/fleurs_hy_corrections.json; every entry can be checked by listening to the FLEURS recording. The same transcripts are used for every system, and Tembra has the lowest WER in this table whichever version is used — the original FLEURS transcripts, the ArmBench transcripts or the corrected ones. More on the Armenian FLEURS transcripts: Errors in the Armenian FLEURS test transcripts. WER and character error rate (CER), %:
| system | WER, ArmBench transcripts | WER, corrected transcripts | CER, ArmBench transcripts | CER, corrected transcripts |
|---|---|---|---|---|
| Tembra | 2.45 (1.7–3.3) | 2.18 (1.5–3.0) | 0.61 | 0.55 |
| ElevenLabs Scribe v2 | 3.90 (3.0–4.9) | 3.71 (2.9–4.7) | 1.12 | 1.07 |
| Google Chirp 3 | 4.25 (3.3–5.4) | 4.09 (3.1–5.2) | 1.47 | 1.42 |
| NVIDIA stt_hy FastConformer | 6.64 (5.5–7.9) | 6.54 (5.3–7.7) | 1.70 | 1.66 |
| OpenAI Whisper large-v3 | 38.65 (36.5–40.8) | 38.62 (36.4–40.9) | 8.03 | 8.01 |
CER counts wrong letters instead of wrong words: in Armenian one wrong letter in a long word makes the whole word an error.
Georgian
| system | Average | Common Voice 22 test | FLEURS test |
|---|---|---|---|
| Tembra | 6.57 | 4.48 (3.9–5.1) | 8.66 (7.8–9.5) |
| ElevenLabs Scribe v2 | 6.86 | 6.45 (5.7–7.3) | 7.27 (6.6–8.0) |
| Google Chirp 3 | 7.11 | 7.66 (6.7–8.7) | 6.56 (5.8–7.3) |
| NVIDIA stt_ka FastConformer | 7.71 | 4.29 (3.8–4.9) | 11.12 (10.2–12.1) |
| Meta Omnilingual ASR CTC 1B v2 | 17.16 | 17.33 (16.2–18.6) | 17.00 (15.9–18.1) |
| Meta MMS-1B-all | 23.18 | 22.78 (21.5–24.1) | 23.57 (22.4–24.8) |
| OpenAI Whisper large-v3 | 67.76 | 69.00 (67.4–70.6) | 66.53 (64.8–68.1) |
Test sets: Common Voice 22 test, random 2,000 clips; FLEURS test, 725 utterances without digits.
NVIDIA stt_ka was trained on Common Voice 17 data that contains 10.5 % of these clips and 24.9 % of their sentences; on the 1,501 clips free of both, Tembra has the lowest WER of all systems: 4.14 % vs 4.69 % (NVIDIA), 5.75 % (ElevenLabs), 7.01 % (Google).
Uzbek
| system | Average | FLEURS test | Common Voice 22 test |
|---|---|---|---|
| Tembra | 6.37 | 7.28 (6.4–8.1) | 5.46 (4.8–6.2) |
| Google Chirp 3 | 8.81 | 7.60 (6.7–8.5) | 10.03 (9.0–11.1) |
| ElevenLabs Scribe v2 | 12.69 | 11.60 (10.3–13.0) | 13.78 (12.4–15.2) |
| Meta MMS-1B-all | 22.01 | 23.96 (22.4–25.5) | 20.07 (18.8–21.4) |
| Meta Omnilingual ASR CTC 1B v2 | 30.96 | 24.41 (22.5–26.3) | 37.50 (35.3–39.8) |
| OpenAI Whisper large-v3 | — | 89.58 (85.5–94.8) | — |
Test sets: FLEURS test, 697 utterances without digits; Common Voice 22 test, 2,000 clips (random sample).
Kazakh
| system | Average | FLEURS test | Common Voice 22 test |
|---|---|---|---|
| Tembra | 5.73 | 3.57 (3.0–4.3) | 7.88 (6.5–9.4) |
| ElevenLabs Scribe v2 | 9.89 | 4.40 (3.7–5.2) | 15.39 (13.3–17.5) |
| Google Chirp 3 | 16.24 | 5.18 (4.4–6.0) | 27.30 (24.3–30.5) |
| Meta Omnilingual ASR CTC 1B v2 | 18.28 | 11.71 (10.7–12.8) | 24.85 (21.8–28.0) |
| Meta MMS-1B-all | 19.88 | 13.02 (11.9–14.2) | 26.75 (24.1–29.5) |
| OpenAI Whisper large-v3 | — | 30.24 (28.9–31.7) | — |
Test sets: FLEURS test, 655 utterances without digits; Common Voice 22 test, 533 clips.
Kyrgyz
| system | Average | FLEURS test | Common Voice 22 test |
|---|---|---|---|
| Tembra | 5.00 | 4.76 (4.1–5.4) | 5.24 (3.9–6.9) |
| ElevenLabs Scribe v2 | 13.10 | 9.37 (8.4–10.4) | 16.83 (14.4–19.7) |
| Google Chirp 3 | 13.49 | 8.69 (7.8–9.6) | 18.29 (15.6–21.5) |
| Meta MMS-1B-all | 19.26 | 15.30 (14.2–16.5) | 23.22 (20.8–26.2) |
| Meta Omnilingual ASR CTC 1B v2 | 21.32 | 12.65 (11.5–13.7) | 29.98 (26.8–33.4) |
Test sets: FLEURS test, 750 utterances without digits; Common Voice 22 test, 1,614 clips.
Azerbaijani
| system | FLEURS test |
|---|---|
| Tembra | 6.25 (5.4–7.1) |
| Google Chirp 3 | 6.55 (5.8–7.4) |
| ElevenLabs Scribe v2 | 7.97 (7.1–8.8) |
| hasan0v/whisper-turbo-az | 10.02 (9.1–10.9) |
| OpenAI Whisper large-v3 | 18.94 (17.7–20.3) |
| Meta Omnilingual ASR CTC 1B v2 | 20.92 (19.8–22.1) |
| Meta MMS-1B-all | 23.48 (22.3–24.7) |
Test sets: FLEURS test, 712 utterances without digits.
How we evaluated
- Every system transcribed the same audio files; references and hypotheses go through the same normalization (lowercase, punctuation removed, letters of the language's script only). Utterances whose reference contains digits or words in another script are not scored, for every system alike, because systems write numbers and foreign names differently (digits or words, Latin or native script) and the recording cannot tell which is right; the rule is applied to the reference before normalization. Armenian is scored with the ArmBench-ASR normalization and, for FLEURS, with the ArmBench-ASR corrected transcripts.
- Commercial APIs were called on 5–6 November 2026 with the language set, ElevenLabs sound-event tags switched off (so that tags such
as «(laughter)» are not scored as words) and every other option at its default (ElevenLabs
scribe_v2; Google Cloud Speech-to-Text v2chirp_3, locationus). Open models were run locally with their default decoding. - Model versions and decoding settings of the Tembra models were chosen on dev splits only, never on test data.
- Neither the audio nor the transcript of any scored utterance was used to train or tune the Tembra models.
- The training data of the commercial APIs is not public. NVIDIA's Georgian model was trained on Common Voice 17 train, dev and «other», its Armenian model on Common Voice 17 and the FLEURS train split (per their model cards); in-domain training data can favour them on these tests.
- Averages and per-language tables use the published transcripts of each test set; for Armenian FLEURS, the corrected transcripts published with ArmBench-ASR by Metric AI Lab (
Metric-AI/fleurs-corrections). Armenian FLEURS with our further corrections is shown separately in the Armenian section. eval/fleurs_hy_corrections.jsoncontains transcripts from FLEURS by Google (google/fleurs, CC BY 4.0) and from the ArmBench-ASR corrections by Metric AI Lab (Metric-AI/fleurs-corrections, CC BY 4.0), with our corrections added; this file is released under CC BY 4.0.
Contact
Tembra AI — https://tembra.ai