seda-v0.2: streaming Turkish speech recognition

seda-v0.2 is a small, low-latency speech recognition system for Turkish that runs on a CPU. At its core is a streaming Zipformer2 transducer (66M parameters) trained with icefall and exported to int8 ONNX for sherpa-onnx. The repo also ships a language model, a voice activity detector and a text formatter. sedalib puts them together.

• Real time on a CPU. Words appear while the speaker is talking. The final text is ready about 0.2–0.4 s after they stop in the fast and balanced modes, and 0.6–0.8 s in accurate.
• More accurate than Whisper medium on FLEURS-TR in the accurate mode: 6.38 WER against 6.62.
• Robust: 2–5× fewer errors than seda-v0.1 on noisy, reverberant and telephone audio.
• Written output (optional): digits, punctuation, casing and apostrophes ("Ankara'da").

What changed since seda-v0.1

• Almost 5× more training audio (1,964 h). 1,527 h are real Turkish YouTube speech (YODAS2), labeled by Whisper large-v3-turbo and filtered by agreement with the video's own subtitles.
• A second pass. Each utterance is decoded again with 20 s of left context and a wider beam while it is spoken, so it adds little delay.
• A larger language model: LSTM 2×1024 instead of 2×512, trained on 40M sentences.
• End of speech from Silero VAD v6. Instead of a fixed 1.2 s silence rule, a pause of 0.15–0.4 s ends an utterance.
• A formatter turns the spoken-form output into written text. It is a small Turkish BERT, ytu-ce-cosmos/turkish-small-bert-uncased, fine-tuned for punctuation, casing and apostrophes.

Usage

The easiest way to run the model is sedalib 1.0 or newer:

pip install "sedalib[format]"  # without [format]: no punctuation, casing or digits

from sedalib import Recognizer, read_wav

samples, sample_rate = read_wav("audio.wav")

recognizer = Recognizer.from_pretrained("atasoglu/seda-v0.2", mode="accurate", punctuate=True)
print(recognizer.transcribe(samples, sample_rate))

For live audio, feed the stream small pieces as they arrive. is_endpoint turns true when the speaker pauses:

stream = recognizer.stream()

for chunk in microphone_chunks():  # e.g. 100 ms of 16 kHz audio
    print(stream.accept(chunk, sample_rate=16000))  # partial text
    if stream.is_endpoint:
        print("final:", stream.end_utterance())

from_pretrained downloads only the files of the chosen mode. sedalib 0.2.x still works with this repo. It uses the root model with beam search and the LM, which is the balanced mode below without the VAD. sedalib 0.1.0 pads too little silence for this model's 1.28 s chunks and may drop the last word, so upgrade.

The files are plain sherpa-onnx streaming transducer files, so sherpa_onnx.OnlineRecognizer.from_transducer loads them directly. The LM settings are modified_beam_search, lm_scale=0.6 and lodr_scale=-0.5.

Modes

Mode	Live text	Final text	FLEURS test WER	RTF, 1 thread	RAM
fast	0.64 s chunks, greedy	the same	9.38	0.042	190 MB
balanced (default)	1.28 s chunks, beam 4 + LM	the same	7.26	0.064	225 MB
accurate	0.64 s chunks, greedy	second pass, beam 8 + LM	6.38	0.137	460 MB

• Measurement: sedalib 1.0 on one thread of an Intel Core 5 210H CPU, with audio fed in 100 ms pieces. RTF covers the whole pipeline, second pass included. RAM is the peak of the whole Python process.
• Formatter: adds about 75 MB and a few ms per utterance.

How soon the final text is ready after the speaker stops (median, ms, same CPU):

Mode	App knows the speaker stopped (push-to-talk, stop button)	VAD, 0.15 s pause	VAD, 0.25 s pause (default)
fast	23	224	323
balanced	73	270	370
accurate	434	639	765

• What the VAD numbers include: the pause the VAD waits for, plus the compute.
• Faster CPU: on a desktop Ryzen 5 7500F, accurate is 216 ms for push-to-talk and 411 ms with a 0.15 s pause.
• Threads: more threads do not help a single stream. Run several streams in parallel instead: one recognizer serves them all, with up to 4× the throughput on 8 cores.

Evaluation

WER (%). Lower is better.

• Normalization: references and hypotheses were normalized the same way: lowercase, no punctuation, numbers spelled out.
• seda: CPU, int8, unformatted output. seda-v0.1 is in its released setup (0.64 s chunks, beam 4 + LM).
• Whisper: faster-whisper on a GPU. It decodes the whole recording offline, so it is not a streaming model.

Test set	Whisper small	Whisper medium	Whisper large-v3	seda-v0.1	seda-v0.2 fast	balanced	accurate
FLEURS-TR test (743 utt.)	11.87	6.62	4.62	11.60	9.38	7.26	6.38

Robustness, FLEURS-TR test with added noise, babble or reverb, or sent through a telephone channel (300–3400 Hz, 8 kHz, μ-law), plus MediaSpeech:

• seda-v0.2 column: the second pass in its evaluation setup, which decodes each utterance at once. The accurate mode decodes it incrementally, and is within 0.1 WER of this setup on FLEURS test (6.38 vs 6.30).
• Test conditions are unseen: no noise recording or impulse response used for these sets was used in training.

Test set	Whisper small	Whisper medium	Whisper large-v3	seda-v0.1	seda-v0.2
FLEURS + noise, SNR 10 dB	17.32	9.66	6.08	21.99	8.30
FLEURS + babble, SNR 10 dB	19.83	10.95	7.13	39.56	8.57
FLEURS + reverb (real room responses)	17.74	10.57	6.31	21.58	9.03
FLEURS + telephone channel	15.20	8.62	5.31	25.96	7.44
MediaSpeech TR (broadcast, 10 h)	23.25	16.09	11.36	17.50	11.27

• The second pass beats Whisper small and medium on every set above, and matches Whisper large-v3 on MediaSpeech.
• Whisper large-v3 is still ahead on clean and noisy FLEURS. It has about 23× more parameters and does not stream.

Formatter

F1 of the punctuation after each word, and accuracy of casing and apostrophes, on the correct spoken-form text. The output also goes through rule-based inverse text normalization (digits, %, ordinals).

	.	,	?	Casing	Apostrophe
FLEURS-TR dev	0.93	0.60	–	0.90	0.81
YODAS test (Whisper text)	0.79	0.51	0.78	0.79	0.89

Files

File	Description
encoder.int8.onnx, decoder.onnx, joiner.int8.onnx, tokens.txt	Streaming model, 1.28 s chunks (balanced; what sedalib 0.x loads)
c32/	The same weights, exported with 0.64 s chunks (fast, live text of accurate)
final/	The same weights, exported with 5.12 s chunks and 20.48 s left context (second pass of accurate)
lm/rnnlm.int8.onnx, lm/2gram.fst	LSTM language model and token bigram for LODR
vad/silero_vad.onnx	Silero VAD v6, 16 kHz (MIT, see vad/LICENSE)
formatter/	Text formatter (int8 ONNX token classifier fine-tuned from turkish-small-bert-uncased) and its tokenizer

Model details

Architecture	Zipformer2 encoder + stateless decoder + joiner (pruned RNN-T), causal, trained with dynamic chunks
Parameters	66.1M
Encoder	6 stacks, layers 2,2,3,4,3,2, dims 192,256,384,512,384,256
Input	80-dim log-mel fbank, 16 kHz mono (other rates are resampled)
Output	500 BPE tokens. Lowercase spoken-form Turkish, without punctuation; numbers are words
Quantization	int8 (dynamic) encoder and joiner, fp32 decoder
Language model	LSTM 2×1024 RNN LM (int8, 17 MB) for shallow fusion, token bigram for LODR
Formatter	ytu-ce-cosmos/turkish-small-bert-uncased: 4-layer BERT (29M), with heads for punctuation, casing and apostrophe position; int8 per channel

Training

• Acoustic data: 1,964 h.
  • Base mix, 437 h, seen twice per epoch:
    • ISSAI Turkish Speech Corpus
    • Alania synthetic speech (CC BY 4.0 subset)
    • Khan Academy Turkish
    • Common Voice 17
  • YODAS2 Turkish, 1,527 h of 3,264 h labeled.
    • YouTube videos under CC BY 3.0, labeled by Whisper large-v3-turbo.
    • A segment was kept when the label agreed with the video's manual or automatic subtitles (character error rate filter) and passed confidence and repetition checks.
    • The labels, with the subtitles, filter scores and keep/drop decisions of every window, are published as atasoglu/yodas-tr-whisper-labels.
  • Decontamination: training sentences that shared any word 8-gram with an evaluation sentence were dropped. The evaluation sets were FLEURS, Common Voice, ISSAI, MediaSpeech and Khan Academy test.
• Recipe:
  • Warm-started from an earlier internal model in the seda-v0.1 line.
  • 3 epochs on 2 GPUs: base LR 0.003, fp16, max duration 1200 s.
  • Chunk sizes 16/32/64/full, sampled per batch, so one set of weights serves every export.
  • MUSAN noise augmentation.
  • The release weights average the model over epoch 3.
• Language model:
  • About 40M sentences (1.75B tokens): FineWeb-2 Turkish, Turkish Wikipedia, Alania domain text and the training transcripts.
  • Decontaminated the same way. Validation perplexity is 7.92.
  • The fusion scales were tuned on FLEURS dev only.
• Formatter:
  • Fine-tuned from ytu-ce-cosmos/turkish-small-bert-uncased (MIT).
  • Training text: Wikipedia, FineWeb-2 and the YODAS Whisper labels (train split only), lowercased and stripped of punctuation as input.

Limitations

• Foreign names and abbreviations are the main error source. The model often spells them by sound.
• Overlapping speakers and strong reverb are still well above the clean-speech error rate.
• Commas are hard to place from text alone. The formatter is reliable for sentence ends and casing, less so for commas.
• Turkish only, 16 kHz. Code-switched English words come out in Turkish spelling.

License

The model files are released under CC BY-SA 4.0, because parts of the training data are share-alike: Khan Academy (CC BY-SA 3.0) for the acoustic model, and Wikipedia (CC BY-SA) for the language model and the formatter.

• vad/silero_vad.onnx is Silero VAD, MIT license (vad/LICENSE).
• Other sources and their licenses:
  • YODAS2: CC BY 3.0, derived from YouTube videos by their creators.
  • ISSAI Turkish Speech Corpus: CC BY 4.0.
  • Alania speech and text: CC BY 4.0.
  • Common Voice: CC0.
  • FineWeb-2: ODC-By 1.0.
  • MUSAN: CC BY 4.0, used for augmentation only.
  • Whisper: MIT, used as a labeler only.
  • ytu-ce-cosmos/turkish-small-bert-uncased (YTU COSMOS): MIT, base of the formatter.

Acknowledgements and citation

Built with k2, icefall, lhotse, sherpa-onnx, Silero VAD and Whisper. The formatter builds on the Turkish BERT models of YTU COSMOS.

@inproceedings{li2023yodas,
  title     = {YODAS: Youtube-Oriented Dataset for Audio and Speech},
  author    = {Li, Xinjian and Takamichi, Shinnosuke and Saeki, Takaaki and Chen, William and Shiota, Sayaka and Watanabe, Shinji},
  booktitle = {IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
  year      = {2023}
}
@article{mussakhojayeva2023turkic,
  title   = {Multilingual Speech Recognition for Turkic Languages},
  author  = {Mussakhojayeva, Saida and Dauletbek, Kaisar and Yeshpanov, Rustem and Varol, Huseyin Atakan},
  journal = {Information},
  volume  = {14},
  number  = {2},
  pages   = {74},
  year    = {2023}
}
@inproceedings{ardila2020common,
  title     = {Common Voice: A Massively-Multilingual Speech Corpus},
  author    = {Ardila, Rosana and Branson, Megan and Davis, Kelly and Henretty, Michael and Kohler, Michael and Meyer, Josh and Morais, Reuben and Saunders, Lindsay and Tyers, Francis M. and Weber, Gregor},
  booktitle = {Proceedings of LREC},
  year      = {2020}
}
@misc{snyder2015musan,
  title         = {MUSAN: A Music, Speech, and Noise Corpus},
  author        = {Snyder, David and Chen, Guoguo and Povey, Daniel},
  year          = {2015},
  eprint        = {1510.08484},
  archivePrefix = {arXiv}
}
@inproceedings{yao2024zipformer,
  title     = {Zipformer: A Faster and Better Encoder for Automatic Speech Recognition},
  author    = {Yao, Zengwei and Guo, Liyong and Yang, Xiaoyu and Kang, Wei and Kuang, Fangjun and Yang, Yifan and Jin, Zengrui and Lin, Long and Povey, Daniel},
  booktitle = {ICLR},
  year      = {2024}
}