Whisper Large V3 Fine-tuned on KSC2 (Sybyrla)

This model is a fine-tuned version of openai/whisper-large-v3. It is designed to provide robust automatic speech recognition (ASR) for the Kazakh language, achieving a Word Error Rate (WER) of approximately 17.7%.

To handle real-world acoustic environments in the region, this model was trained on a strategic mix of Kazakh and Russian data.

Developed by: Inflexion Lab
License: Apache License 2.0

Model Description

• Model type: Transformer-based sequence-to-sequence model (Whisper Large V3)
• Language(s): Kazakh (kk), Russian (ru) auxiliary
• Task: Automatic Speech Recognition (ASR)
• Base Model: openai/whisper-large-v3

Performance

The model was evaluated on the held-out test split of the KSC2 dataset.

Metric	Score
WER	~17.7%

Training Data & Methodology

The training dataset was curated to address specific challenges in Kazakh ASR, particularly the lack of punctuation in raw datasets and the prevalence of code-switching in daily speech.

Dataset Composition (80/20 Split)

We utilized a 80% / 20% data mixing strategy to prevent model degradation and improve stability when encountering non-Kazakh phonemes.

1. Kazakh Speech Corpus 2 (KSC2) - ~80%

   • Volume: ~1,200 hours.
   • Processing: The original transcripts are in plain lowercase. We utilized Gemma 27B to syntactically restructure the text, restoring proper capitalization and punctuation.
   • Sources: Parliament speeches, TV/Radio broadcasts, podcasts, and crowdsourced recordings.

2. Common Voice Scripted Speech 23.0 (Russian) - ~20%

   • Volume: ~250 hours.
   • Purpose: Including high-quality Russian speech helps the model distinguish between languages and handle loanwords or code-switching without hallucinating or degrading into gibberish.

Usage

Using with Hugging Face transformers

You can use this model directly with the Hugging Face pipeline.

from transformers import pipeline

# Load the pipeline
pipe = pipeline("automatic-speech-recognition", model="InflexionLab/sybyrla")

# Transcribe an audio file
# The pipeline handles chunking automatically if configured (see batch inference below).
result = pipe("path/to/your/audio.mp3")

print(result["text"])

Citation

If you use this model, its weights, or the punctuation-restored KSC2 transcripts in your research or products, please cite the following paper:

D. Ilyas and A. Mimenbayeva, "Sybyrla: End-to-End Low-Resource Kazakh Speech Recognition Using Whisper with Multilingual Code Switching," 2026 IEEE 6th International Conference on Smart Information Systems and Technologies (SIST), Astana, Kazakhstan, 2026, pp. 1–6, doi: 10.1109/SIST61674.2026.11596103.

BibTeX

@inproceedings{ilyas2026sybyrla,
  author    = {Ilyas, D. and Mimenbayeva, A.},
  title     = {Sybyrla: End-to-End Low-Resource Kazakh Speech Recognition Using Whisper with Multilingual Code Switching},
  booktitle = {2026 IEEE 6th International Conference on Smart Information Systems and Technologies (SIST)},
  address   = {Astana, Kazakhstan},
  year      = {2026},
  pages     = {1--6},
  doi       = {10.1109/SIST61674.2026.11596103},
  keywords  = {Modeling; Multilingual; Automatic speech recognition; Codes; Speech; Switches; Printing; Error analysis; Training; Tuning; Kazakh speech recognition; low-resource ASR; code switching; Whisper; language identification}
}

Please also cite the underlying resources this model builds on: Whisper, the Kazakh Speech Corpus 2 (ISSAI), and Mozilla Common Voice.