Gama, streaming speech recognition for Indic languages mixed with English

Gama

Realtime speech to text for how people in India actually talk. One stream covers English together with Hindi, Marathi, Tamil, and Malayalam. Each of those languages is code mixed with English, and the speaker can switch language in the same sentence.

Audio goes in as it is spoken. The first chunk is 60 ms. Words start coming back while the person is still talking. No second model and no language detector in front.

Why Gama

Code mix, in every language. Hindi, Marathi, Tamil, and Malayalam are each mixed with English inside the same utterance. Gama keeps the English words and the Indic words together instead of forcing the sentence into one language.

Language switch in one stream. A speaker can move from English into Hindi, Marathi, Tamil, or Malayalam without restarting the session.

Built for live use. First chunk 60 ms. Suitable for captions, calls, and voice agents.

Languages

Language Code mix with English
English
Hindi yes
Marathi yes
Tamil yes
Malayalam yes

16 kHz mono. Wav, flac, or anything you convert to that.

How to use

The loader ships inside this repository.

pip install torch torchaudio nemo_toolkit[asr]
import sys
from huggingface_hub import snapshot_download

model_dir = snapshot_download("Thanushttk/gama")
sys.path.insert(0, model_dir)

from indic_stream import IndicStream

asr = IndicStream.from_pretrained(model_dir)
print(asr("audio.wav", lang="hi"))

lang is en, hi, mr, ta, ml, or auto. Use the Indic code when the sentence mixes that language with English. auto is for when you do not know which one is coming.

Boost words

You can pass domain words at runtime. No finetune. Gama raises the probability of those words so names, policies, and product terms from that domain come out correctly.

print(asr.transcribe(
    "audio.wav",
    lang="hi",
    boost=["Jeevan Suraksha Policy", "Family Health Shield", "Golden Future Plan"],
    boost_weight=1.0,
))

boost_weight is how strongly those words are preferred. Start near 1.

Disclaimer. If you give more weight, the decoder can bias toward that word and print it even when it was not clearly spoken.

Numbers and dates

Numbers and dates are a normalization step after the transcript. It takes about 2 ms. You get digits, not the spoken number words: 2011, not the words for two thousand eleven.

Accuracy

Word error rate on held out speech. Lower is better. Read speech is easier than conversation.

Speech WER
English, read 12.4%
Hindi with English, read 11.6%
Hindi with English, conversation 12.2%
Marathi with English, read 14.5%
Tamil with English, read 19.1%
Malayalam with English, read 21.3%

This is the first version

This is the first public version of Gama. I am actively training it and making it better. A distilled model with fewer layers is in progress, aimed at low end devices, with lower word error rate on code mixed speech.

If you like this model, give it a like, and follow me for future updates: https://www.linkedin.com/in/thanush-thankachan-255b31148/

File

indic-streaming-stt.nemo

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support