kungfupanda12/bolo-sun

Bolo Sun v1: Hinglish voice-command speech recognition

Bolo Sun is the speech recogniser in Bolo, an offline Hinglish voice-command assistant for laptops. It takes short spoken commands such as "downloads folder mein terminal kholo" or "volume 40 pe kar do" and writes them in Roman-script Hinglish.

• Base: Oriserve/Whisper-Hindi2Hinglish-Swift (whisper-base size, 73M parameters).
• Training data: fully fine-tuned on synthetic speech only. That means 3,287 command clips covering 20 command types, made with Indic Parler-TTS, with real room impulse responses and noise (OpenSLR 28) and about 3% noise-only clips. Training ran for 5 epochs, about 12 minutes on a free Kaggle T4.
• Format: float16, 142 MB. On an Apple M2 it takes about 100 ms per command with greedy decoding and about 180 ms with beam 5.

Use

from transformers import pipeline

asr = pipeline("automatic-speech-recognition", model="kungfupanda12/bolo-sun")
print(asr("command.wav", generate_kwargs={"language": "en", "task": "transcribe"})["text"])

Passing a file path needs ffmpeg installed; otherwise pass {"raw": samples, "sampling_rate": 16000}.

The full assistant does more than transcribe:

• it keeps the 5 best transcripts;
• it reranks them with the Bolo Tiny command model;
• it checks names against the apps and folders that really exist on the machine.

See the GitHub repo.

Results

Test set	Swift (base)	Bolo Sun
Synthetic, held-out TTS voices: WER	32.1%	16.2%
Real speech, 55 commands, 1 speaker: WER	–	18.3% (v0: 23.7%)
Real speech, same 55: correct action (full Bolo stack)	about 51%	80.0%

Limits on the real-speech numbers:

• They come from one speaker (the author), using a laptop mic in a quiet room. With n = 55 the uncertainty is about ±11 points.
• Some pipeline fixes were found on this same set, so 80% is optimistic.
• Treat this as a research preview until it is tested on more voices.

Known weaknesses:

• English words inside Hindi sentences ("quit arc" heard as "width arduino", "google" as "go well").
• Slight forgetting on long general speech. On the OpenSLR 104 Hindi-English test it was v0: 59.1% WER vs Swift's 55.2%.
• Trained on commands, not dictation.

Every experiment, including the failures, is in WHAT_WE_TRIED.md. One example: training for 10 epochs lowered synthetic WER but made real speech worse.

Data and privacy

• No recordings of real people saying commands were used in training.
• The author's own test clips stay on the author's laptop.
• Bolo runs fully offline, so no audio leaves the device.

License

Apache-2.0, the same as the base model and the TTS teacher.