sea-asr-ph — fast Filipino (Tagalog) and Taglish speech recognition on EmbeddingGemma 2 (CTC)

Pixel ML builds video intelligence for Southeast Asian media. These are the speech models we run for Southeast Asian media: captions, search, clips. Open weights, open data, runs on your GPU — or on a CPU.

Google's EmbeddingGemma 2 audio encoder with a CTC head, trained by Pixel ML on Creative Commons Filipino (Tagalog) and Taglish speech.

Speed: about 24x faster and cheaper per audio hour than batched Whisper large-v3 on the same H100 (FLEURS fil test). About 3,800x real time on H100, 860x on A10G, and faster than real time on a laptop CPU (sea-asr-id: 4.4x on Apple M2 Max, 8 threads). One pass, no autoregressive decode, streaming in 0.48 s chunks. Word timestamps and SRT/VTT output included.

RA 10905 requires closed captions on newscasts and pre-scripted programs in the Philippines; this model makes Filipino captioning affordable on one GPU.

No human-transcribed Filipino YouTube corpus existed. We made the labels: PixelML/sea-asr-ph-data. Trained on CC-BY YouTube. No purchased data.

Results

Same H100 for every open model, greedy decoding, one normaliser for all (quanphid/text.py), WER % (lower is better). Avg = mean of the two test sets. Speed = audio seconds / wall seconds on FLEURS fil, batched, model load excluded. Params = Hugging Face safetensors total (ours: counted from this repo).

Model	Params	Avg	Speed (× RT)	FLEURS fil test (928)	PH conversational, silver (934)
Whisper large-v3	1,543M	14.0	160	10.6	17.4
Whisper large-v3-turbo	809M	14.3	274	11.1	17.5
sea-asr-ph	364M	18.6	3,817	17.2	20.1
Whisper medium	764M	18.6	220	16.5	20.7
Whisper small	242M	25.1	348	25.6	24.5
omniASR CTC-1B ⚠	~1B	—	1,053–1,157	14.0	—
omniASR CTC-300M ⚠	~0.3B	—	1,319–3,058	20.7	—
Qwen3-ASR-1.7B	2,349M	—	2,015	23.8	—
Qwen3-ASR-0.6B	938M	—	1,570	36.1	—
Gemini 3.5 Transcribe (closed API)	—	—	—	6.8	—
sea-asr-ph, streaming 0.48 s	364M	—	931	19.0	—

Size class. Under 0.5B parameters, sea-asr-ph is the most accurate open model we tested on both test sets. It ties Whisper medium (2× its size) on both, at about 17× its speed. Paired 95 % CI of the WER difference (sea-asr-ph − Whisper medium): FLEURS +0.6 [−0.1, +1.3], silver −0.5 [−2.0, +0.5]. Larger Whisper models are more accurate.

• PH conversational, silver: 934 segments (2.68 h) from 150 held-out CC-BY YouTube videos that no sea-asr-ph run saw. Reference = Qwen3-ASR-1.7B text that agreed with omniASR CTC (CER ≤ 0.10); no human check yet. Qwen3-ASR and omniASR made the references, so they are not scored on it; Whisper and sea-asr-ph had no part in them.
• ⚠ omniASR was trained on FLEURS (Meta, arXiv 2511.09690): its FLEURS row is not a held-out score.

Error tail on FLEURS fil, as % of reference words — sea-asr-ph: substitutions 12.6, deletions 2.9, insertions 1.6; Whisper large-v3: substitutions 8.4, deletions 1.1, insertions 1.1. No system had utterances with WER > 100 % on this set; CTC cannot loop by design (no autoregressive decoder).

Same-GPU setup: Whisper via faster-whisper fp16, beam 5, Silero VAD, batch 16 clips; ours 600 s of padded audio per forward pass. A Whisper setup tuned for throughput would narrow the gap. On an A10G: sea-asr-ph 860x, Whisper large-v3 59x real time. Ours: speed from the previous checkpoint of the same architecture and size (this checkpoint measured 3,640× in a separate run); WER from this checkpoint.

The labels come from Whisper large-v3; on read speech (FLEURS) Whisper large-v3 is more accurate. Gemini is a closed benchmark row only; its outputs were not used for training (Gemini API terms).

Use

pip install "git+https://github.com/PixelML/sea-asr@v0.1.0"

from sea_asr import SeaASR, to_srt

asr = SeaASR.from_pretrained("PixelML/sea-asr-ph")          # device="cuda" or "cpu"
print(asr.transcribe("clip.wav"))

out = asr.transcribe("newscast.wav", return_words=True)    # {"text", "words": [{"word", "start", "end"}]}
open("newscast.srt", "w").write(to_srt(out["words"]))

Cues: at most 2 lines × 32 characters, at most 6 s. Input: 16 kHz mono (other rates are resampled). Inputs longer than 30 s are cut at quiet points; timestamps stay on the original timeline.

Limits

• Lower case, no punctuation; caption-ready output needs a punctuation + casing restorer (next, on the EG2 text backbone).
• Machine labels from Whisper large-v3: the model inherits Whisper's errors and does not beat it on read speech.
• Use greedy decoding. We do not ship an n-gram LM.
• Other Philippine languages (Cebuano, Ilocano, …) are not supported and not evaluated.
• Do not use for speaker identification or surveillance.

Training

6,859 CC-BY YouTube videos (1,918.2 h) labelled with Whisper large-v3 (language-ID gate), + FLEURS train, 8×H100, two stages from the base: 3,000 steps on the first 782 h, then 4,000 steps on all data. Labels: PixelML/sea-asr-ph-data. Inference code: PixelML/sea-asr (v0.1.0); the training recipe follows.

Roadmap: punctuation + casing head; Malay, Vietnamese, Thai.

Credits

• Base encoder: Google DeepMind, EmbeddingGemma 2 (914f7f89)
• Data: FLEURS (Google, google/fleurs, 70bb2e84, CC-BY-4.0); CC-BY YouTube creators listed in ATTRIBUTION (6,859 videos, 1,918.2 h)
• Teacher: Whisper large-v3 (OpenAI, openai/whisper, MIT) as Systran/faster-whisper-large-v3 (edaa852e) via faster-whisper 1.2.1 (SYSTRAN)
• Test set: FLEURS fil_ph test (same revision)
• Recipe: Quan Nguyen (@qnguyen3), QUAN-Transcribe — https://github.com/qnguyen3/QUAN-Transcribe (4f45e02)

Licence

Apache-2.0. Base encoder EmbeddingGemma 2 is Apache-2.0; deployments must adhere to the Gemma Prohibited Use Policy.