beabucayan/lingkodai-whisper-eng

LingkodAI Whisper: English

Speech-to-text for English, fine-tuned from openai/whisper-medium with only the decoder trained. It transcribes queries identified as English by the audio-based language identification stage of the LingkodAI pipeline.

How it works

Uses Whisper's built-in English language token (<|en|>), so no new tokens were added. The encoder was kept frozen, so it is identical to the original whisper-medium encoder.

Training data

English read speech from the Philippine Languages Database (PLD), resampled to 16 kHz, with a speaker-independent train/validation/test split.

Usage

Set the language to "english" so Whisper doesn't guess the language on its own.

import torch, librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

repo = "lingkodai/lingkodai-whisper-eng"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if device == "cuda" else torch.float32

processor = WhisperProcessor.from_pretrained(repo, language="english", task="transcribe")
model = WhisperForConditionalGeneration.from_pretrained(repo, dtype=dtype).to(device).eval()
model.generation_config.forced_decoder_ids = processor.get_decoder_prompt_ids(language="english", task="transcribe")

audio, _ = librosa.load("clip.wav", sr=16000)
features = processor.feature_extractor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device, dtype=dtype)
with torch.no_grad():
    ids = model.generate(features, max_length=225)
print(processor.tokenizer.batch_decode(ids, skip_special_tokens=True)[0].strip())