Oswald-ASR-0.6-V2

Haitian Creole speech recognition, the second version of Oswald-ASR-0.6-V1: fine-tuned from Qwen/Qwen3-ASR-0.6B-hf, which does not speak Kreyòl.

Gold-transcript test sets: WER 9.1 / CER 3.5 (V1: 10.3 / 3.6), on 1316 clips, identical for both models. Numbers recognised whole: 90.3% (V1: 70.7%) of 290 numbers. Real phones, new speakers (Datamio, 10 speakers never trained on): WER 5.2 (V1: 11.5).

What changed from V1

• Datamio: 9.42 h of paid contributors reading on their phones, 91 speakers from all 10 departments, approved by validators. Split by speaker: 10 held out as a new real-phone test, 4 for validation, the rest trained on three times over.
• The fr-ht Bible clips with gold text: recordings of the Bible in the fr-ht language-ID sets, relabelled with the printed text instead of a machine transcript; clips from the held-out Bible books removed. The first Bible set is capped lower so scripture keeps the share it had in V1.
• Numbers and named entities: kreyol-fatiman-vocab, sentences full of dates, prices, times, phone numbers, percentages, ages, names, places, stations and government services.
• Numbers written as digits. V1 wrote every number in words. V2's training targets are converted by an inverse speller (scripts/whisper_ft/itn.py): amounts, dates, years, times, phone numbers, percentages, decimals and units become digits and symbols. Scoring reads both models' output back into one canonical spoken form, so V2's digits and V1's words are compared on the same words; a number counts as right only if it is right whole.
• Same recipe as V1: LoRA r=64 on the decoder and on the attention projections (q/k/v) of the audio encoder's 18 layers, the encoder's other weights frozen; lr 0.0001, up to 3 epochs with early stopping on validation; 166.29 h, 73983 clips (Datamio counted three times).

Results

V1 and V2 transcribed the same test clips, beam 4. Numbers is the share of the reference's numbers that come out whole: a year heard as "de mil kat" instead of "de mil vennkat" is wrong.

Gold transcripts

Test set	Clips	V1 WER	V2 WER	V1 CER	V2 CER	V1 numbers	V2 numbers
bible_read	150	4.0	4.4	1.5	1.6	100% of 10	100% of 10
datamio_phone	441	11.5	5.2	3.6	1.8	64% of 179	96% of 179
read_cmu	150	6.8	7.1	2.2	2.3	78% of 27	81% of 27
studio_finance	63	4.1	4.3	0.8	1.0	100% of 1	100% of 1
studio_health	116	5.7	5.7	2.2	2.2	0% of 1	100% of 1
studio_history	52	16.8	16.4	5.0	5.0	50% of 18	50% of 18
studio_legal	46	3.2	3.9	0.8	1.1	–	–
studio_numbers	85	6.7	5.0	1.8	1.6	100% of 39	95% of 39
studio_social	93	7.0	6.9	2.3	2.2	100% of 4	100% of 4
youtube	120	24.6	29.8	10.1	14.3	64% of 11	55% of 11

Other test sets (machine or synthetic references: agreement, not accuracy)

Test set	Clips	V1 WER	V2 WER	V1 CER	V2 CER	V1 numbers	V2 numbers
radio	150	12.0	12.1	4.1	4.1	93% of 30	93% of 30
synth_vocab	32	18.2	11.8	5.2	3.2	94% of 17	100% of 17

Training mix

Domain	Hours	Train / val / test clips	Gold
radio	51.0	16,340 / 1,276 / 1,918	no
bible_read	26.8	9,966 / 223 / 493	yes
synth	21.4	10,985 / 0 / 0	no
read_cmu	12.1	3,782 / 344 / 360	yes
datamio_phone	9.4	3,313 / 225 / 441	yes
synth_openai	7.9	4,639 / 0 / 0	no
youtube	7.0	2,396 / 114 / 120	yes
studio_health	5.3	2,854 / 114 / 116	yes
bible_frht	5.1	1,827 / 0 / 0	yes
bible_narrowband	3.8	2,786 / 0 / 0	yes
studio_social	3.7	2,116 / 96 / 93	yes
synth_vocab	3.7	1,531 / 37 / 32	no
bible_frht_yt	2.1	763 / 0 / 0	yes
studio_legal	2.0	828 / 48 / 46	yes
studio_numbers	1.4	1,008 / 83 / 85	yes
studio_finance	1.2	669 / 66 / 63	yes
studio_history	1.2	492 / 46 / 52	yes
synth_google	0.7	553 / 0 / 0	no
synth_modern	0.5	217 / 0 / 0	no
studio_ecom	0.5	292 / 0 / 0	yes

Use

from transformers import AutoProcessor, Qwen3ASRForConditionalGeneration

processor = AutoProcessor.from_pretrained("jsbeaudry/Oswald-ASR-0.6-V2")
model = Qwen3ASRForConditionalGeneration.from_pretrained("jsbeaudry/Oswald-ASR-0.6-V2", device_map="auto")

conversation = [[
    {"role": "user", "content": [{"type": "audio", "path": "clip.wav"}]},
    {"role": "assistant", "content": [{"type": "text", "text": "language Haitian Creole<asr_text>"}]},
]]
inputs = processor.apply_chat_template(
    conversation, tokenize=True, return_dict=True, continue_final_message=True,
).to(model.device, model.dtype)
ids = model.generate(**inputs, max_new_tokens=256, num_beams=4)
print(processor.decode(ids[:, inputs["input_ids"].shape[1]:], return_format="transcription_only")[0])

The assistant turn is prefilled with the language tag: the processor's own language= argument checks the base model's 30 names and rejects "Haitian Creole", the language this model adds. Requires transformers>=5.13.

Numbers come out as digits: "1500 goud", "17 desanm 2024", "25%", "32°", "5 km", "3:30", "3456-7890", "90.5". Text is lowercase without punctuation otherwise. A number below 10 on its own stays a word ("twa moun") unless money, a unit, a month or a time follows it ("5 goud", "3 me", "8è").

GGUF for llama.cpp: jsbeaudry/Oswald-ASR-0.6-V2-GGUF.

Limitations

• Datamio takes were accepted only after V1 understood them (≥ 60–70%), so its test set leans toward speech V1 already handled; it measures new voices and phones, not the hardest audio.
• Radio transcripts are machine-made; that domain is never scored as accuracy.
• Synthetic speech is about a fifth of the mix (numbers, units, modern vocabulary, extra voices).
• Repetition loops. On one of 25 YouTube test clips V2 repeats a phrase until the token limit, which alone moves that set from V1's 24.6 to 29.8 WER; without that clip the two are level (24.2 vs 24.3). Cap max_new_tokens by the clip's length (about 6 tokens a second is generous for Kreyòl speech) and drop a phrase repeated more than three times in a row.