WhisperLv3-FT: Whisper large-v3 fine-tuned for European Portuguese — pré-Acordo

Whisper large-v3 fine-tuned on 425 h of European Portuguese (EP-425), in 🤗 Transformers format. For long-form audio, use the WhisperX version of this model, WhisperLv3-EP-X, which avoids most hallucinations.

From the paper CAMÕES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguese (INESC-ID / Instituto Superior Técnico). In the paper, this model is Table III → FT → WhisperLv3, trained on EP-425.

Results on the CAMÕES benchmark (WER %, from Table III)

Model	RS	BN	T/L	CS	SI	EP Avg.	AAP	BP
This model (WhisperLv3, fine-tuned)	7.2	4.6	13.6	14.9	43.2	16.7	101.4	28.2
Baseline: original Whisper large-v3, no fine-tuning	32.4	7.9	15.4	18.3	49.0	24.6	29.9	25.8
Baseline: original Whisper large-v3 + WhisperX, no fine-tuning	16.4	8.2	16.6	15.3	39.3	19.2	29.0	24.6

The baselines are the public openai/whisper-large-v3 used as released (zero-shot rows of Table III), with plain Whisper decoding and with WhisperX (voice-activity detection before transcription), respectively.

EP domains: RS = read speech, BN = broadcast news, T/L = talks/lectures, CS = conversational speech, SI = sociolinguistic interviews. EP Avg. gives each domain equal weight. AAP = African and Asian Portuguese, BP = Brazilian Portuguese. All results use the CAMÕES text normaliser (lower-case, no punctuation, numbers spelled out). Leaderboard and test sets: inesc-id/camoes_asr.

Model details

• Base model: openai/whisper-large-v3 (1.55B parameters, all fine-tuned)
• Training data: EP-425, 425 h of European Portuguese from 14 corpora (Section III-A)
• Fine-tuning: 10 epochs, batch size 64, gradient accumulation 4, learning rate 1e-5, 10% warm-up (Section V)
• Decoding: standard Whisper decoding, without voice-activity detection
• Format: 🤗 Transformers (safetensors)

Usage

import torch
from transformers import pipeline

asr = pipeline("automatic-speech-recognition", model="inesc-id/WhisperLv3-FT",
               torch_dtype=torch.float16, device="cuda:0")
print(asr("audio.wav", generate_kwargs={"language": "portuguese", "task": "transcribe"})["text"])

For recordings longer than 30 s, add chunk_length_s=30 to pipeline(...), or use WhisperLv3-EP-X.

Output format

Pre-Acordo Ortográfico. All CAMÕES models are trained on transcripts written in the spelling from before the 1990 Orthographic Agreement (pré-Acordo), e.g. projecto, director, actual.

Transcriptions are normalised like the training transcripts: lower-case, no punctuation, and numbers written out in words.

Other CAMÕES models

Model	Paper row (Table III)	Training data	EP Avg.	BP
WhisperLv3-EP-X	WhisperLv3-X, fine-tuned	EP-425	12.5	27.2
WhisperLv3-FT (this model)	WhisperLv3, fine-tuned	EP-425	16.7	28.2
EBranch-w2vBERT2-EP	EBranch-w2vBERT2, trained from scratch	EP-425	14.4	42.4
WhisperLv3-X-PT-All	WhisperLv3-X, PT-All	EP-425 + BP-417 + AAP-8	13.0	18.8

Citation

@inproceedings{camoes,
  title     = {{CAMÕES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguese}},
  author    = {Carvalho, Carlos and Teixeira, Francisco and Botelho, Catarina and Pompili, Anna and
               Solera-Ure{\~n}a, Rub{\'e}n and Paulo, S{\'e}rgio and Juli{\~a}o, Mariana and Rolland, Thomas and
               Mendon{\c{c}}a, John and Pereira, Diogo and Trancoso, Isabel and Abad, Alberto},
  booktitle = {Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
  year      = {2025}
}