nadinev/naija-voice-model

Accent Labs — Whisper Large v3 Turbo (merged LoRA)

Merged full-weight checkpoint for automatic speech recognition. LoRA adapters trained in-repo were merged into the base encoder–decoder weights so you can load a single WhisperForConditionalGeneration without PEFT at inference time.

Model summary

Field	Value
Run id	2026-05-01_whisper-enhanced_v1
Base model	openai/whisper-large-v3-turbo
Fine-tuning	LoRA (PEFT), then merge into base
Best adapter step	1350 → see ../adapter/checkpoints/checkpoint-1350/
Training manifest	whisper_train_enriched_v3 (data/processed/manifests/whisper_train_enriched_v3.csv)
dtype (config)	bfloat16 (see config.json)

LoRA configuration (pre-merge)

From ../run.yaml / ../adapter/checkpoints/checkpoint-1350/adapter_config.json:

• r = 32, alpha = 64, dropout = 0.05
• Target modules: q_proj, k_proj, v_proj, out_proj, fc1, fc2
• bias = none

Architecture (from config.json)

Key	Value
Model type	whisper (WhisperForConditionalGeneration)
Encoder layers	32
Decoder layers	4
d_model	1280
Encoder/decoder heads	20
FFN dim	5120
Mel bins	128
Vocab size	51866

Total parameter count matches the base Whisper Large v3 Turbo scale (on the order of ~800M parameters for the full merged model; exact count not stored in config.json).

Intended use

• Primary: Speech-to-text for English audio, with project focus on Nigerian English and multi-accent backgrounds (Yoruba-, Igbo-, Hausa-influenced, Pidgin-adjacent, etc.) per Accent Labs strategy docs.
• Not for: Languages outside the training/eval distribution, whisper-quiet or heavy code-switching without validation, or high-stakes decisions without human review.

Evaluation (reported from training export)

The following metrics come from Reports/training_reports/training_run_train_enhanced_30042026.json (notebook train_enhanced-30042026, model_version: whisper-large-v3-turbo-lora-r32). They describe the training/eval pipeline used for that export; re-run eval on this merged folder if you need a guarantee of step-for-step parity with checkpoint-1350.

Metric	Value
Baseline overall WER	0.1851
Best validation WER	0.1611
Final eval WER (overall)	0.1582
Eval sample count (overall row)	1580

WER by dialect (final eval, same report):

Dialect tag	n	WER
hausa	214	0.1459
igbo	1074	0.1599
pidgin	20	0.1375
yoruba	238	0.1620
unknown	19	0.1723
multilingual	15	0.1932

Decode and eval settings should match the training notebook (e.g. beam size, max_new_tokens) when comparing numbers.

How to use (transformers)

from transformers import WhisperProcessor, WhisperForConditionalGeneration
import torch

model_dir = "."  # path to this merged folder
processor = WhisperProcessor.from_pretrained(model_dir)
model = WhisperForConditionalGeneration.from_pretrained(
    model_dir,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# Load audio → input_features, then:
# generated_ids = model.generate(**inputs)
# text = processor.batch_decode(generated_ids, skip_special_tokens=True)

Use processor + audio sampling rate expected by Whisper (16 kHz after processor handling).

Training details

• Framework: Transformers + PEFT (adapter training); merged artifacts built with compatible versions (see config.json → transformers_version).
• Epochs (recorded): 3
• Warm-start (report JSON): prior adapter checkpoint path recorded in training_run_train_enhanced_30042026.json (see warm_start_adapter).

Full run metadata: ../run.yaml.

Bias, risks, and limitations

• Metrics above are on a specific eval split; performance will vary by noise, domain, and speaker demographics.
• Small n for some dialect buckets (e.g. pidgin, multilingual) — treat those WERs as indicative, not production guarantees.
• ASR can hallucinate or mis-transcribe; downstream use for compliance, medical, or legal contexts needs human oversight.
• License / attribution: honor the base model license and terms for openai/whisper-large-v3-turbo. This adapter/merge does not remove those obligations.

Citation

Base model (Whisper):

@misc{radford2022whisper,
  title={Robust Speech Recognition via Large-Scale Weak Supervision},
  author={Alec Radford and Jong Wook Kim and Tao Xu and Greg Brockman and Christine McLeavey and Ilya Sutskever},
  year={2022},
  eprint={2212.04356},
  archivePrefix={arXiv},
  primaryClass={eess.AS}
}

Contact

Accent Labs — internal project; see repository README.md at repo root for layout and support processes.

Acknowledgments & Data Sources

• Initial Training Data: Portions of the initial dataset used for acoustic adaptation and baseline development were sourced from African Voice AI (authentic human-generated voice data from underserved regions). We acknowledge and thank their contributors for providing rights-cleared regional speech resources.