Arbi-Houssem/TunLangModel_test1.13

Whisper Tunisien

This model is a fine-tuned version of openai/whisper-small on the Tunisian_dataset_STT-TTS15s_filtred1.0_Mixed dataset. It achieves the following results on the evaluation set:

• Loss: 1.1841
• Wer: 44.4699

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.3626	4.5045	500	0.8379	53.0340
0.0527	9.0090	1000	0.9350	48.5440
0.0111	13.5135	1500	1.0400	49.4907
0.0049	18.0180	2000	1.1030	44.6564
0.0017	22.5225	2500	1.1338	44.7568
0.0014	27.0270	3000	1.1618	44.8142
0.0009	31.5315	3500	1.1784	44.8429
0.0009	36.0360	4000	1.1841	44.4699

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1