Arbi-Houssem/TunLangModel_test1.9

Whisper Tunisien

This model is a fine-tuned version of openai/whisper-base on the Tunisian_dataset_STT-TTS15s_filtred1.0 dataset. It achieves the following results on the evaluation set:

• Loss: 4.7577
• Wer: 117.6907

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.5147	7.7519	500	3.1417	128.5618
0.0559	15.5039	1000	3.8111	132.5456
0.01	23.2558	1500	4.2115	120.1891
0.0029	31.0078	2000	4.4628	120.3916
0.0017	38.7597	2500	4.6127	111.2086
0.0011	46.5116	3000	4.6945	124.6455
0.0009	54.2636	3500	4.7426	113.3018
0.0009	62.0155	4000	4.7577	117.6907

Framework versions

• Transformers 4.42.0.dev0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1