Arbi-Houssem/TunLangModel_test1.8

Whisper Tunisien

This model is a fine-tuned version of openai/whisper-small on the Tunisian_dataset_STT-TTS30s_filtred1.0 dataset. It achieves the following results on the evaluation set:

• Loss: 4.3957
• Wer: 103.8961

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.4403	16.9492	1000	3.2224	99.5671
0.1001	33.8983	2000	3.4970	104.5214
0.0159	50.8475	3000	3.8371	102.8379
0.0057	67.7966	4000	4.0061	103.9442
0.003	84.7458	5000	4.1216	104.8581
0.0018	101.6949	6000	4.2086	104.4733
0.0012	118.6441	7000	4.2855	104.9543
0.0009	135.5932	8000	4.3365	103.6556
0.0007	152.5424	9000	4.3818	103.9442
0.0006	169.4915	10000	4.3957	103.8961

Framework versions

• Transformers 4.42.0.dev0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1