w2v-bert-2.0-sinhala-30h

This model is a fine-tuned version of facebook/w2v-bert-2.0 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2217
  • Wer: 34.1484
  • Cer: 7.0189
  • Wer Np: 28.9988
  • Cer Np: 5.9716
  • Wer Nz: 28.7399
  • Cer Nz: 5.9108
  • Empty Pct: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 8
  • eval_batch_size: 1
  • seed: 3407
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • optimizer: Use OptimizerNames.ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 200
  • num_epochs: 3
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer Wer Np Cer Np Wer Nz Cer Nz Empty Pct
0.6838 0.3234 500 0.6585 56.5017 15.1627 53.0207 14.2190 52.7906 14.0921 0.0
0.5962 0.6468 1000 0.4345 47.8711 11.2543 43.4695 10.2704 43.2969 10.1678 0.0
0.4908 0.9702 1500 0.4159 44.3613 10.4466 40.1323 9.4631 39.8446 9.3556 0.0
0.4314 1.2937 2000 0.3456 41.7722 9.3745 37.0829 8.3593 36.7952 8.2791 0.0
0.3796 1.6171 2500 0.3226 40.6214 9.0380 35.8458 8.0286 35.6732 7.9708 0.0
0.3137 1.9405 3000 0.2931 37.8308 8.2784 32.9402 7.2603 32.7100 7.1586 0.0
0.2701 2.2639 3500 0.2672 36.5650 7.8698 31.6168 6.8323 31.3003 6.7378 0.0
0.2437 2.5873 4000 0.2359 34.6375 7.3650 29.4879 6.3217 29.2002 6.2387 0.0
0.2251 2.9107 4500 0.2217 34.1484 7.0189 28.9988 5.9716 28.7399 5.9108 0.0

Framework versions

  • Transformers 4.56.2
  • Pytorch 2.10.0+cu128
  • Datasets 3.6.0
  • Tokenizers 0.22.2
Downloads last month
309
Safetensors
Model size
0.6B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for TriLinSL/w2v-bert-2.0-sinhala-30h

Finetuned
(539)
this model