jehone-shqip / README.md
klaudiaX's picture
Model save
befd3fc verified
|
Raw History Blame Contribute Delete
3.71 kB
metadata
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: jehone-shqip
    results: []

jehone-shqip

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0986
  • Wer: 50.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 8
  • eval_batch_size: 16
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • training_steps: 40
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
0.2919 0.31 1 1.0503 57.3333
0.4118 0.62 2 1.0144 56.6667
0.2816 0.92 3 0.9834 55.3333
0.1495 1.23 4 0.9662 53.3333
0.1293 1.54 5 0.9698 50.0
0.0617 1.85 6 0.9868 54.0
0.0802 2.15 7 1.0159 55.3333
0.0519 2.46 8 1.0413 56.6667
0.0367 2.77 9 1.0518 58.0000
0.0304 3.08 10 1.0545 57.3333
0.022 3.38 11 1.0491 58.6667
0.0169 3.69 12 1.0441 56.6667
0.014 4.0 13 1.0361 57.3333
0.0106 4.31 14 1.0315 56.0000
0.0079 4.62 15 1.0294 58.0000
0.0081 4.92 16 1.0300 56.6667
0.006 5.23 17 1.0319 56.6667
0.006 5.54 18 1.0353 56.6667
0.0048 5.85 19 1.0398 57.3333
0.0045 6.15 20 1.0453 56.0000
0.0041 6.46 21 1.0506 55.3333
0.0039 6.77 22 1.0558 55.3333
0.004 7.08 23 1.0613 54.0
0.0033 7.38 24 1.0663 52.6667
0.0034 7.69 25 1.0706 52.6667
0.0031 8.0 26 1.0751 52.0
0.003 8.31 27 1.0791 52.0
0.0028 8.62 28 1.0828 51.3333
0.0026 8.92 29 1.0861 50.6667
0.003 9.23 30 1.0890 50.6667
0.0026 9.54 31 1.0912 50.6667
0.0022 9.85 32 1.0928 50.0
0.0024 10.15 33 1.0944 50.0
0.0023 10.46 34 1.0957 50.0
0.0024 10.77 35 1.0966 50.0
0.0024 11.08 36 1.0974 50.0
0.0023 11.38 37 1.0979 50.0
0.0023 11.69 38 1.0983 50.0
0.0023 12.0 39 1.0983 50.0
0.002 12.31 40 1.0986 50.0

Framework versions

  • Transformers 4.40.0.dev0
  • Pytorch 2.2.1+cu121
  • Datasets 2.18.0
  • Tokenizers 0.15.2