demo-tjbert-morph

This model is a fine-tuned version of google-bert/bert-base-multilingual-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.5950

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.3339	200	2.2239
No log	0.6678	400	2.0909
No log	1.0017	600	1.9933
No log	1.3356	800	1.9482
No log	1.6694	1000	1.8904
No log	2.0033	1200	1.8300
No log	2.3372	1400	1.7884
No log	2.6711	1600	1.7494
No log	3.0050	1800	1.7091
No log	3.3389	2000	1.6628
No log	3.6728	2200	1.6603
No log	4.0067	2400	1.6243
No log	4.3406	2600	1.6028
No log	4.6745	2800	1.5906
No log	5.0	2995	1.5950

Framework versions

• Transformers 5.17.0
• Pytorch 2.11.0+cu128
• Datasets 5.0.1
• Tokenizers 0.23.2