ania3000/ossbert-lemm-v2-1

trainer_output

This model is a fine-tuned version of AlexeySorokin/ossbert-onc-unlab-from_multilingual-bs64-5epochs on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1166
• Lemma accuracy: 98.6967
• Sentence accuracy (lemmas): 85.1376

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Lemma accuracy	Sentence accuracy (lemmas)
0.7292	1.0	546	0.2585	95.4135	59.4495
0.2212	2.0	1092	0.1648	97.3559	70.8257
0.1307	3.0	1638	0.1367	97.5815	75.0459
0.0951	4.0	2184	0.1183	98.2080	80.9174
0.0692	5.0	2730	0.1078	98.3208	81.4679
0.0488	6.0	3276	0.1061	98.2832	81.4679
0.041	7.0	3822	0.0979	98.5714	83.6697
0.0289	8.0	4368	0.1050	98.5965	83.4862
0.0223	9.0	4914	0.1098	98.5213	83.6697
0.0171	10.0	5460	0.1014	98.6967	85.1376
0.0084	11.0	6006	0.1028	98.6967	86.0550
0.0074	12.0	6552	0.1009	98.6466	84.5872
0.0057	13.0	7098	0.1009	98.7469	85.3211
0.0052	14.0	7644	0.1059	98.7093	85.5046
0.0035	15.0	8190	0.1070	98.6842	84.7706
0.0035	16.0	8736	0.1103	98.6591	84.5872
0.0022	17.0	9282	0.1166	98.6967	85.1376

Framework versions

• Transformers 4.57.3
• Pytorch 2.11.0+cu130
• Datasets 4.0.0
• Tokenizers 0.22.2