moonshine_tiny_pt_v05

This model is a fine-tuned version of aomocelin/moonshine_tiny_pt_v04 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 11.9352
• Wer: 0.2474

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 4
• eval_batch_size: 128
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.03
• training_steps: 15000
• mixed_precision_training: Native AMP
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.0023	0.5	100	13.1872	21.0266
1.7984	1.0	200	13.1439	8.3488
1.6498	1.5	300	13.0107	3.8961
1.6029	2.0	400	12.9967	2.1645
1.6167	2.5	500	12.9098	1.0513
1.5469	3.0	600	12.8407	0.9895
1.5817	3.5	700	12.8092	0.9895
1.5700	4.0	800	12.7583	0.6184
1.5333	4.5	900	12.7479	0.4947
1.5514	5.0	1000	12.6626	0.3711
1.4990	5.5	1100	12.5735	0.4329
1.5154	6.0	1200	12.5925	0.3711
1.4950	6.5	1300	12.5183	0.3711
1.5409	7.0	1400	12.5098	0.3711
1.4874	7.5	1500	12.4580	0.3711
1.4793	8.0	1600	12.4716	0.3711
1.4829	8.5	1700	12.4669	0.3711
1.5057	9.0	1800	12.4577	0.2474
1.4910	9.5	1900	12.4348	0.3092
1.4876	10.0	2000	12.3987	0.3092
1.4864	10.5	2100	12.3854	0.3092
1.4620	11.0	2200	12.3094	0.3092
1.4675	11.5	2300	12.3030	0.3711
1.4660	12.0	2400	12.3779	0.4329
1.4661	12.5	2500	12.3669	0.4329
1.4481	13.0	2600	12.3337	0.3711
1.4636	13.5	2700	12.3011	0.3092
1.4426	14.0	2800	12.2574	0.3711
1.4514	14.5	2900	12.2546	0.4329
1.4630	15.0	3000	12.3591	0.3711
1.4626	15.5	3100	12.2930	0.3711
1.4434	16.0	3200	12.2102	0.4329
1.4526	16.5	3300	12.2332	0.3711
1.4401	17.0	3400	12.2895	0.3711
1.4565	17.5	3500	12.2564	0.4329
1.4371	18.0	3600	12.2756	0.4329
1.4426	18.5	3700	12.2213	0.4329
1.4332	19.0	3800	12.2142	0.4329
1.4362	19.5	3900	12.2627	0.3711
1.4330	20.0	4000	12.1834	0.4329
1.4546	20.5	4100	12.2190	0.4329
1.4315	21.0	4200	12.2211	0.4947
1.4321	21.5	4300	12.1692	0.4329
1.4220	22.0	4400	12.1869	0.4329
1.4396	22.5	4500	12.1676	0.4947
1.4323	23.0	4600	12.1698	0.4329
1.4180	23.5	4700	12.1681	0.4329
1.4222	24.0	4800	12.1668	0.4329
1.4404	24.5	4900	12.1615	0.4329
1.4186	25.0	5000	12.1415	0.3711
1.4212	25.5	5100	12.1518	0.3711
1.4290	26.0	5200	12.1478	0.4329
1.4337	26.5	5300	12.1383	0.3711
1.4169	27.0	5400	12.0746	0.3711
1.4251	27.5	5500	12.1263	0.3711
1.4240	28.0	5600	12.1202	0.3711
1.4193	28.5	5700	12.0612	0.3711
1.4163	29.0	5800	12.1191	0.3711
1.4291	29.5	5900	12.0887	0.3711
1.4132	30.0	6000	12.0535	0.3711
1.4256	30.5	6100	12.0614	0.3711
1.4169	31.0	6200	12.0862	0.3711
1.4105	31.5	6300	12.1086	0.3711
1.4190	32.0	6400	12.0513	0.3711
1.4239	32.5	6500	12.0702	0.3092
1.4121	33.0	6600	12.0945	0.3092
1.4167	33.5	6700	12.0368	0.3711
1.4153	34.0	6800	12.0486	0.3092
1.4234	34.5	6900	12.0511	0.2474
1.4118	35.0	7000	12.0451	0.3092
1.4186	35.5	7100	12.0541	0.3092
1.4107	36.0	7200	12.0567	0.2474
1.4050	36.5	7300	12.0197	0.3092
1.4203	37.0	7400	12.0170	0.2474
1.4141	37.5	7500	12.0215	0.2474
1.4116	38.0	7600	11.9823	0.3092
1.4130	38.5	7700	12.0061	0.2474
1.4133	39.0	7800	12.0201	0.1855
1.4128	39.5	7900	12.0131	0.1237
1.4109	40.0	8000	12.0168	0.3092
1.4104	40.5	8100	12.0005	0.3092
1.4142	41.0	8200	12.0037	0.1855
1.4067	41.5	8300	11.9886	0.2474
1.4121	42.0	8400	12.0256	0.1855
1.4028	42.5	8500	12.0091	0.1855
1.4106	43.0	8600	12.0162	0.2474
1.4073	43.5	8700	11.9857	0.1855
1.4108	44.0	8800	11.9763	0.2474
1.4102	44.5	8900	11.9710	0.2474
1.4025	45.0	9000	11.9984	0.2474
1.4151	45.5	9100	11.9793	0.2474
1.4004	46.0	9200	11.9892	0.2474
1.4023	46.5	9300	12.0179	0.2474
1.4057	47.0	9400	11.9705	0.2474
1.4021	47.5	9500	12.0061	0.3092
1.4084	48.0	9600	11.9523	0.2474
1.4055	48.5	9700	11.9870	0.2474
1.4035	49.0	9800	11.9755	0.3092
1.4039	49.5	9900	11.9805	0.2474
1.4045	50.0	10000	11.9973	0.3092
1.4036	50.5	10100	12.0119	0.2474
1.4083	51.0	10200	11.9815	0.3092
1.4060	51.5	10300	11.9829	0.2474
1.4011	52.0	10400	11.9878	0.2474
1.4020	52.5	10500	11.9707	0.2474
1.4029	53.0	10600	11.9887	0.2474
1.4057	53.5	10700	11.9829	0.3092
1.4029	54.0	10800	11.9609	0.2474
1.4021	54.5	10900	11.9624	0.2474
1.4017	55.0	11000	11.9813	0.3092
1.4059	55.5	11100	11.9701	0.3092
1.4046	56.0	11200	11.9602	0.2474
1.3987	56.5	11300	11.9707	0.3092
1.4003	57.0	11400	11.9798	0.3092
1.3988	57.5	11500	11.9726	0.2474
1.3989	58.0	11600	11.9846	0.3092
1.3998	58.5	11700	11.9756	0.3092
1.4005	59.0	11800	11.9630	0.2474
1.3981	59.5	11900	11.9629	0.2474
1.3982	60.0	12000	11.9718	0.3092
1.4015	60.5	12100	11.9552	0.2474
1.4045	61.0	12200	11.9519	0.3092
1.4010	61.5	12300	11.9481	0.3092
1.4026	62.0	12400	11.9587	0.3092
1.4054	62.5	12500	11.9356	0.3092
1.3977	63.0	12600	11.9508	0.2474
1.3981	63.5	12700	11.9513	0.2474
1.4080	64.0	12800	11.9629	0.3092
1.4002	64.5	12900	11.9438	0.3092
1.3968	65.0	13000	11.9542	0.3092
1.4008	65.5	13100	11.9455	0.2474
1.3998	66.0	13200	11.9536	0.2474
1.4004	66.5	13300	11.9429	0.2474
1.4014	67.0	13400	11.9375	0.2474
1.3975	67.5	13500	11.9395	0.2474
1.3953	68.0	13600	11.9426	0.2474
1.3968	68.5	13700	11.9387	0.2474
1.4027	69.0	13800	11.9415	0.2474
1.4002	69.5	13900	11.9374	0.2474
1.3973	70.0	14000	11.9329	0.2474
1.4039	70.5	14100	11.9391	0.2474
1.4006	71.0	14200	11.9395	0.2474
1.3992	71.5	14300	11.9313	0.2474
1.4015	72.0	14400	11.9364	0.2474
1.3964	72.5	14500	11.9287	0.2474
1.4040	73.0	14600	11.9344	0.2474
1.3968	73.5	14700	11.9324	0.2474
1.4014	74.0	14800	11.9343	0.2474
1.3969	74.5	14900	11.9352	0.2474
1.3990	75.0	15000	11.9352	0.2474

Framework versions

• Transformers 5.12.1
• Pytorch 2.11.0+cu128
• Datasets 5.0.0
• Tokenizers 0.22.2