mamba_text_classification

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1792
• Accuracy: {'accuracy': 0.936}
• F1: {'f1': 0.9344262295081968}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.01
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.1361	0.1004	157	0.2596	{'accuracy': 0.892}	{'f1': 0.8778280542986425}
0.1835	0.2009	314	0.2199	{'accuracy': 0.912}	{'f1': 0.9067796610169492}
0.5121	0.3013	471	0.1870	{'accuracy': 0.916}	{'f1': 0.9156626506024096}
0.2751	0.4018	628	0.1886	{'accuracy': 0.924}	{'f1': 0.9224489795918367}
0.3423	0.5022	785	0.1977	{'accuracy': 0.944}	{'f1': 0.9421487603305785}
0.3017	0.6027	942	0.1722	{'accuracy': 0.936}	{'f1': 0.9344262295081968}
0.0069	0.7031	1099	0.1821	{'accuracy': 0.932}	{'f1': 0.9306122448979591}
0.0677	0.8036	1256	0.1775	{'accuracy': 0.936}	{'f1': 0.9344262295081968}
0.2241	0.9040	1413	0.1792	{'accuracy': 0.936}	{'f1': 0.9344262295081968}

Framework versions

• Transformers 4.50.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1