metadata
license: apache-2.0
library_name: transformers
tags:
- text-classification
- roberta
- hyperparameter-sweep
SweepBestModel
Overview
SweepBestModel is a RoBERTa-base model fine-tuned for sequence classification through a systematic hyperparameter sweep. We explored learning rates and weight decay values to find the optimal configuration.
Training Configuration
| Run | Learning Rate | Weight Decay | Best Checkpoint | Best F1 |
|---|---|---|---|---|
| run_lr2e-5_wd0.01 | 2e-5 | 0.01 | \u2014 | {RESULT} |
| run_lr5e-5_wd0.01 | 5e-5 | 0.01 | \u2014 | {RESULT} |
| run_lr1e-4_wd0.01 | 1e-4 | 0.01 | \u2014 | {RESULT} |
| run_lr2e-5_wd0.1 | 2e-5 | 0.1 | \u2014 | {RESULT} |
Sweep Results
| Run | Learning Rate | Weight Decay | Best Eval F1 |
|---|---|---|---|
| run_lr2e-5_wd0.01 | 2e-5 | 0.01 | 0.827 |
| run_lr5e-5_wd0.01 | 5e-5 | 0.01 | 0.856 |
| run_lr1e-4_wd0.01 | 1e-4 | 0.01 | 0.793 |
| run_lr2e-5_wd0.1 | 2e-5 | 0.1 | 0.741 |
The best performing configuration used a learning rate of 5e-5 with weight decay 0.01, achieving the highest F1 score across all sweep runs.
Usage
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model = AutoModelForSequenceClassification.from_pretrained("SweepBest-TestRepo")
tokenizer = AutoTokenizer.from_pretrained("SweepBest-TestRepo")
License
This model is released under the Apache 2.0 license.