dusersad12 commited on
Commit
a61aaee
·
verified ·
1 Parent(s): 55f571c

Add model card with sweep results filled in

Browse files
Files changed (1) hide show
  1. README.md +58 -0
README.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: transformers
4
+ tags:
5
+ - text-classification
6
+ - roberta
7
+ - hyperparameter-sweep
8
+ ---
9
+ # SweepBestModel
10
+
11
+ <div align="center">
12
+ <img src="figures/fig1.png" width="70%" alt="SweepBestModel overview" />
13
+ </div>
14
+
15
+ ## Overview
16
+
17
+ SweepBestModel is a RoBERTa-base model fine-tuned for sequence classification through a systematic hyperparameter sweep. We explored learning rates and weight decay values to find the optimal configuration.
18
+
19
+ ## Training Configuration
20
+
21
+ | Run | Learning Rate | Weight Decay | Best Checkpoint | Best F1 |
22
+ |---|---|---|---|---|
23
+ | run_lr2e-5_wd0.01 | 2e-5 | 0.01 | \u2014 | {RESULT} |
24
+ | run_lr5e-5_wd0.01 | 5e-5 | 0.01 | \u2014 | {RESULT} |
25
+ | run_lr1e-4_wd0.01 | 1e-4 | 0.01 | \u2014 | {RESULT} |
26
+ | run_lr2e-5_wd0.1 | 2e-5 | 0.1 | \u2014 | {RESULT} |
27
+
28
+ ## Sweep Results
29
+
30
+ <div align="center">
31
+
32
+ | Run | Learning Rate | Weight Decay | Best Eval F1 |
33
+ |---|---|---|---|
34
+ | run_lr2e-5_wd0.01 | 2e-5 | 0.01 | 0.827 |
35
+ | run_lr5e-5_wd0.01 | 5e-5 | 0.01 | 0.856 |
36
+ | run_lr1e-4_wd0.01 | 1e-4 | 0.01 | 0.793 |
37
+ | run_lr2e-5_wd0.1 | 2e-5 | 0.1 | 0.741 |
38
+
39
+ </div>
40
+
41
+ <p align="center">
42
+ <img width="60%" src="figures/fig2.png">
43
+ </p>
44
+
45
+ The best performing configuration used a learning rate of 5e-5 with weight decay 0.01, achieving the highest F1 score across all sweep runs.
46
+
47
+ ## Usage
48
+
49
+ ```python
50
+ from transformers import AutoModelForSequenceClassification, AutoTokenizer
51
+
52
+ model = AutoModelForSequenceClassification.from_pretrained("SweepBest-TestRepo")
53
+ tokenizer = AutoTokenizer.from_pretrained("SweepBest-TestRepo")
54
+ ```
55
+
56
+ ## License
57
+
58
+ This model is released under the Apache 2.0 license.