| --- |
| license: cc-by-4.0 |
| base_model: deepset/bert-base-cased-squad2 |
| tags: |
| - generated_from_trainer |
| model-index: |
| - name: bert-12 |
| results: [] |
| --- |
| |
| <!-- This model card has been generated automatically according to the information the Trainer had access to. You |
| should probably proofread and complete it, then remove this comment. --> |
|
|
| # bert-12 |
|
|
| This model is a fine-tuned version of [deepset/bert-base-cased-squad2](https://huggingface.co/deepset/bert-base-cased-squad2) on an unknown dataset. |
| It achieves the following results on the evaluation set: |
| - Loss: 5.4923 |
|
|
| ## Model description |
|
|
| More information needed |
|
|
| ## Intended uses & limitations |
|
|
| More information needed |
|
|
| ## Training and evaluation data |
|
|
| More information needed |
|
|
| ## Training procedure |
|
|
| ### Training hyperparameters |
|
|
| The following hyperparameters were used during training: |
| - learning_rate: 2e-06 |
| - train_batch_size: 8 |
| - eval_batch_size: 8 |
| - seed: 42 |
| - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
| - lr_scheduler_type: linear |
| - num_epochs: 3 |
|
|
| ### Training results |
|
|
| | Training Loss | Epoch | Step | Validation Loss | |
| |:-------------:|:-----:|:----:|:---------------:| |
| | 10.7735 | 0.05 | 5 | 11.9148 | |
| | 10.2311 | 0.09 | 10 | 11.2495 | |
| | 9.7299 | 0.14 | 15 | 10.3860 | |
| | 9.6068 | 0.18 | 20 | 9.5703 | |
| | 8.4179 | 0.23 | 25 | 8.8146 | |
| | 7.4033 | 0.28 | 30 | 8.1396 | |
| | 6.9589 | 0.32 | 35 | 7.5510 | |
| | 6.7006 | 0.37 | 40 | 7.0590 | |
| | 6.4072 | 0.41 | 45 | 6.6821 | |
| | 5.9393 | 0.46 | 50 | 6.4175 | |
| | 5.9838 | 0.5 | 55 | 6.2292 | |
| | 5.7825 | 0.55 | 60 | 6.1019 | |
| | 5.4686 | 0.6 | 65 | 6.0218 | |
| | 5.4427 | 0.64 | 70 | 5.9665 | |
| | 5.3708 | 0.69 | 75 | 5.9212 | |
| | 5.5801 | 0.73 | 80 | 5.8795 | |
| | 5.3894 | 0.78 | 85 | 5.8378 | |
| | 5.4065 | 0.83 | 90 | 5.7994 | |
| | 5.2206 | 0.87 | 95 | 5.7603 | |
| | 5.2044 | 0.92 | 100 | 5.7286 | |
| | 5.1113 | 0.96 | 105 | 5.7037 | |
| | 5.0342 | 1.01 | 110 | 5.6786 | |
| | 4.8333 | 1.06 | 115 | 5.6551 | |
| | 5.0003 | 1.1 | 120 | 5.6361 | |
| | 4.7931 | 1.15 | 125 | 5.6200 | |
| | 4.8148 | 1.19 | 130 | 5.6066 | |
| | 4.9347 | 1.24 | 135 | 5.5940 | |
| | 5.0362 | 1.28 | 140 | 5.5797 | |
| | 4.8616 | 1.33 | 145 | 5.5692 | |
| | 4.3509 | 1.38 | 150 | 5.5624 | |
| | 4.6121 | 1.42 | 155 | 5.5595 | |
| | 4.3364 | 1.47 | 160 | 5.5624 | |
| | 4.5721 | 1.51 | 165 | 5.5667 | |
| | 4.4084 | 1.56 | 170 | 5.5684 | |
| | 4.3097 | 1.61 | 175 | 5.5703 | |
| | 4.3166 | 1.65 | 180 | 5.5751 | |
| | 4.257 | 1.7 | 185 | 5.5807 | |
| | 3.9935 | 1.74 | 190 | 5.5841 | |
| | 4.1078 | 1.79 | 195 | 5.5872 | |
| | 4.2609 | 1.83 | 200 | 5.5840 | |
| | 4.4875 | 1.88 | 205 | 5.5797 | |
| | 4.1496 | 1.93 | 210 | 5.5747 | |
| | 4.333 | 1.97 | 215 | 5.5652 | |
| | 3.9003 | 2.02 | 220 | 5.5594 | |
| | 3.838 | 2.06 | 225 | 5.5551 | |
| | 3.8715 | 2.11 | 230 | 5.5525 | |
| | 4.1982 | 2.16 | 235 | 5.5462 | |
| | 3.9447 | 2.2 | 240 | 5.5395 | |
| | 3.8479 | 2.25 | 245 | 5.5352 | |
| | 3.9253 | 2.29 | 250 | 5.5338 | |
| | 4.0716 | 2.34 | 255 | 5.5316 | |
| | 4.0531 | 2.39 | 260 | 5.5295 | |
| | 4.1593 | 2.43 | 265 | 5.5262 | |
| | 3.8605 | 2.48 | 270 | 5.5230 | |
| | 4.1406 | 2.52 | 275 | 5.5170 | |
| | 4.1568 | 2.57 | 280 | 5.5105 | |
| | 3.5203 | 2.61 | 285 | 5.5061 | |
| | 3.8279 | 2.66 | 290 | 5.5042 | |
| | 4.3043 | 2.71 | 295 | 5.5008 | |
| | 3.9359 | 2.75 | 300 | 5.4978 | |
| | 3.6847 | 2.8 | 305 | 5.4954 | |
| | 3.5765 | 2.84 | 310 | 5.4940 | |
| | 4.035 | 2.89 | 315 | 5.4927 | |
| | 3.4865 | 2.94 | 320 | 5.4924 | |
| | 3.7252 | 2.98 | 325 | 5.4923 | |
|
|
|
|
| ### Framework versions |
|
|
| - Transformers 4.34.0 |
| - Pytorch 2.0.1+cu118 |
| - Datasets 2.14.5 |
| - Tokenizers 0.14.1 |
|
|
