---
language: en
license: mit
base_model:
- avsolatorio/GIST-small-Embedding-v0
datasets:
- agentlans/en-document-format-classification
tags:
- bert
- text-classification
- sequence-classification
metrics:
- f1
- loss
model-index:
- name: GIST-small-format
  results:
  - task:
      type: text-classification
      name: Text Classification
    metrics:
    - type: f1
      value: 0.8518
      name: Evaluation F1
    - type: loss
      value: 0.3809
      name: Evaluation Loss
---
# GIST-small-format

A fine-tuned version of the **bert** architecture (`BertForSequenceClassification`) optimized for the `text-classification` task.

- **Model type:** bert
- **Problem Type:** single_label_classification
- **Number of Labels:** 24
- **Vocabulary Size:** 30522
- **License:** MIT

## Use

To get started with this model in Python using the Hugging Face Transformers library, run the following code:

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "agentlans/GIST-small-format"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Replace this with your input text."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
predicted_class_name = model.config.id2label[predicted_class_id]

print(f"Predicted Class ID: {predicted_class_id}")
print(f"Predicted Class Name: {predicted_class_name}")
```

## Intended Uses & Limitations

### Intended Use
This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:

| Label ID | Label Name |
|---|---|
| 0 | Academic Writing |
| 1 | Content Listing |
| 2 | Creative Writing |
| 3 | Customer Support Page |
| 4 | Discussion Forum / Comment Section |
| 5 | FAQs |
| 6 | Incomplete Content |
| 7 | Knowledge Article |
| 8 | Legal Notices |
| 9 | Listicle |
| 10 | News Article |
| 11 | Nonfiction Writing |
| 12 | Organizational About Page |
| 13 | Organizational Announcement |
| 14 | Personal About Page |
| 15 | Personal Blog |
| 16 | Product Page |
| 17 | Q&A Forum |
| 18 | Spam / Ads |
| 19 | Structured Data |
| 20 | Technical Writing |
| 21 | Transcript / Interview |
| 22 | Tutorial / How-To Guide |
| 23 | User Reviews |

## Training Details

### Hyperparameters
The following hyperparameters were used during fine-tuning:
- **Learning Rate:** 5e-05
- **Train Batch Size:** 8
- **Eval Batch Size:** 8
- **Optimizer:** OptimizerNames.ADAMW_TORCH_FUSED
- **Number of Epochs:** 3.0
- **Mixed Precision:** BF16

<details>
<summary><b>Show Advanced Training Configuration</b></summary>

#### Optimization & Regularization
- **Gradient Accumulation Steps:** 1
- **Learning Rate Scheduler:** SchedulerType.LINEAR
- **Warmup Steps:** 0
- **Warmup Ratio:** None
- **Weight Decay:** 0.0
- **Max Gradient Norm:** 1.0

#### Hardware & Reproducibility
- **Number of GPUs:** 1
- **Seed:** 42

</details>

## Training Results & Evaluation

During fine-tuning, the model achieved the following results on the evaluation set:

| Metric | Value |
|---|---|
| **Train Loss** | 0.3033 |
| **Validation Loss** | 0.3809 |
| **Validation F1 Score** | 0.8518 |
| **Total FLOPs** | 1.8015e+16 |

For performance on the test set, [click here](report.md).

### Speed Performance
- **Training Runtime:** 2259.7352 seconds
- **Train Samples per Second:** 483.897
- **Evaluation Runtime:** 21.817 seconds
- **Eval Samples per Second:** 1856.355


<details>
<summary><b>Show Detailed Training Logs</b></summary>

### Training Logs History

| Step | Epoch | Learning Rate | Training Loss | Validation Loss | Validation F1 |
|---|---|---|---|---|---|
| 500 | 0.011 | 4.9817e-05 | 1.7311 | N/A | N/A |
| 1000 | 0.022 | 4.9635e-05 | 1.0143 | N/A | N/A |
| 1500 | 0.033 | 4.9452e-05 | 0.8819 | N/A | N/A |
| 2000 | 0.044 | 4.9269e-05 | 0.6946 | N/A | N/A |
| 2500 | 0.055 | 4.9086e-05 | 0.6733 | N/A | N/A |
| 3000 | 0.066 | 4.8903e-05 | 0.6471 | N/A | N/A |
| 3500 | 0.077 | 4.8720e-05 | 0.5822 | N/A | N/A |
| 4000 | 0.088 | 4.8537e-05 | 0.5588 | N/A | N/A |
| 4500 | 0.099 | 4.8354e-05 | 0.5702 | N/A | N/A |
| 5000 | 0.11 | 4.8171e-05 | 0.5574 | N/A | N/A |
| 5500 | 0.121 | 4.7988e-05 | 0.5319 | N/A | N/A |
| 6000 | 0.132 | 4.7806e-05 | 0.5464 | N/A | N/A |
| 6500 | 0.143 | 4.7623e-05 | 0.5369 | N/A | N/A |
| 7000 | 0.154 | 4.7440e-05 | 0.4969 | N/A | N/A |
| 7500 | 0.165 | 4.7257e-05 | 0.5199 | N/A | N/A |
| 8000 | 0.176 | 4.7074e-05 | 0.4873 | N/A | N/A |
| 8500 | 0.187 | 4.6891e-05 | 0.4851 | N/A | N/A |
| 9000 | 0.198 | 4.6708e-05 | 0.4956 | N/A | N/A |
| 9500 | 0.209 | 4.6525e-05 | 0.4595 | N/A | N/A |
| 10000 | 0.219 | 4.6342e-05 | 0.4893 | N/A | N/A |
| 10500 | 0.23 | 4.6159e-05 | 0.4713 | N/A | N/A |
| 11000 | 0.241 | 4.5977e-05 | 0.4707 | N/A | N/A |
| 11500 | 0.252 | 4.5794e-05 | 0.4377 | N/A | N/A |
| 12000 | 0.263 | 4.5611e-05 | 0.4906 | N/A | N/A |
| 12500 | 0.274 | 4.5428e-05 | 0.4677 | N/A | N/A |
| 13000 | 0.285 | 4.5245e-05 | 0.472 | N/A | N/A |
| 13500 | 0.296 | 4.5062e-05 | 0.4726 | N/A | N/A |
| 14000 | 0.307 | 4.4879e-05 | 0.4702 | N/A | N/A |
| 14500 | 0.318 | 4.4696e-05 | 0.4567 | N/A | N/A |
| 15000 | 0.329 | 4.4513e-05 | 0.4336 | N/A | N/A |
| 15500 | 0.34 | 4.4330e-05 | 0.4196 | N/A | N/A |
| 16000 | 0.351 | 4.4148e-05 | 0.478 | N/A | N/A |
| 16500 | 0.362 | 4.3965e-05 | 0.4753 | N/A | N/A |
| 17000 | 0.373 | 4.3782e-05 | 0.4525 | N/A | N/A |
| 17500 | 0.384 | 4.3599e-05 | 0.428 | N/A | N/A |
| 18000 | 0.395 | 4.3416e-05 | 0.4129 | N/A | N/A |
| 18500 | 0.406 | 4.3233e-05 | 0.4529 | N/A | N/A |
| 19000 | 0.417 | 4.3050e-05 | 0.4624 | N/A | N/A |
| 19500 | 0.428 | 4.2867e-05 | 0.4271 | N/A | N/A |
| 20000 | 0.439 | 4.2684e-05 | 0.415 | N/A | N/A |
| 20500 | 0.45 | 4.2501e-05 | 0.4233 | N/A | N/A |
| 21000 | 0.461 | 4.2319e-05 | 0.4353 | N/A | N/A |
| 21500 | 0.472 | 4.2136e-05 | 0.4334 | N/A | N/A |
| 22000 | 0.483 | 4.1953e-05 | 0.4031 | N/A | N/A |
| 22500 | 0.494 | 4.1770e-05 | 0.4011 | N/A | N/A |
| 23000 | 0.505 | 4.1587e-05 | 0.4137 | N/A | N/A |
| 23500 | 0.516 | 4.1404e-05 | 0.4478 | N/A | N/A |
| 24000 | 0.527 | 4.1221e-05 | 0.4157 | N/A | N/A |
| 24500 | 0.538 | 4.1038e-05 | 0.4276 | N/A | N/A |
| 25000 | 0.549 | 4.0855e-05 | 0.4124 | N/A | N/A |
| 25500 | 0.56 | 4.0672e-05 | 0.4065 | N/A | N/A |
| 26000 | 0.571 | 4.0490e-05 | 0.4333 | N/A | N/A |
| 26500 | 0.582 | 4.0307e-05 | 0.4098 | N/A | N/A |
| 27000 | 0.593 | 4.0124e-05 | 0.4091 | N/A | N/A |
| 27500 | 0.604 | 3.9941e-05 | 0.4346 | N/A | N/A |
| 28000 | 0.615 | 3.9758e-05 | 0.4063 | N/A | N/A |
| 28500 | 0.626 | 3.9575e-05 | 0.4149 | N/A | N/A |
| 29000 | 0.636 | 3.9392e-05 | 0.4145 | N/A | N/A |
| 29500 | 0.647 | 3.9209e-05 | 0.3984 | N/A | N/A |
| 30000 | 0.658 | 3.9026e-05 | 0.4113 | N/A | N/A |
| 30500 | 0.669 | 3.8843e-05 | 0.4131 | N/A | N/A |
| 31000 | 0.68 | 3.8661e-05 | 0.4114 | N/A | N/A |
| 31500 | 0.691 | 3.8478e-05 | 0.3915 | N/A | N/A |
| 32000 | 0.702 | 3.8295e-05 | 0.4179 | N/A | N/A |
| 32500 | 0.713 | 3.8112e-05 | 0.3832 | N/A | N/A |
| 33000 | 0.724 | 3.7929e-05 | 0.3826 | N/A | N/A |
| 33500 | 0.735 | 3.7746e-05 | 0.3716 | N/A | N/A |
| 34000 | 0.746 | 3.7563e-05 | 0.3827 | N/A | N/A |
| 34500 | 0.757 | 3.7380e-05 | 0.389 | N/A | N/A |
| 35000 | 0.768 | 3.7197e-05 | 0.4051 | N/A | N/A |
| 35500 | 0.779 | 3.7014e-05 | 0.401 | N/A | N/A |
| 36000 | 0.79 | 3.6831e-05 | 0.3962 | N/A | N/A |
| 36500 | 0.801 | 3.6649e-05 | 0.3653 | N/A | N/A |
| 37000 | 0.812 | 3.6466e-05 | 0.4085 | N/A | N/A |
| 37500 | 0.823 | 3.6283e-05 | 0.3887 | N/A | N/A |
| 38000 | 0.834 | 3.6100e-05 | 0.4255 | N/A | N/A |
| 38500 | 0.845 | 3.5917e-05 | 0.3765 | N/A | N/A |
| 39000 | 0.856 | 3.5734e-05 | 0.3964 | N/A | N/A |
| 39500 | 0.867 | 3.5551e-05 | 0.3849 | N/A | N/A |
| 40000 | 0.878 | 3.5368e-05 | 0.3928 | N/A | N/A |
| 40500 | 0.889 | 3.5185e-05 | 0.3885 | N/A | N/A |
| 41000 | 0.9 | 3.5002e-05 | 0.3638 | N/A | N/A |
| 41500 | 0.911 | 3.4820e-05 | 0.3788 | N/A | N/A |
| 42000 | 0.922 | 3.4637e-05 | 0.3845 | N/A | N/A |
| 42500 | 0.933 | 3.4454e-05 | 0.397 | N/A | N/A |
| 43000 | 0.944 | 3.4271e-05 | 0.3914 | N/A | N/A |
| 43500 | 0.955 | 3.4088e-05 | 0.3809 | N/A | N/A |
| 44000 | 0.966 | 3.3905e-05 | 0.3856 | N/A | N/A |
| 44500 | 0.977 | 3.3722e-05 | 0.3618 | N/A | N/A |
| 45000 | 0.988 | 3.3539e-05 | 0.3732 | N/A | N/A |
| 45500 | 0.999 | 3.3356e-05 | 0.3595 | N/A | N/A |
| 45562 | 1.0 | N/A | N/A | 0.3376 | 0.7906 |
| 46000 | 1.01 | 3.3173e-05 | 0.2933 | N/A | N/A |
| 46500 | 1.021 | 3.2991e-05 | 0.2831 | N/A | N/A |
| 47000 | 1.032 | 3.2808e-05 | 0.2768 | N/A | N/A |
| 47500 | 1.043 | 3.2625e-05 | 0.2947 | N/A | N/A |
| 48000 | 1.054 | 3.2442e-05 | 0.3016 | N/A | N/A |
| 48500 | 1.064 | 3.2259e-05 | 0.2888 | N/A | N/A |
| 49000 | 1.075 | 3.2076e-05 | 0.2698 | N/A | N/A |
| 49500 | 1.086 | 3.1893e-05 | 0.3016 | N/A | N/A |
| 50000 | 1.097 | 3.1710e-05 | 0.2799 | N/A | N/A |
| 50500 | 1.108 | 3.1527e-05 | 0.3036 | N/A | N/A |
| 51000 | 1.119 | 3.1344e-05 | 0.2973 | N/A | N/A |
| 51500 | 1.13 | 3.1162e-05 | 0.2932 | N/A | N/A |
| 52000 | 1.141 | 3.0979e-05 | 0.2789 | N/A | N/A |
| 52500 | 1.152 | 3.0796e-05 | 0.318 | N/A | N/A |
| 53000 | 1.163 | 3.0613e-05 | 0.2868 | N/A | N/A |
| 53500 | 1.174 | 3.0430e-05 | 0.3062 | N/A | N/A |
| 54000 | 1.185 | 3.0247e-05 | 0.3081 | N/A | N/A |
| 54500 | 1.196 | 3.0064e-05 | 0.2754 | N/A | N/A |
| 55000 | 1.207 | 2.9881e-05 | 0.3054 | N/A | N/A |
| 55500 | 1.218 | 2.9698e-05 | 0.2972 | N/A | N/A |
| 56000 | 1.229 | 2.9515e-05 | 0.3048 | N/A | N/A |
| 56500 | 1.24 | 2.9333e-05 | 0.2779 | N/A | N/A |
| 57000 | 1.251 | 2.9150e-05 | 0.2782 | N/A | N/A |
| 57500 | 1.262 | 2.8967e-05 | 0.3115 | N/A | N/A |
| 58000 | 1.273 | 2.8784e-05 | 0.2761 | N/A | N/A |
| 58500 | 1.284 | 2.8601e-05 | 0.287 | N/A | N/A |
| 59000 | 1.295 | 2.8418e-05 | 0.2967 | N/A | N/A |
| 59500 | 1.306 | 2.8235e-05 | 0.2844 | N/A | N/A |
| 60000 | 1.317 | 2.8052e-05 | 0.2875 | N/A | N/A |
| 60500 | 1.328 | 2.7869e-05 | 0.2998 | N/A | N/A |
| 61000 | 1.339 | 2.7686e-05 | 0.2731 | N/A | N/A |
| 61500 | 1.35 | 2.7504e-05 | 0.2941 | N/A | N/A |
| 62000 | 1.361 | 2.7321e-05 | 0.2899 | N/A | N/A |
| 62500 | 1.372 | 2.7138e-05 | 0.305 | N/A | N/A |
| 63000 | 1.383 | 2.6955e-05 | 0.295 | N/A | N/A |
| 63500 | 1.394 | 2.6772e-05 | 0.2861 | N/A | N/A |
| 64000 | 1.405 | 2.6589e-05 | 0.2653 | N/A | N/A |
| 64500 | 1.416 | 2.6406e-05 | 0.2749 | N/A | N/A |
| 65000 | 1.427 | 2.6223e-05 | 0.2906 | N/A | N/A |
| 65500 | 1.438 | 2.6040e-05 | 0.2712 | N/A | N/A |
| 66000 | 1.449 | 2.5857e-05 | 0.3095 | N/A | N/A |
| 66500 | 1.46 | 2.5675e-05 | 0.3181 | N/A | N/A |
| 67000 | 1.471 | 2.5492e-05 | 0.2544 | N/A | N/A |
| 67500 | 1.481 | 2.5309e-05 | 0.2504 | N/A | N/A |
| 68000 | 1.492 | 2.5126e-05 | 0.3013 | N/A | N/A |
| 68500 | 1.503 | 2.4943e-05 | 0.2951 | N/A | N/A |
| 69000 | 1.514 | 2.4760e-05 | 0.2822 | N/A | N/A |
| 69500 | 1.525 | 2.4577e-05 | 0.2482 | N/A | N/A |
| 70000 | 1.536 | 2.4394e-05 | 0.2707 | N/A | N/A |
| 70500 | 1.547 | 2.4211e-05 | 0.2601 | N/A | N/A |
| 71000 | 1.558 | 2.4028e-05 | 0.3073 | N/A | N/A |
| 71500 | 1.569 | 2.3846e-05 | 0.2815 | N/A | N/A |
| 72000 | 1.58 | 2.3663e-05 | 0.2587 | N/A | N/A |
| 72500 | 1.591 | 2.3480e-05 | 0.2825 | N/A | N/A |
| 73000 | 1.602 | 2.3297e-05 | 0.2921 | N/A | N/A |
| 73500 | 1.613 | 2.3114e-05 | 0.2775 | N/A | N/A |
| 74000 | 1.624 | 2.2931e-05 | 0.2611 | N/A | N/A |
| 74500 | 1.635 | 2.2748e-05 | 0.2656 | N/A | N/A |
| 75000 | 1.646 | 2.2565e-05 | 0.2757 | N/A | N/A |
| 75500 | 1.657 | 2.2382e-05 | 0.2574 | N/A | N/A |
| 76000 | 1.668 | 2.2199e-05 | 0.2874 | N/A | N/A |
| 76500 | 1.679 | 2.2017e-05 | 0.2507 | N/A | N/A |
| 77000 | 1.69 | 2.1834e-05 | 0.2621 | N/A | N/A |
| 77500 | 1.701 | 2.1651e-05 | 0.2704 | N/A | N/A |
| 78000 | 1.712 | 2.1468e-05 | 0.2748 | N/A | N/A |
| 78500 | 1.723 | 2.1285e-05 | 0.2705 | N/A | N/A |
| 79000 | 1.734 | 2.1102e-05 | 0.3062 | N/A | N/A |
| 79500 | 1.745 | 2.0919e-05 | 0.298 | N/A | N/A |
| 80000 | 1.756 | 2.0736e-05 | 0.2788 | N/A | N/A |
| 80500 | 1.767 | 2.0553e-05 | 0.2542 | N/A | N/A |
| 81000 | 1.778 | 2.0370e-05 | 0.2742 | N/A | N/A |
| 81500 | 1.789 | 2.0188e-05 | 0.2559 | N/A | N/A |
| 82000 | 1.8 | 2.0005e-05 | 0.2805 | N/A | N/A |
| 82500 | 1.811 | 1.9822e-05 | 0.2572 | N/A | N/A |
| 83000 | 1.822 | 1.9639e-05 | 0.2701 | N/A | N/A |
| 83500 | 1.833 | 1.9456e-05 | 0.2636 | N/A | N/A |
| 84000 | 1.844 | 1.9273e-05 | 0.2724 | N/A | N/A |
| 84500 | 1.855 | 1.9090e-05 | 0.2652 | N/A | N/A |
| 85000 | 1.866 | 1.8907e-05 | 0.2604 | N/A | N/A |
| 85500 | 1.877 | 1.8724e-05 | 0.262 | N/A | N/A |
| 86000 | 1.888 | 1.8541e-05 | 0.2594 | N/A | N/A |
| 86500 | 1.899 | 1.8359e-05 | 0.2609 | N/A | N/A |
| 87000 | 1.909 | 1.8176e-05 | 0.2769 | N/A | N/A |
| 87500 | 1.92 | 1.7993e-05 | 0.2803 | N/A | N/A |
| 88000 | 1.931 | 1.7810e-05 | 0.2635 | N/A | N/A |
| 88500 | 1.942 | 1.7627e-05 | 0.2759 | N/A | N/A |
| 89000 | 1.953 | 1.7444e-05 | 0.2669 | N/A | N/A |
| 89500 | 1.964 | 1.7261e-05 | 0.2887 | N/A | N/A |
| 90000 | 1.975 | 1.7078e-05 | 0.2838 | N/A | N/A |
| 90500 | 1.986 | 1.6895e-05 | 0.2482 | N/A | N/A |
| 91000 | 1.997 | 1.6712e-05 | 0.2344 | N/A | N/A |
| 91124 | 2.0 | N/A | N/A | 0.3564 | 0.8288 |
| 91500 | 2.008 | 1.6529e-05 | 0.1882 | N/A | N/A |
| 92000 | 2.019 | 1.6347e-05 | 0.1598 | N/A | N/A |
| 92500 | 2.03 | 1.6164e-05 | 0.1572 | N/A | N/A |
| 93000 | 2.041 | 1.5981e-05 | 0.1645 | N/A | N/A |
| 93500 | 2.052 | 1.5798e-05 | 0.1709 | N/A | N/A |
| 94000 | 2.063 | 1.5615e-05 | 0.1649 | N/A | N/A |
| 94500 | 2.074 | 1.5432e-05 | 0.1513 | N/A | N/A |
| 95000 | 2.085 | 1.5249e-05 | 0.1468 | N/A | N/A |
| 95500 | 2.096 | 1.5066e-05 | 0.2013 | N/A | N/A |
| 96000 | 2.107 | 1.4883e-05 | 0.1819 | N/A | N/A |
| 96500 | 2.118 | 1.4700e-05 | 0.1817 | N/A | N/A |
| 97000 | 2.129 | 1.4518e-05 | 0.1869 | N/A | N/A |
| 97500 | 2.14 | 1.4335e-05 | 0.164 | N/A | N/A |
| 98000 | 2.151 | 1.4152e-05 | 0.1722 | N/A | N/A |
| 98500 | 2.162 | 1.3969e-05 | 0.1798 | N/A | N/A |
| 99000 | 2.173 | 1.3786e-05 | 0.1633 | N/A | N/A |
| 99500 | 2.184 | 1.3603e-05 | 0.1812 | N/A | N/A |
| 100000 | 2.195 | 1.3420e-05 | 0.1719 | N/A | N/A |
| 100500 | 2.206 | 1.3237e-05 | 0.1556 | N/A | N/A |
| 101000 | 2.217 | 1.3054e-05 | 0.2062 | N/A | N/A |
| 101500 | 2.228 | 1.2871e-05 | 0.1641 | N/A | N/A |
| 102000 | 2.239 | 1.2689e-05 | 0.1656 | N/A | N/A |
| 102500 | 2.25 | 1.2506e-05 | 0.1639 | N/A | N/A |
| 103000 | 2.261 | 1.2323e-05 | 0.173 | N/A | N/A |
| 103500 | 2.272 | 1.2140e-05 | 0.1895 | N/A | N/A |
| 104000 | 2.283 | 1.1957e-05 | 0.1707 | N/A | N/A |
| 104500 | 2.294 | 1.1774e-05 | 0.1851 | N/A | N/A |
| 105000 | 2.305 | 1.1591e-05 | 0.1782 | N/A | N/A |
| 105500 | 2.316 | 1.1408e-05 | 0.1703 | N/A | N/A |
| 106000 | 2.327 | 1.1225e-05 | 0.1581 | N/A | N/A |
| 106500 | 2.337 | 1.1042e-05 | 0.1648 | N/A | N/A |
| 107000 | 2.348 | 1.0860e-05 | 0.1986 | N/A | N/A |
| 107500 | 2.359 | 1.0677e-05 | 0.1799 | N/A | N/A |
| 108000 | 2.37 | 1.0494e-05 | 0.1611 | N/A | N/A |
| 108500 | 2.381 | 1.0311e-05 | 0.1644 | N/A | N/A |
| 109000 | 2.392 | 1.0128e-05 | 0.1457 | N/A | N/A |
| 109500 | 2.403 | 9.9451e-06 | 0.1498 | N/A | N/A |
| 110000 | 2.414 | 9.7622e-06 | 0.1812 | N/A | N/A |
| 110500 | 2.425 | 9.5793e-06 | 0.1837 | N/A | N/A |
| 111000 | 2.436 | 9.3964e-06 | 0.167 | N/A | N/A |
| 111500 | 2.447 | 9.2135e-06 | 0.1736 | N/A | N/A |
| 112000 | 2.458 | 9.0306e-06 | 0.1638 | N/A | N/A |
| 112500 | 2.469 | 8.8477e-06 | 0.1659 | N/A | N/A |
| 113000 | 2.48 | 8.6647e-06 | 0.1822 | N/A | N/A |
| 113500 | 2.491 | 8.4818e-06 | 0.1583 | N/A | N/A |
| 114000 | 2.502 | 8.2989e-06 | 0.1486 | N/A | N/A |
| 114500 | 2.513 | 8.1160e-06 | 0.1723 | N/A | N/A |
| 115000 | 2.524 | 7.9331e-06 | 0.1505 | N/A | N/A |
| 115500 | 2.535 | 7.7502e-06 | 0.1588 | N/A | N/A |
| 116000 | 2.546 | 7.5673e-06 | 0.1401 | N/A | N/A |
| 116500 | 2.557 | 7.3844e-06 | 0.1572 | N/A | N/A |
| 117000 | 2.568 | 7.2015e-06 | 0.1915 | N/A | N/A |
| 117500 | 2.579 | 7.0186e-06 | 0.1555 | N/A | N/A |
| 118000 | 2.59 | 6.8357e-06 | 0.1793 | N/A | N/A |
| 118500 | 2.601 | 6.6528e-06 | 0.1671 | N/A | N/A |
| 119000 | 2.612 | 6.4699e-06 | 0.1405 | N/A | N/A |
| 119500 | 2.623 | 6.2870e-06 | 0.1504 | N/A | N/A |
| 120000 | 2.634 | 6.1041e-06 | 0.1683 | N/A | N/A |
| 120500 | 2.645 | 5.9212e-06 | 0.1562 | N/A | N/A |
| 121000 | 2.656 | 5.7383e-06 | 0.1648 | N/A | N/A |
| 121500 | 2.667 | 5.5554e-06 | 0.1531 | N/A | N/A |
| 122000 | 2.678 | 5.3725e-06 | 0.1582 | N/A | N/A |
| 122500 | 2.689 | 5.1896e-06 | 0.1415 | N/A | N/A |
| 123000 | 2.7 | 5.0067e-06 | 0.1391 | N/A | N/A |
| 123500 | 2.711 | 4.8238e-06 | 0.1349 | N/A | N/A |
| 124000 | 2.722 | 4.6409e-06 | 0.1772 | N/A | N/A |
| 124500 | 2.733 | 4.4580e-06 | 0.1687 | N/A | N/A |
| 125000 | 2.744 | 4.2751e-06 | 0.1536 | N/A | N/A |
| 125500 | 2.754 | 4.0922e-06 | 0.1394 | N/A | N/A |
| 126000 | 2.765 | 3.9093e-06 | 0.1729 | N/A | N/A |
| 126500 | 2.776 | 3.7264e-06 | 0.1582 | N/A | N/A |
| 127000 | 2.787 | 3.5435e-06 | 0.176 | N/A | N/A |
| 127500 | 2.798 | 3.3606e-06 | 0.1595 | N/A | N/A |
| 128000 | 2.809 | 3.1777e-06 | 0.1665 | N/A | N/A |
| 128500 | 2.82 | 2.9948e-06 | 0.1628 | N/A | N/A |
| 129000 | 2.831 | 2.8119e-06 | 0.1426 | N/A | N/A |
| 129500 | 2.842 | 2.6290e-06 | 0.164 | N/A | N/A |
| 130000 | 2.853 | 2.4461e-06 | 0.1562 | N/A | N/A |
| 130500 | 2.864 | 2.2632e-06 | 0.1632 | N/A | N/A |
| 131000 | 2.875 | 2.0803e-06 | 0.1562 | N/A | N/A |
| 131500 | 2.886 | 1.8974e-06 | 0.1538 | N/A | N/A |
| 132000 | 2.897 | 1.7145e-06 | 0.1649 | N/A | N/A |
| 132500 | 2.908 | 1.5316e-06 | 0.1576 | N/A | N/A |
| 133000 | 2.919 | 1.3487e-06 | 0.1481 | N/A | N/A |
| 133500 | 2.93 | 1.1658e-06 | 0.1535 | N/A | N/A |
| 134000 | 2.941 | 9.8291e-07 | 0.1552 | N/A | N/A |
| 134500 | 2.952 | 8.0001e-07 | 0.1542 | N/A | N/A |
| 135000 | 2.963 | 6.1711e-07 | 0.148 | N/A | N/A |
| 135500 | 2.974 | 4.3421e-07 | 0.1502 | N/A | N/A |
| 136000 | 2.985 | 2.5131e-07 | 0.1423 | N/A | N/A |
| 136500 | 2.996 | 6.8405e-08 | 0.152 | N/A | N/A |
| 136686 | 3.0 | N/A | N/A | 0.3809 | 0.8518 |

</details>


## Framework Versions

- **Transformers:** 5.14.0.dev0
- **PyTorch:** 2.13.0+cu130