Deploy backend

training/training-checkpoints/checkpoint-1700/README.md

@@ -0,0 +1,202 @@
+---
+base_model: distilbert-base-uncased
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

training/training-checkpoints/checkpoint-1700/adapter_config.json

@@ -0,0 +1,32 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "distilbert-base-uncased",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_lin",
+    "q_lin"
+  ],
+  "task_type": "SEQ_CLS",
+  "use_dora": false,
+  "use_rslora": false
+}

training/training-checkpoints/checkpoint-1700/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b52ac92aa40a1961914c95fecbf881e70122a9d3ebeee62134c2225bddd1d3a
+size 2962192

training/training-checkpoints/checkpoint-1700/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:729b1404d5313567c877b13e089e0ea2800af9b90b5da1641dd58d50f6710905
+size 5940315

training/training-checkpoints/checkpoint-1700/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b8eeb2605876a3277ab8cfa76859ef162300347ed099232fc6d280a4d0ca430
+size 14455

training/training-checkpoints/checkpoint-1700/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e0ff435ef12a33c4c275d84760be406df49508c4b1bcebad4951172631c2c9b
+size 1465

training/training-checkpoints/checkpoint-1700/tokenizer_config.json

@@ -0,0 +1,15 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

training/training-checkpoints/checkpoint-1700/trainer_state.json

@@ -0,0 +1,1233 @@
+{
+  "best_global_step": 1700,
+  "best_metric": 0.9833333333333333,
+  "best_model_checkpoint": "./training-checkpoints\\checkpoint-1700",
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 1700,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0058823529411764705,
+      "grad_norm": 1.510934829711914,
+      "learning_rate": 0.00019964705882352944,
+      "loss": 0.6532491683959961,
+      "step": 10
+    },
+    {
+      "epoch": 0.011764705882352941,
+      "grad_norm": 2.264005661010742,
+      "learning_rate": 0.0001992549019607843,
+      "loss": 0.49631505012512206,
+      "step": 20
+    },
+    {
+      "epoch": 0.01764705882352941,
+      "grad_norm": 1.0282903909683228,
+      "learning_rate": 0.0001988627450980392,
+      "loss": 0.29676687717437744,
+      "step": 30
+    },
+    {
+      "epoch": 0.023529411764705882,
+      "grad_norm": 1.0084155797958374,
+      "learning_rate": 0.00019847058823529412,
+      "loss": 0.1594167470932007,
+      "step": 40
+    },
+    {
+      "epoch": 0.029411764705882353,
+      "grad_norm": 0.23450879752635956,
+      "learning_rate": 0.00019807843137254903,
+      "loss": 0.09263747334480285,
+      "step": 50
+    },
+    {
+      "epoch": 0.03529411764705882,
+      "grad_norm": 0.44396597146987915,
+      "learning_rate": 0.00019768627450980392,
+      "loss": 0.18005683422088622,
+      "step": 60
+    },
+    {
+      "epoch": 0.041176470588235294,
+      "grad_norm": 0.1429290622472763,
+      "learning_rate": 0.00019729411764705883,
+      "loss": 0.08807324171066284,
+      "step": 70
+    },
+    {
+      "epoch": 0.047058823529411764,
+      "grad_norm": 0.847214937210083,
+      "learning_rate": 0.00019690196078431374,
+      "loss": 0.058199501037597655,
+      "step": 80
+    },
+    {
+      "epoch": 0.052941176470588235,
+      "grad_norm": 3.822068929672241,
+      "learning_rate": 0.00019650980392156862,
+      "loss": 0.1674056887626648,
+      "step": 90
+    },
+    {
+      "epoch": 0.058823529411764705,
+      "grad_norm": 1.4017611742019653,
+      "learning_rate": 0.00019611764705882353,
+      "loss": 0.11085774898529052,
+      "step": 100
+    },
+    {
+      "epoch": 0.06470588235294118,
+      "grad_norm": 1.6271415948867798,
+      "learning_rate": 0.00019572549019607844,
+      "loss": 0.10848402976989746,
+      "step": 110
+    },
+    {
+      "epoch": 0.07058823529411765,
+      "grad_norm": 0.17168374359607697,
+      "learning_rate": 0.00019533333333333336,
+      "loss": 0.08450813293457031,
+      "step": 120
+    },
+    {
+      "epoch": 0.07647058823529412,
+      "grad_norm": 2.110332489013672,
+      "learning_rate": 0.00019494117647058824,
+      "loss": 0.08588783144950866,
+      "step": 130
+    },
+    {
+      "epoch": 0.08235294117647059,
+      "grad_norm": 1.973347783088684,
+      "learning_rate": 0.00019454901960784315,
+      "loss": 0.07138832211494446,
+      "step": 140
+    },
+    {
+      "epoch": 0.08823529411764706,
+      "grad_norm": 2.4691383838653564,
+      "learning_rate": 0.00019415686274509806,
+      "loss": 0.1480124831199646,
+      "step": 150
+    },
+    {
+      "epoch": 0.09411764705882353,
+      "grad_norm": 0.8067235946655273,
+      "learning_rate": 0.00019376470588235295,
+      "loss": 0.03993956744670868,
+      "step": 160
+    },
+    {
+      "epoch": 0.1,
+      "grad_norm": 2.022461175918579,
+      "learning_rate": 0.00019337254901960786,
+      "loss": 0.06878547668457032,
+      "step": 170
+    },
+    {
+      "epoch": 0.10588235294117647,
+      "grad_norm": 1.4254077672958374,
+      "learning_rate": 0.00019298039215686274,
+      "loss": 0.12785714864730835,
+      "step": 180
+    },
+    {
+      "epoch": 0.11176470588235295,
+      "grad_norm": 3.5772814750671387,
+      "learning_rate": 0.00019258823529411765,
+      "loss": 0.07902452945709229,
+      "step": 190
+    },
+    {
+      "epoch": 0.11764705882352941,
+      "grad_norm": 0.06748726963996887,
+      "learning_rate": 0.00019219607843137256,
+      "loss": 0.06560993194580078,
+      "step": 200
+    },
+    {
+      "epoch": 0.12352941176470589,
+      "grad_norm": 5.759597301483154,
+      "learning_rate": 0.00019180392156862745,
+      "loss": 0.16235246658325195,
+      "step": 210
+    },
+    {
+      "epoch": 0.12941176470588237,
+      "grad_norm": 3.6009647846221924,
+      "learning_rate": 0.00019141176470588236,
+      "loss": 0.06680867075920105,
+      "step": 220
+    },
+    {
+      "epoch": 0.13529411764705881,
+      "grad_norm": 2.608488082885742,
+      "learning_rate": 0.00019101960784313727,
+      "loss": 0.08542757034301758,
+      "step": 230
+    },
+    {
+      "epoch": 0.1411764705882353,
+      "grad_norm": 1.4347050189971924,
+      "learning_rate": 0.00019062745098039215,
+      "loss": 0.06062295436859131,
+      "step": 240
+    },
+    {
+      "epoch": 0.14705882352941177,
+      "grad_norm": 3.4521889686584473,
+      "learning_rate": 0.00019023529411764706,
+      "loss": 0.11894316673278808,
+      "step": 250
+    },
+    {
+      "epoch": 0.15294117647058825,
+      "grad_norm": 2.5669424533843994,
+      "learning_rate": 0.00018984313725490197,
+      "loss": 0.04931341111660004,
+      "step": 260
+    },
+    {
+      "epoch": 0.1588235294117647,
+      "grad_norm": 0.5097957253456116,
+      "learning_rate": 0.00018945098039215689,
+      "loss": 0.0895132601261139,
+      "step": 270
+    },
+    {
+      "epoch": 0.16470588235294117,
+      "grad_norm": 0.1470881551504135,
+      "learning_rate": 0.00018905882352941177,
+      "loss": 0.07752076387405396,
+      "step": 280
+    },
+    {
+      "epoch": 0.17058823529411765,
+      "grad_norm": 0.12387822568416595,
+      "learning_rate": 0.00018866666666666668,
+      "loss": 0.0910811185836792,
+      "step": 290
+    },
+    {
+      "epoch": 0.17647058823529413,
+      "grad_norm": 0.1579340398311615,
+      "learning_rate": 0.0001882745098039216,
+      "loss": 0.024501879513263703,
+      "step": 300
+    },
+    {
+      "epoch": 0.18235294117647058,
+      "grad_norm": 0.19159211218357086,
+      "learning_rate": 0.00018788235294117648,
+      "loss": 0.025437775254249572,
+      "step": 310
+    },
+    {
+      "epoch": 0.18823529411764706,
+      "grad_norm": 0.3070846199989319,
+      "learning_rate": 0.0001874901960784314,
+      "loss": 0.0452291190624237,
+      "step": 320
+    },
+    {
+      "epoch": 0.19411764705882353,
+      "grad_norm": 1.647754192352295,
+      "learning_rate": 0.0001870980392156863,
+      "loss": 0.07508809566497802,
+      "step": 330
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 0.2747861444950104,
+      "learning_rate": 0.00018670588235294118,
+      "loss": 0.08880412578582764,
+      "step": 340
+    },
+    {
+      "epoch": 0.20588235294117646,
+      "grad_norm": 0.20420950651168823,
+      "learning_rate": 0.00018631372549019607,
+      "loss": 0.047925958037376405,
+      "step": 350
+    },
+    {
+      "epoch": 0.21176470588235294,
+      "grad_norm": 0.5672637820243835,
+      "learning_rate": 0.00018592156862745098,
+      "loss": 0.04176274836063385,
+      "step": 360
+    },
+    {
+      "epoch": 0.21764705882352942,
+      "grad_norm": 1.5938951969146729,
+      "learning_rate": 0.0001855294117647059,
+      "loss": 0.06371157765388488,
+      "step": 370
+    },
+    {
+      "epoch": 0.2235294117647059,
+      "grad_norm": 0.019027238711714745,
+      "learning_rate": 0.0001851372549019608,
+      "loss": 0.05675685405731201,
+      "step": 380
+    },
+    {
+      "epoch": 0.22941176470588234,
+      "grad_norm": 1.8165510892868042,
+      "learning_rate": 0.00018474509803921568,
+      "loss": 0.10704467296600342,
+      "step": 390
+    },
+    {
+      "epoch": 0.23529411764705882,
+      "grad_norm": 0.3322693109512329,
+      "learning_rate": 0.0001843529411764706,
+      "loss": 0.08637534379959107,
+      "step": 400
+    },
+    {
+      "epoch": 0.2411764705882353,
+      "grad_norm": 0.06156998872756958,
+      "learning_rate": 0.0001839607843137255,
+      "loss": 0.17954374551773072,
+      "step": 410
+    },
+    {
+      "epoch": 0.24705882352941178,
+      "grad_norm": 3.327615737915039,
+      "learning_rate": 0.0001835686274509804,
+      "loss": 0.1696428656578064,
+      "step": 420
+    },
+    {
+      "epoch": 0.2529411764705882,
+      "grad_norm": 2.1997804641723633,
+      "learning_rate": 0.0001831764705882353,
+      "loss": 0.11364388465881348,
+      "step": 430
+    },
+    {
+      "epoch": 0.25882352941176473,
+      "grad_norm": 2.8368852138519287,
+      "learning_rate": 0.0001827843137254902,
+      "loss": 0.03553434014320374,
+      "step": 440
+    },
+    {
+      "epoch": 0.2647058823529412,
+      "grad_norm": 0.2659618556499481,
+      "learning_rate": 0.00018239215686274512,
+      "loss": 0.03715018928050995,
+      "step": 450
+    },
+    {
+      "epoch": 0.27058823529411763,
+      "grad_norm": 0.04204031080007553,
+      "learning_rate": 0.000182,
+      "loss": 0.13120421171188354,
+      "step": 460
+    },
+    {
+      "epoch": 0.27647058823529413,
+      "grad_norm": 3.284900665283203,
+      "learning_rate": 0.00018160784313725492,
+      "loss": 0.08410873413085937,
+      "step": 470
+    },
+    {
+      "epoch": 0.2823529411764706,
+      "grad_norm": 1.841749906539917,
+      "learning_rate": 0.00018121568627450983,
+      "loss": 0.056668424606323244,
+      "step": 480
+    },
+    {
+      "epoch": 0.28823529411764703,
+      "grad_norm": 0.03970719501376152,
+      "learning_rate": 0.0001808235294117647,
+      "loss": 0.043222978711128235,
+      "step": 490
+    },
+    {
+      "epoch": 0.29411764705882354,
+      "grad_norm": 0.17564676702022552,
+      "learning_rate": 0.0001804313725490196,
+      "loss": 0.029093235731124878,
+      "step": 500
+    },
+    {
+      "epoch": 0.3,
+      "grad_norm": 2.7894046306610107,
+      "learning_rate": 0.0001800392156862745,
+      "loss": 0.18836088180541993,
+      "step": 510
+    },
+    {
+      "epoch": 0.3058823529411765,
+      "grad_norm": 0.015540416352450848,
+      "learning_rate": 0.00017964705882352942,
+      "loss": 0.04498102068901062,
+      "step": 520
+    },
+    {
+      "epoch": 0.31176470588235294,
+      "grad_norm": 1.4599756002426147,
+      "learning_rate": 0.00017925490196078433,
+      "loss": 0.06498492360115052,
+      "step": 530
+    },
+    {
+      "epoch": 0.3176470588235294,
+      "grad_norm": 0.03984140232205391,
+      "learning_rate": 0.0001788627450980392,
+      "loss": 0.06558265089988709,
+      "step": 540
+    },
+    {
+      "epoch": 0.3235294117647059,
+      "grad_norm": 2.5549182891845703,
+      "learning_rate": 0.00017847058823529412,
+      "loss": 0.02280822992324829,
+      "step": 550
+    },
+    {
+      "epoch": 0.32941176470588235,
+      "grad_norm": 0.31564125418663025,
+      "learning_rate": 0.00017807843137254903,
+      "loss": 0.05171017646789551,
+      "step": 560
+    },
+    {
+      "epoch": 0.3352941176470588,
+      "grad_norm": 1.2470703125,
+      "learning_rate": 0.00017768627450980392,
+      "loss": 0.05321676135063171,
+      "step": 570
+    },
+    {
+      "epoch": 0.3411764705882353,
+      "grad_norm": 0.19033531844615936,
+      "learning_rate": 0.00017729411764705883,
+      "loss": 0.14463623762130737,
+      "step": 580
+    },
+    {
+      "epoch": 0.34705882352941175,
+      "grad_norm": 2.2864792346954346,
+      "learning_rate": 0.00017690196078431374,
+      "loss": 0.06809618473052978,
+      "step": 590
+    },
+    {
+      "epoch": 0.35294117647058826,
+      "grad_norm": 1.959888219833374,
+      "learning_rate": 0.00017650980392156865,
+      "loss": 0.03620869815349579,
+      "step": 600
+    },
+    {
+      "epoch": 0.3588235294117647,
+      "grad_norm": 2.4784562587738037,
+      "learning_rate": 0.00017611764705882354,
+      "loss": 0.02364923506975174,
+      "step": 610
+    },
+    {
+      "epoch": 0.36470588235294116,
+      "grad_norm": 0.02134384773671627,
+      "learning_rate": 0.00017572549019607845,
+      "loss": 0.09724082350730896,
+      "step": 620
+    },
+    {
+      "epoch": 0.37058823529411766,
+      "grad_norm": 0.06821486353874207,
+      "learning_rate": 0.00017533333333333336,
+      "loss": 0.06447332501411437,
+      "step": 630
+    },
+    {
+      "epoch": 0.3764705882352941,
+      "grad_norm": 0.6358501315116882,
+      "learning_rate": 0.00017494117647058824,
+      "loss": 0.06500619649887085,
+      "step": 640
+    },
+    {
+      "epoch": 0.38235294117647056,
+      "grad_norm": 3.4506707191467285,
+      "learning_rate": 0.00017454901960784315,
+      "loss": 0.08725794553756713,
+      "step": 650
+    },
+    {
+      "epoch": 0.38823529411764707,
+      "grad_norm": 0.22893691062927246,
+      "learning_rate": 0.00017415686274509804,
+      "loss": 0.10881921052932739,
+      "step": 660
+    },
+    {
+      "epoch": 0.3941176470588235,
+      "grad_norm": 0.10322771966457367,
+      "learning_rate": 0.00017376470588235295,
+      "loss": 0.03397703766822815,
+      "step": 670
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 2.8205652236938477,
+      "learning_rate": 0.00017337254901960783,
+      "loss": 0.027051010727882387,
+      "step": 680
+    },
+    {
+      "epoch": 0.40588235294117647,
+      "grad_norm": 0.013308421708643436,
+      "learning_rate": 0.00017298039215686274,
+      "loss": 0.04930915534496307,
+      "step": 690
+    },
+    {
+      "epoch": 0.4117647058823529,
+      "grad_norm": 3.44494891166687,
+      "learning_rate": 0.00017258823529411765,
+      "loss": 0.07341324090957642,
+      "step": 700
+    },
+    {
+      "epoch": 0.4176470588235294,
+      "grad_norm": 2.4105072021484375,
+      "learning_rate": 0.00017219607843137256,
+      "loss": 0.05853899717330933,
+      "step": 710
+    },
+    {
+      "epoch": 0.4235294117647059,
+      "grad_norm": 4.644453048706055,
+      "learning_rate": 0.00017180392156862745,
+      "loss": 0.12287142276763915,
+      "step": 720
+    },
+    {
+      "epoch": 0.4294117647058823,
+      "grad_norm": 0.38577258586883545,
+      "learning_rate": 0.00017141176470588236,
+      "loss": 0.06102405190467834,
+      "step": 730
+    },
+    {
+      "epoch": 0.43529411764705883,
+      "grad_norm": 0.07725336402654648,
+      "learning_rate": 0.00017101960784313727,
+      "loss": 0.047857701778411865,
+      "step": 740
+    },
+    {
+      "epoch": 0.4411764705882353,
+      "grad_norm": 0.06397025287151337,
+      "learning_rate": 0.00017062745098039215,
+      "loss": 0.09663112759590149,
+      "step": 750
+    },
+    {
+      "epoch": 0.4470588235294118,
+      "grad_norm": 3.3719794750213623,
+      "learning_rate": 0.00017023529411764707,
+      "loss": 0.09325426816940308,
+      "step": 760
+    },
+    {
+      "epoch": 0.45294117647058824,
+      "grad_norm": 0.46245869994163513,
+      "learning_rate": 0.00016984313725490198,
+      "loss": 0.08240602612495422,
+      "step": 770
+    },
+    {
+      "epoch": 0.4588235294117647,
+      "grad_norm": 0.032528385519981384,
+      "learning_rate": 0.0001694509803921569,
+      "loss": 0.050741398334503175,
+      "step": 780
+    },
+    {
+      "epoch": 0.4647058823529412,
+      "grad_norm": 0.24650231003761292,
+      "learning_rate": 0.00016905882352941177,
+      "loss": 0.04027060270309448,
+      "step": 790
+    },
+    {
+      "epoch": 0.47058823529411764,
+      "grad_norm": 0.12963470816612244,
+      "learning_rate": 0.00016866666666666668,
+      "loss": 0.05930449962615967,
+      "step": 800
+    },
+    {
+      "epoch": 0.4764705882352941,
+      "grad_norm": 0.4593453109264374,
+      "learning_rate": 0.0001682745098039216,
+      "loss": 0.018743452429771424,
+      "step": 810
+    },
+    {
+      "epoch": 0.4823529411764706,
+      "grad_norm": 2.126039981842041,
+      "learning_rate": 0.00016788235294117648,
+      "loss": 0.014739298820495605,
+      "step": 820
+    },
+    {
+      "epoch": 0.48823529411764705,
+      "grad_norm": 0.1071607917547226,
+      "learning_rate": 0.00016749019607843136,
+      "loss": 0.03930833339691162,
+      "step": 830
+    },
+    {
+      "epoch": 0.49411764705882355,
+      "grad_norm": 1.272904634475708,
+      "learning_rate": 0.00016709803921568627,
+      "loss": 0.01983778029680252,
+      "step": 840
+    },
+    {
+      "epoch": 0.5,
+      "grad_norm": 4.284426212310791,
+      "learning_rate": 0.00016670588235294118,
+      "loss": 0.05801863074302673,
+      "step": 850
+    },
+    {
+      "epoch": 0.5058823529411764,
+      "grad_norm": 0.21516184508800507,
+      "learning_rate": 0.0001663137254901961,
+      "loss": 0.0344509094953537,
+      "step": 860
+    },
+    {
+      "epoch": 0.5117647058823529,
+      "grad_norm": 0.05401306226849556,
+      "learning_rate": 0.00016592156862745098,
+      "loss": 0.1013040542602539,
+      "step": 870
+    },
+    {
+      "epoch": 0.5176470588235295,
+      "grad_norm": 2.2851767539978027,
+      "learning_rate": 0.0001655294117647059,
+      "loss": 0.06278380155563354,
+      "step": 880
+    },
+    {
+      "epoch": 0.5235294117647059,
+      "grad_norm": 2.387756586074829,
+      "learning_rate": 0.0001651372549019608,
+      "loss": 0.1075211524963379,
+      "step": 890
+    },
+    {
+      "epoch": 0.5294117647058824,
+      "grad_norm": 0.6298794150352478,
+      "learning_rate": 0.00016474509803921568,
+      "loss": 0.04051249623298645,
+      "step": 900
+    },
+    {
+      "epoch": 0.5352941176470588,
+      "grad_norm": 3.3592398166656494,
+      "learning_rate": 0.0001643529411764706,
+      "loss": 0.048895838856697085,
+      "step": 910
+    },
+    {
+      "epoch": 0.5411764705882353,
+      "grad_norm": 0.004421881400048733,
+      "learning_rate": 0.0001639607843137255,
+      "loss": 0.06308169960975647,
+      "step": 920
+    },
+    {
+      "epoch": 0.5470588235294118,
+      "grad_norm": 0.09252121299505234,
+      "learning_rate": 0.00016356862745098042,
+      "loss": 0.058104753494262695,
+      "step": 930
+    },
+    {
+      "epoch": 0.5529411764705883,
+      "grad_norm": 0.7001014351844788,
+      "learning_rate": 0.0001631764705882353,
+      "loss": 0.021149225533008575,
+      "step": 940
+    },
+    {
+      "epoch": 0.5588235294117647,
+      "grad_norm": 0.3170143663883209,
+      "learning_rate": 0.0001627843137254902,
+      "loss": 0.01627269834280014,
+      "step": 950
+    },
+    {
+      "epoch": 0.5647058823529412,
+      "grad_norm": 3.7102761268615723,
+      "learning_rate": 0.00016239215686274512,
+      "loss": 0.14663417339324952,
+      "step": 960
+    },
+    {
+      "epoch": 0.5705882352941176,
+      "grad_norm": 0.006750724744051695,
+      "learning_rate": 0.000162,
+      "loss": 0.10339630842208862,
+      "step": 970
+    },
+    {
+      "epoch": 0.5764705882352941,
+      "grad_norm": 1.808908462524414,
+      "learning_rate": 0.0001616078431372549,
+      "loss": 0.026013880968093872,
+      "step": 980
+    },
+    {
+      "epoch": 0.5823529411764706,
+      "grad_norm": 0.4191270172595978,
+      "learning_rate": 0.0001612156862745098,
+      "loss": 0.039347043633461,
+      "step": 990
+    },
+    {
+      "epoch": 0.5882352941176471,
+      "grad_norm": 0.3325525224208832,
+      "learning_rate": 0.0001608235294117647,
+      "loss": 0.059603160619735716,
+      "step": 1000
+    },
+    {
+      "epoch": 0.5941176470588235,
+      "grad_norm": 2.7802844047546387,
+      "learning_rate": 0.0001604313725490196,
+      "loss": 0.09034895300865173,
+      "step": 1010
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 0.04968420788645744,
+      "learning_rate": 0.0001600392156862745,
+      "loss": 0.05755581855773926,
+      "step": 1020
+    },
+    {
+      "epoch": 0.6058823529411764,
+      "grad_norm": 0.00875788927078247,
+      "learning_rate": 0.00015964705882352942,
+      "loss": 0.08298594951629638,
+      "step": 1030
+    },
+    {
+      "epoch": 0.611764705882353,
+      "grad_norm": 0.38597434759140015,
+      "learning_rate": 0.00015925490196078433,
+      "loss": 0.009630383551120758,
+      "step": 1040
+    },
+    {
+      "epoch": 0.6176470588235294,
+      "grad_norm": 0.42396023869514465,
+      "learning_rate": 0.00015886274509803921,
+      "loss": 0.05784105658531189,
+      "step": 1050
+    },
+    {
+      "epoch": 0.6235294117647059,
+      "grad_norm": 3.5692520141601562,
+      "learning_rate": 0.00015847058823529413,
+      "loss": 0.1575460195541382,
+      "step": 1060
+    },
+    {
+      "epoch": 0.6294117647058823,
+      "grad_norm": 0.12411337345838547,
+      "learning_rate": 0.00015807843137254904,
+      "loss": 0.052008318901062014,
+      "step": 1070
+    },
+    {
+      "epoch": 0.6352941176470588,
+      "grad_norm": 0.12469016760587692,
+      "learning_rate": 0.00015768627450980392,
+      "loss": 0.06983086466789246,
+      "step": 1080
+    },
+    {
+      "epoch": 0.6411764705882353,
+      "grad_norm": 0.0010811894899234176,
+      "learning_rate": 0.00015729411764705883,
+      "loss": 0.009204772859811782,
+      "step": 1090
+    },
+    {
+      "epoch": 0.6470588235294118,
+      "grad_norm": 0.7795768976211548,
+      "learning_rate": 0.00015690196078431374,
+      "loss": 0.07802544236183166,
+      "step": 1100
+    },
+    {
+      "epoch": 0.6529411764705882,
+      "grad_norm": 3.0475451946258545,
+      "learning_rate": 0.00015650980392156865,
+      "loss": 0.06048353910446167,
+      "step": 1110
+    },
+    {
+      "epoch": 0.6588235294117647,
+      "grad_norm": 0.054235588759183884,
+      "learning_rate": 0.00015611764705882354,
+      "loss": 0.007759802043437958,
+      "step": 1120
+    },
+    {
+      "epoch": 0.6647058823529411,
+      "grad_norm": 0.022670522332191467,
+      "learning_rate": 0.00015572549019607845,
+      "loss": 0.121265709400177,
+      "step": 1130
+    },
+    {
+      "epoch": 0.6705882352941176,
+      "grad_norm": 4.404564380645752,
+      "learning_rate": 0.00015533333333333333,
+      "loss": 0.11734864711761475,
+      "step": 1140
+    },
+    {
+      "epoch": 0.6764705882352942,
+      "grad_norm": 0.04260774329304695,
+      "learning_rate": 0.00015494117647058824,
+      "loss": 0.04804904460906982,
+      "step": 1150
+    },
+    {
+      "epoch": 0.6823529411764706,
+      "grad_norm": 0.37849345803260803,
+      "learning_rate": 0.00015454901960784313,
+      "loss": 0.08949239253997802,
+      "step": 1160
+    },
+    {
+      "epoch": 0.6882352941176471,
+      "grad_norm": 0.07910435646772385,
+      "learning_rate": 0.00015415686274509804,
+      "loss": 0.11236823797225952,
+      "step": 1170
+    },
+    {
+      "epoch": 0.6941176470588235,
+      "grad_norm": 0.020735427737236023,
+      "learning_rate": 0.00015376470588235295,
+      "loss": 0.020005679130554198,
+      "step": 1180
+    },
+    {
+      "epoch": 0.7,
+      "grad_norm": 0.6041317582130432,
+      "learning_rate": 0.00015337254901960786,
+      "loss": 0.047528034448623656,
+      "step": 1190
+    },
+    {
+      "epoch": 0.7058823529411765,
+      "grad_norm": 0.07179152965545654,
+      "learning_rate": 0.00015298039215686274,
+      "loss": 0.03959411382675171,
+      "step": 1200
+    },
+    {
+      "epoch": 0.711764705882353,
+      "grad_norm": 0.009498389437794685,
+      "learning_rate": 0.00015258823529411766,
+      "loss": 0.003311210125684738,
+      "step": 1210
+    },
+    {
+      "epoch": 0.7176470588235294,
+      "grad_norm": 0.433115154504776,
+      "learning_rate": 0.00015219607843137257,
+      "loss": 0.00858706161379814,
+      "step": 1220
+    },
+    {
+      "epoch": 0.7235294117647059,
+      "grad_norm": 0.008886504918336868,
+      "learning_rate": 0.00015180392156862745,
+      "loss": 0.06958861947059632,
+      "step": 1230
+    },
+    {
+      "epoch": 0.7294117647058823,
+      "grad_norm": 0.008790708146989346,
+      "learning_rate": 0.00015141176470588236,
+      "loss": 0.008029435575008393,
+      "step": 1240
+    },
+    {
+      "epoch": 0.7352941176470589,
+      "grad_norm": 0.005836977623403072,
+      "learning_rate": 0.00015101960784313727,
+      "loss": 0.05358666181564331,
+      "step": 1250
+    },
+    {
+      "epoch": 0.7411764705882353,
+      "grad_norm": 0.14504218101501465,
+      "learning_rate": 0.00015062745098039218,
+      "loss": 0.049011257290840146,
+      "step": 1260
+    },
+    {
+      "epoch": 0.7470588235294118,
+      "grad_norm": 0.2803121507167816,
+      "learning_rate": 0.00015023529411764707,
+      "loss": 0.006191087141633034,
+      "step": 1270
+    },
+    {
+      "epoch": 0.7529411764705882,
+      "grad_norm": 3.1994972229003906,
+      "learning_rate": 0.00014984313725490198,
+      "loss": 0.10595870018005371,
+      "step": 1280
+    },
+    {
+      "epoch": 0.7588235294117647,
+      "grad_norm": 0.00779637461528182,
+      "learning_rate": 0.0001494509803921569,
+      "loss": 0.07793840169906616,
+      "step": 1290
+    },
+    {
+      "epoch": 0.7647058823529411,
+      "grad_norm": 0.04689478874206543,
+      "learning_rate": 0.00014905882352941177,
+      "loss": 0.08080050349235535,
+      "step": 1300
+    },
+    {
+      "epoch": 0.7705882352941177,
+      "grad_norm": 0.9595724940299988,
+      "learning_rate": 0.00014866666666666666,
+      "loss": 0.11690077781677247,
+      "step": 1310
+    },
+    {
+      "epoch": 0.7764705882352941,
+      "grad_norm": 4.141428470611572,
+      "learning_rate": 0.00014827450980392157,
+      "loss": 0.05245780348777771,
+      "step": 1320
+    },
+    {
+      "epoch": 0.7823529411764706,
+      "grad_norm": 0.7606799602508545,
+      "learning_rate": 0.00014788235294117648,
+      "loss": 0.039501425623893735,
+      "step": 1330
+    },
+    {
+      "epoch": 0.788235294117647,
+      "grad_norm": 2.8373472690582275,
+      "learning_rate": 0.00014749019607843136,
+      "loss": 0.08525887727737427,
+      "step": 1340
+    },
+    {
+      "epoch": 0.7941176470588235,
+      "grad_norm": 0.20202526450157166,
+      "learning_rate": 0.00014709803921568627,
+      "loss": 0.03491775095462799,
+      "step": 1350
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.00345594366081059,
+      "learning_rate": 0.00014670588235294119,
+      "loss": 0.03784347176551819,
+      "step": 1360
+    },
+    {
+      "epoch": 0.8058823529411765,
+      "grad_norm": 3.3252758979797363,
+      "learning_rate": 0.0001463137254901961,
+      "loss": 0.08734545707702637,
+      "step": 1370
+    },
+    {
+      "epoch": 0.8117647058823529,
+      "grad_norm": 1.1744837760925293,
+      "learning_rate": 0.00014592156862745098,
+      "loss": 0.060222625732421875,
+      "step": 1380
+    },
+    {
+      "epoch": 0.8176470588235294,
+      "grad_norm": 0.012418321333825588,
+      "learning_rate": 0.0001455294117647059,
+      "loss": 0.034251919388771056,
+      "step": 1390
+    },
+    {
+      "epoch": 0.8235294117647058,
+      "grad_norm": 0.18622326850891113,
+      "learning_rate": 0.0001451372549019608,
+      "loss": 0.02356344759464264,
+      "step": 1400
+    },
+    {
+      "epoch": 0.8294117647058824,
+      "grad_norm": 3.9867045879364014,
+      "learning_rate": 0.00014474509803921569,
+      "loss": 0.07036476135253907,
+      "step": 1410
+    },
+    {
+      "epoch": 0.8352941176470589,
+      "grad_norm": 3.0563607215881348,
+      "learning_rate": 0.0001443529411764706,
+      "loss": 0.037895727157592776,
+      "step": 1420
+    },
+    {
+      "epoch": 0.8411764705882353,
+      "grad_norm": 0.13856367766857147,
+      "learning_rate": 0.0001439607843137255,
+      "loss": 0.12252027988433838,
+      "step": 1430
+    },
+    {
+      "epoch": 0.8470588235294118,
+      "grad_norm": 6.263605117797852,
+      "learning_rate": 0.00014356862745098042,
+      "loss": 0.04287775456905365,
+      "step": 1440
+    },
+    {
+      "epoch": 0.8529411764705882,
+      "grad_norm": 0.6084681749343872,
+      "learning_rate": 0.0001431764705882353,
+      "loss": 0.050480049848556516,
+      "step": 1450
+    },
+    {
+      "epoch": 0.8588235294117647,
+      "grad_norm": 0.13102763891220093,
+      "learning_rate": 0.0001427843137254902,
+      "loss": 0.027205181121826173,
+      "step": 1460
+    },
+    {
+      "epoch": 0.8647058823529412,
+      "grad_norm": 0.0334654301404953,
+      "learning_rate": 0.0001423921568627451,
+      "loss": 0.07116500139236451,
+      "step": 1470
+    },
+    {
+      "epoch": 0.8705882352941177,
+      "grad_norm": 5.854252815246582,
+      "learning_rate": 0.000142,
+      "loss": 0.10154651403427124,
+      "step": 1480
+    },
+    {
+      "epoch": 0.8764705882352941,
+      "grad_norm": 0.04299909248948097,
+      "learning_rate": 0.0001416078431372549,
+      "loss": 0.06952143907546997,
+      "step": 1490
+    },
+    {
+      "epoch": 0.8823529411764706,
+      "grad_norm": 0.03385253995656967,
+      "learning_rate": 0.0001412156862745098,
+      "loss": 0.042281192541122434,
+      "step": 1500
+    },
+    {
+      "epoch": 0.888235294117647,
+      "grad_norm": 3.0850558280944824,
+      "learning_rate": 0.00014082352941176472,
+      "loss": 0.04330833852291107,
+      "step": 1510
+    },
+    {
+      "epoch": 0.8941176470588236,
+      "grad_norm": 1.0124292373657227,
+      "learning_rate": 0.00014043137254901963,
+      "loss": 0.013002160191535949,
+      "step": 1520
+    },
+    {
+      "epoch": 0.9,
+      "grad_norm": 0.4087558388710022,
+      "learning_rate": 0.0001400392156862745,
+      "loss": 0.08435083031654358,
+      "step": 1530
+    },
+    {
+      "epoch": 0.9058823529411765,
+      "grad_norm": 1.3371378183364868,
+      "learning_rate": 0.00013964705882352942,
+      "loss": 0.04774133563041687,
+      "step": 1540
+    },
+    {
+      "epoch": 0.9117647058823529,
+      "grad_norm": 3.8198187351226807,
+      "learning_rate": 0.00013925490196078433,
+      "loss": 0.05395355820655823,
+      "step": 1550
+    },
+    {
+      "epoch": 0.9176470588235294,
+      "grad_norm": 0.07719609141349792,
+      "learning_rate": 0.00013886274509803922,
+      "loss": 0.05478907227516174,
+      "step": 1560
+    },
+    {
+      "epoch": 0.9235294117647059,
+      "grad_norm": 0.14217644929885864,
+      "learning_rate": 0.00013847058823529413,
+      "loss": 0.037067908048629764,
+      "step": 1570
+    },
+    {
+      "epoch": 0.9294117647058824,
+      "grad_norm": 0.0115964999422431,
+      "learning_rate": 0.00013807843137254904,
+      "loss": 0.06596972942352294,
+      "step": 1580
+    },
+    {
+      "epoch": 0.9352941176470588,
+      "grad_norm": 0.11416631937026978,
+      "learning_rate": 0.00013768627450980395,
+      "loss": 0.06003996729850769,
+      "step": 1590
+    },
+    {
+      "epoch": 0.9411764705882353,
+      "grad_norm": 0.8297443985939026,
+      "learning_rate": 0.00013729411764705883,
+      "loss": 0.012542881071567535,
+      "step": 1600
+    },
+    {
+      "epoch": 0.9470588235294117,
+      "grad_norm": 2.7102794647216797,
+      "learning_rate": 0.00013690196078431374,
+      "loss": 0.04482048749923706,
+      "step": 1610
+    },
+    {
+      "epoch": 0.9529411764705882,
+      "grad_norm": 0.04676695913076401,
+      "learning_rate": 0.00013650980392156863,
+      "loss": 0.04008999168872833,
+      "step": 1620
+    },
+    {
+      "epoch": 0.9588235294117647,
+      "grad_norm": 0.024560345336794853,
+      "learning_rate": 0.00013611764705882354,
+      "loss": 0.004030698910355568,
+      "step": 1630
+    },
+    {
+      "epoch": 0.9647058823529412,
+      "grad_norm": 0.004613540135324001,
+      "learning_rate": 0.00013572549019607842,
+      "loss": 0.016170585155487062,
+      "step": 1640
+    },
+    {
+      "epoch": 0.9705882352941176,
+      "grad_norm": 0.07655692845582962,
+      "learning_rate": 0.00013533333333333333,
+      "loss": 0.002394220232963562,
+      "step": 1650
+    },
+    {
+      "epoch": 0.9764705882352941,
+      "grad_norm": 1.5127253532409668,
+      "learning_rate": 0.00013494117647058825,
+      "loss": 0.08604814410209656,
+      "step": 1660
+    },
+    {
+      "epoch": 0.9823529411764705,
+      "grad_norm": 4.647606372833252,
+      "learning_rate": 0.00013454901960784313,
+      "loss": 0.05253278613090515,
+      "step": 1670
+    },
+    {
+      "epoch": 0.9882352941176471,
+      "grad_norm": 0.0031903591006994247,
+      "learning_rate": 0.00013415686274509804,
+      "loss": 0.05996617674827576,
+      "step": 1680
+    },
+    {
+      "epoch": 0.9941176470588236,
+      "grad_norm": 3.4767584800720215,
+      "learning_rate": 0.00013376470588235295,
+      "loss": 0.09587143063545227,
+      "step": 1690
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.6253292560577393,
+      "learning_rate": 0.00013337254901960786,
+      "loss": 0.004908151552081108,
+      "step": 1700
+    },
+    {
+      "epoch": 1.0,
+      "eval_accuracy": 0.9833333333333333,
+      "eval_loss": 0.052190084010362625,
+      "eval_runtime": 107.7754,
+      "eval_samples_per_second": 44.537,
+      "eval_steps_per_second": 2.784,
+      "step": 1700
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 5100,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 144276430488960.0,
+  "train_batch_size": 16,
+  "trial_name": null,
+  "trial_params": null
+}

training/training-checkpoints/checkpoint-1700/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:404392ce274606d42167ab282bdb788e763b38ede0704b0120ec45be3aba6567
+size 5201

training/training-checkpoints/checkpoint-3400/README.md

@@ -0,0 +1,202 @@
+---
+base_model: distilbert-base-uncased
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

training/training-checkpoints/checkpoint-3400/adapter_config.json

@@ -0,0 +1,32 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "distilbert-base-uncased",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_lin",
+    "q_lin"
+  ],
+  "task_type": "SEQ_CLS",
+  "use_dora": false,
+  "use_rslora": false
+}

training/training-checkpoints/checkpoint-3400/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3efc79c7eb5833b34e822345fb70ee2173e80b4828996657e6db01a3f2ec01e2
+size 2962192

training/training-checkpoints/checkpoint-3400/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c9e6c856440e62b525eebb5d0a7dde53f1558e65e21804d25911bbd40980f0c
+size 5940315

training/training-checkpoints/checkpoint-3400/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e429eb1781646b5274a99b5e36c82dbc61a3865d7755b8859fa26a3972944dad
+size 14455

training/training-checkpoints/checkpoint-3400/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:184a9ded095bbedd783e752989df8b4dad755674acfc75d117a431c0266daf32
+size 1465

training/training-checkpoints/checkpoint-3400/tokenizer_config.json

@@ -0,0 +1,15 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

training/training-checkpoints/checkpoint-3400/trainer_state.json

@@ -0,0 +1,2432 @@
+{
+  "best_global_step": 1700,
+  "best_metric": 0.9833333333333333,
+  "best_model_checkpoint": "./training-checkpoints\\checkpoint-1700",
+  "epoch": 2.0,
+  "eval_steps": 500,
+  "global_step": 3400,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0058823529411764705,
+      "grad_norm": 1.510934829711914,
+      "learning_rate": 0.00019964705882352944,
+      "loss": 0.6532491683959961,
+      "step": 10
+    },
+    {
+      "epoch": 0.011764705882352941,
+      "grad_norm": 2.264005661010742,
+      "learning_rate": 0.0001992549019607843,
+      "loss": 0.49631505012512206,
+      "step": 20
+    },
+    {
+      "epoch": 0.01764705882352941,
+      "grad_norm": 1.0282903909683228,
+      "learning_rate": 0.0001988627450980392,
+      "loss": 0.29676687717437744,
+      "step": 30
+    },
+    {
+      "epoch": 0.023529411764705882,
+      "grad_norm": 1.0084155797958374,
+      "learning_rate": 0.00019847058823529412,
+      "loss": 0.1594167470932007,
+      "step": 40
+    },
+    {
+      "epoch": 0.029411764705882353,
+      "grad_norm": 0.23450879752635956,
+      "learning_rate": 0.00019807843137254903,
+      "loss": 0.09263747334480285,
+      "step": 50
+    },
+    {
+      "epoch": 0.03529411764705882,
+      "grad_norm": 0.44396597146987915,
+      "learning_rate": 0.00019768627450980392,
+      "loss": 0.18005683422088622,
+      "step": 60
+    },
+    {
+      "epoch": 0.041176470588235294,
+      "grad_norm": 0.1429290622472763,
+      "learning_rate": 0.00019729411764705883,
+      "loss": 0.08807324171066284,
+      "step": 70
+    },
+    {
+      "epoch": 0.047058823529411764,
+      "grad_norm": 0.847214937210083,
+      "learning_rate": 0.00019690196078431374,
+      "loss": 0.058199501037597655,
+      "step": 80
+    },
+    {
+      "epoch": 0.052941176470588235,
+      "grad_norm": 3.822068929672241,
+      "learning_rate": 0.00019650980392156862,
+      "loss": 0.1674056887626648,
+      "step": 90
+    },
+    {
+      "epoch": 0.058823529411764705,
+      "grad_norm": 1.4017611742019653,
+      "learning_rate": 0.00019611764705882353,
+      "loss": 0.11085774898529052,
+      "step": 100
+    },
+    {
+      "epoch": 0.06470588235294118,
+      "grad_norm": 1.6271415948867798,
+      "learning_rate": 0.00019572549019607844,
+      "loss": 0.10848402976989746,
+      "step": 110
+    },
+    {
+      "epoch": 0.07058823529411765,
+      "grad_norm": 0.17168374359607697,
+      "learning_rate": 0.00019533333333333336,
+      "loss": 0.08450813293457031,
+      "step": 120
+    },
+    {
+      "epoch": 0.07647058823529412,
+      "grad_norm": 2.110332489013672,
+      "learning_rate": 0.00019494117647058824,
+      "loss": 0.08588783144950866,
+      "step": 130
+    },
+    {
+      "epoch": 0.08235294117647059,
+      "grad_norm": 1.973347783088684,
+      "learning_rate": 0.00019454901960784315,
+      "loss": 0.07138832211494446,
+      "step": 140
+    },
+    {
+      "epoch": 0.08823529411764706,
+      "grad_norm": 2.4691383838653564,
+      "learning_rate": 0.00019415686274509806,
+      "loss": 0.1480124831199646,
+      "step": 150
+    },
+    {
+      "epoch": 0.09411764705882353,
+      "grad_norm": 0.8067235946655273,
+      "learning_rate": 0.00019376470588235295,
+      "loss": 0.03993956744670868,
+      "step": 160
+    },
+    {
+      "epoch": 0.1,
+      "grad_norm": 2.022461175918579,
+      "learning_rate": 0.00019337254901960786,
+      "loss": 0.06878547668457032,
+      "step": 170
+    },
+    {
+      "epoch": 0.10588235294117647,
+      "grad_norm": 1.4254077672958374,
+      "learning_rate": 0.00019298039215686274,
+      "loss": 0.12785714864730835,
+      "step": 180
+    },
+    {
+      "epoch": 0.11176470588235295,
+      "grad_norm": 3.5772814750671387,
+      "learning_rate": 0.00019258823529411765,
+      "loss": 0.07902452945709229,
+      "step": 190
+    },
+    {
+      "epoch": 0.11764705882352941,
+      "grad_norm": 0.06748726963996887,
+      "learning_rate": 0.00019219607843137256,
+      "loss": 0.06560993194580078,
+      "step": 200
+    },
+    {
+      "epoch": 0.12352941176470589,
+      "grad_norm": 5.759597301483154,
+      "learning_rate": 0.00019180392156862745,
+      "loss": 0.16235246658325195,
+      "step": 210
+    },
+    {
+      "epoch": 0.12941176470588237,
+      "grad_norm": 3.6009647846221924,
+      "learning_rate": 0.00019141176470588236,
+      "loss": 0.06680867075920105,
+      "step": 220
+    },
+    {
+      "epoch": 0.13529411764705881,
+      "grad_norm": 2.608488082885742,
+      "learning_rate": 0.00019101960784313727,
+      "loss": 0.08542757034301758,
+      "step": 230
+    },
+    {
+      "epoch": 0.1411764705882353,
+      "grad_norm": 1.4347050189971924,
+      "learning_rate": 0.00019062745098039215,
+      "loss": 0.06062295436859131,
+      "step": 240
+    },
+    {
+      "epoch": 0.14705882352941177,
+      "grad_norm": 3.4521889686584473,
+      "learning_rate": 0.00019023529411764706,
+      "loss": 0.11894316673278808,
+      "step": 250
+    },
+    {
+      "epoch": 0.15294117647058825,
+      "grad_norm": 2.5669424533843994,
+      "learning_rate": 0.00018984313725490197,
+      "loss": 0.04931341111660004,
+      "step": 260
+    },
+    {
+      "epoch": 0.1588235294117647,
+      "grad_norm": 0.5097957253456116,
+      "learning_rate": 0.00018945098039215689,
+      "loss": 0.0895132601261139,
+      "step": 270
+    },
+    {
+      "epoch": 0.16470588235294117,
+      "grad_norm": 0.1470881551504135,
+      "learning_rate": 0.00018905882352941177,
+      "loss": 0.07752076387405396,
+      "step": 280
+    },
+    {
+      "epoch": 0.17058823529411765,
+      "grad_norm": 0.12387822568416595,
+      "learning_rate": 0.00018866666666666668,
+      "loss": 0.0910811185836792,
+      "step": 290
+    },
+    {
+      "epoch": 0.17647058823529413,
+      "grad_norm": 0.1579340398311615,
+      "learning_rate": 0.0001882745098039216,
+      "loss": 0.024501879513263703,
+      "step": 300
+    },
+    {
+      "epoch": 0.18235294117647058,
+      "grad_norm": 0.19159211218357086,
+      "learning_rate": 0.00018788235294117648,
+      "loss": 0.025437775254249572,
+      "step": 310
+    },
+    {
+      "epoch": 0.18823529411764706,
+      "grad_norm": 0.3070846199989319,
+      "learning_rate": 0.0001874901960784314,
+      "loss": 0.0452291190624237,
+      "step": 320
+    },
+    {
+      "epoch": 0.19411764705882353,
+      "grad_norm": 1.647754192352295,
+      "learning_rate": 0.0001870980392156863,
+      "loss": 0.07508809566497802,
+      "step": 330
+    },
+    {
+      "epoch": 0.2,
+      "grad_norm": 0.2747861444950104,
+      "learning_rate": 0.00018670588235294118,
+      "loss": 0.08880412578582764,
+      "step": 340
+    },
+    {
+      "epoch": 0.20588235294117646,
+      "grad_norm": 0.20420950651168823,
+      "learning_rate": 0.00018631372549019607,
+      "loss": 0.047925958037376405,
+      "step": 350
+    },
+    {
+      "epoch": 0.21176470588235294,
+      "grad_norm": 0.5672637820243835,
+      "learning_rate": 0.00018592156862745098,
+      "loss": 0.04176274836063385,
+      "step": 360
+    },
+    {
+      "epoch": 0.21764705882352942,
+      "grad_norm": 1.5938951969146729,
+      "learning_rate": 0.0001855294117647059,
+      "loss": 0.06371157765388488,
+      "step": 370
+    },
+    {
+      "epoch": 0.2235294117647059,
+      "grad_norm": 0.019027238711714745,
+      "learning_rate": 0.0001851372549019608,
+      "loss": 0.05675685405731201,
+      "step": 380
+    },
+    {
+      "epoch": 0.22941176470588234,
+      "grad_norm": 1.8165510892868042,
+      "learning_rate": 0.00018474509803921568,
+      "loss": 0.10704467296600342,
+      "step": 390
+    },
+    {
+      "epoch": 0.23529411764705882,
+      "grad_norm": 0.3322693109512329,
+      "learning_rate": 0.0001843529411764706,
+      "loss": 0.08637534379959107,
+      "step": 400
+    },
+    {
+      "epoch": 0.2411764705882353,
+      "grad_norm": 0.06156998872756958,
+      "learning_rate": 0.0001839607843137255,
+      "loss": 0.17954374551773072,
+      "step": 410
+    },
+    {
+      "epoch": 0.24705882352941178,
+      "grad_norm": 3.327615737915039,
+      "learning_rate": 0.0001835686274509804,
+      "loss": 0.1696428656578064,
+      "step": 420
+    },
+    {
+      "epoch": 0.2529411764705882,
+      "grad_norm": 2.1997804641723633,
+      "learning_rate": 0.0001831764705882353,
+      "loss": 0.11364388465881348,
+      "step": 430
+    },
+    {
+      "epoch": 0.25882352941176473,
+      "grad_norm": 2.8368852138519287,
+      "learning_rate": 0.0001827843137254902,
+      "loss": 0.03553434014320374,
+      "step": 440
+    },
+    {
+      "epoch": 0.2647058823529412,
+      "grad_norm": 0.2659618556499481,
+      "learning_rate": 0.00018239215686274512,
+      "loss": 0.03715018928050995,
+      "step": 450
+    },
+    {
+      "epoch": 0.27058823529411763,
+      "grad_norm": 0.04204031080007553,
+      "learning_rate": 0.000182,
+      "loss": 0.13120421171188354,
+      "step": 460
+    },
+    {
+      "epoch": 0.27647058823529413,
+      "grad_norm": 3.284900665283203,
+      "learning_rate": 0.00018160784313725492,
+      "loss": 0.08410873413085937,
+      "step": 470
+    },
+    {
+      "epoch": 0.2823529411764706,
+      "grad_norm": 1.841749906539917,
+      "learning_rate": 0.00018121568627450983,
+      "loss": 0.056668424606323244,
+      "step": 480
+    },
+    {
+      "epoch": 0.28823529411764703,
+      "grad_norm": 0.03970719501376152,
+      "learning_rate": 0.0001808235294117647,
+      "loss": 0.043222978711128235,
+      "step": 490
+    },
+    {
+      "epoch": 0.29411764705882354,
+      "grad_norm": 0.17564676702022552,
+      "learning_rate": 0.0001804313725490196,
+      "loss": 0.029093235731124878,
+      "step": 500
+    },
+    {
+      "epoch": 0.3,
+      "grad_norm": 2.7894046306610107,
+      "learning_rate": 0.0001800392156862745,
+      "loss": 0.18836088180541993,
+      "step": 510
+    },
+    {
+      "epoch": 0.3058823529411765,
+      "grad_norm": 0.015540416352450848,
+      "learning_rate": 0.00017964705882352942,
+      "loss": 0.04498102068901062,
+      "step": 520
+    },
+    {
+      "epoch": 0.31176470588235294,
+      "grad_norm": 1.4599756002426147,
+      "learning_rate": 0.00017925490196078433,
+      "loss": 0.06498492360115052,
+      "step": 530
+    },
+    {
+      "epoch": 0.3176470588235294,
+      "grad_norm": 0.03984140232205391,
+      "learning_rate": 0.0001788627450980392,
+      "loss": 0.06558265089988709,
+      "step": 540
+    },
+    {
+      "epoch": 0.3235294117647059,
+      "grad_norm": 2.5549182891845703,
+      "learning_rate": 0.00017847058823529412,
+      "loss": 0.02280822992324829,
+      "step": 550
+    },
+    {
+      "epoch": 0.32941176470588235,
+      "grad_norm": 0.31564125418663025,
+      "learning_rate": 0.00017807843137254903,
+      "loss": 0.05171017646789551,
+      "step": 560
+    },
+    {
+      "epoch": 0.3352941176470588,
+      "grad_norm": 1.2470703125,
+      "learning_rate": 0.00017768627450980392,
+      "loss": 0.05321676135063171,
+      "step": 570
+    },
+    {
+      "epoch": 0.3411764705882353,
+      "grad_norm": 0.19033531844615936,
+      "learning_rate": 0.00017729411764705883,
+      "loss": 0.14463623762130737,
+      "step": 580
+    },
+    {
+      "epoch": 0.34705882352941175,
+      "grad_norm": 2.2864792346954346,
+      "learning_rate": 0.00017690196078431374,
+      "loss": 0.06809618473052978,
+      "step": 590
+    },
+    {
+      "epoch": 0.35294117647058826,
+      "grad_norm": 1.959888219833374,
+      "learning_rate": 0.00017650980392156865,
+      "loss": 0.03620869815349579,
+      "step": 600
+    },
+    {
+      "epoch": 0.3588235294117647,
+      "grad_norm": 2.4784562587738037,
+      "learning_rate": 0.00017611764705882354,
+      "loss": 0.02364923506975174,
+      "step": 610
+    },
+    {
+      "epoch": 0.36470588235294116,
+      "grad_norm": 0.02134384773671627,
+      "learning_rate": 0.00017572549019607845,
+      "loss": 0.09724082350730896,
+      "step": 620
+    },
+    {
+      "epoch": 0.37058823529411766,
+      "grad_norm": 0.06821486353874207,
+      "learning_rate": 0.00017533333333333336,
+      "loss": 0.06447332501411437,
+      "step": 630
+    },
+    {
+      "epoch": 0.3764705882352941,
+      "grad_norm": 0.6358501315116882,
+      "learning_rate": 0.00017494117647058824,
+      "loss": 0.06500619649887085,
+      "step": 640
+    },
+    {
+      "epoch": 0.38235294117647056,
+      "grad_norm": 3.4506707191467285,
+      "learning_rate": 0.00017454901960784315,
+      "loss": 0.08725794553756713,
+      "step": 650
+    },
+    {
+      "epoch": 0.38823529411764707,
+      "grad_norm": 0.22893691062927246,
+      "learning_rate": 0.00017415686274509804,
+      "loss": 0.10881921052932739,
+      "step": 660
+    },
+    {
+      "epoch": 0.3941176470588235,
+      "grad_norm": 0.10322771966457367,
+      "learning_rate": 0.00017376470588235295,
+      "loss": 0.03397703766822815,
+      "step": 670
+    },
+    {
+      "epoch": 0.4,
+      "grad_norm": 2.8205652236938477,
+      "learning_rate": 0.00017337254901960783,
+      "loss": 0.027051010727882387,
+      "step": 680
+    },
+    {
+      "epoch": 0.40588235294117647,
+      "grad_norm": 0.013308421708643436,
+      "learning_rate": 0.00017298039215686274,
+      "loss": 0.04930915534496307,
+      "step": 690
+    },
+    {
+      "epoch": 0.4117647058823529,
+      "grad_norm": 3.44494891166687,
+      "learning_rate": 0.00017258823529411765,
+      "loss": 0.07341324090957642,
+      "step": 700
+    },
+    {
+      "epoch": 0.4176470588235294,
+      "grad_norm": 2.4105072021484375,
+      "learning_rate": 0.00017219607843137256,
+      "loss": 0.05853899717330933,
+      "step": 710
+    },
+    {
+      "epoch": 0.4235294117647059,
+      "grad_norm": 4.644453048706055,
+      "learning_rate": 0.00017180392156862745,
+      "loss": 0.12287142276763915,
+      "step": 720
+    },
+    {
+      "epoch": 0.4294117647058823,
+      "grad_norm": 0.38577258586883545,
+      "learning_rate": 0.00017141176470588236,
+      "loss": 0.06102405190467834,
+      "step": 730
+    },
+    {
+      "epoch": 0.43529411764705883,
+      "grad_norm": 0.07725336402654648,
+      "learning_rate": 0.00017101960784313727,
+      "loss": 0.047857701778411865,
+      "step": 740
+    },
+    {
+      "epoch": 0.4411764705882353,
+      "grad_norm": 0.06397025287151337,
+      "learning_rate": 0.00017062745098039215,
+      "loss": 0.09663112759590149,
+      "step": 750
+    },
+    {
+      "epoch": 0.4470588235294118,
+      "grad_norm": 3.3719794750213623,
+      "learning_rate": 0.00017023529411764707,
+      "loss": 0.09325426816940308,
+      "step": 760
+    },
+    {
+      "epoch": 0.45294117647058824,
+      "grad_norm": 0.46245869994163513,
+      "learning_rate": 0.00016984313725490198,
+      "loss": 0.08240602612495422,
+      "step": 770
+    },
+    {
+      "epoch": 0.4588235294117647,
+      "grad_norm": 0.032528385519981384,
+      "learning_rate": 0.0001694509803921569,
+      "loss": 0.050741398334503175,
+      "step": 780
+    },
+    {
+      "epoch": 0.4647058823529412,
+      "grad_norm": 0.24650231003761292,
+      "learning_rate": 0.00016905882352941177,
+      "loss": 0.04027060270309448,
+      "step": 790
+    },
+    {
+      "epoch": 0.47058823529411764,
+      "grad_norm": 0.12963470816612244,
+      "learning_rate": 0.00016866666666666668,
+      "loss": 0.05930449962615967,
+      "step": 800
+    },
+    {
+      "epoch": 0.4764705882352941,
+      "grad_norm": 0.4593453109264374,
+      "learning_rate": 0.0001682745098039216,
+      "loss": 0.018743452429771424,
+      "step": 810
+    },
+    {
+      "epoch": 0.4823529411764706,
+      "grad_norm": 2.126039981842041,
+      "learning_rate": 0.00016788235294117648,
+      "loss": 0.014739298820495605,
+      "step": 820
+    },
+    {
+      "epoch": 0.48823529411764705,
+      "grad_norm": 0.1071607917547226,
+      "learning_rate": 0.00016749019607843136,
+      "loss": 0.03930833339691162,
+      "step": 830
+    },
+    {
+      "epoch": 0.49411764705882355,
+      "grad_norm": 1.272904634475708,
+      "learning_rate": 0.00016709803921568627,
+      "loss": 0.01983778029680252,
+      "step": 840
+    },
+    {
+      "epoch": 0.5,
+      "grad_norm": 4.284426212310791,
+      "learning_rate": 0.00016670588235294118,
+      "loss": 0.05801863074302673,
+      "step": 850
+    },
+    {
+      "epoch": 0.5058823529411764,
+      "grad_norm": 0.21516184508800507,
+      "learning_rate": 0.0001663137254901961,
+      "loss": 0.0344509094953537,
+      "step": 860
+    },
+    {
+      "epoch": 0.5117647058823529,
+      "grad_norm": 0.05401306226849556,
+      "learning_rate": 0.00016592156862745098,
+      "loss": 0.1013040542602539,
+      "step": 870
+    },
+    {
+      "epoch": 0.5176470588235295,
+      "grad_norm": 2.2851767539978027,
+      "learning_rate": 0.0001655294117647059,
+      "loss": 0.06278380155563354,
+      "step": 880
+    },
+    {
+      "epoch": 0.5235294117647059,
+      "grad_norm": 2.387756586074829,
+      "learning_rate": 0.0001651372549019608,
+      "loss": 0.1075211524963379,
+      "step": 890
+    },
+    {
+      "epoch": 0.5294117647058824,
+      "grad_norm": 0.6298794150352478,
+      "learning_rate": 0.00016474509803921568,
+      "loss": 0.04051249623298645,
+      "step": 900
+    },
+    {
+      "epoch": 0.5352941176470588,
+      "grad_norm": 3.3592398166656494,
+      "learning_rate": 0.0001643529411764706,
+      "loss": 0.048895838856697085,
+      "step": 910
+    },
+    {
+      "epoch": 0.5411764705882353,
+      "grad_norm": 0.004421881400048733,
+      "learning_rate": 0.0001639607843137255,
+      "loss": 0.06308169960975647,
+      "step": 920
+    },
+    {
+      "epoch": 0.5470588235294118,
+      "grad_norm": 0.09252121299505234,
+      "learning_rate": 0.00016356862745098042,
+      "loss": 0.058104753494262695,
+      "step": 930
+    },
+    {
+      "epoch": 0.5529411764705883,
+      "grad_norm": 0.7001014351844788,
+      "learning_rate": 0.0001631764705882353,
+      "loss": 0.021149225533008575,
+      "step": 940
+    },
+    {
+      "epoch": 0.5588235294117647,
+      "grad_norm": 0.3170143663883209,
+      "learning_rate": 0.0001627843137254902,
+      "loss": 0.01627269834280014,
+      "step": 950
+    },
+    {
+      "epoch": 0.5647058823529412,
+      "grad_norm": 3.7102761268615723,
+      "learning_rate": 0.00016239215686274512,
+      "loss": 0.14663417339324952,
+      "step": 960
+    },
+    {
+      "epoch": 0.5705882352941176,
+      "grad_norm": 0.006750724744051695,
+      "learning_rate": 0.000162,
+      "loss": 0.10339630842208862,
+      "step": 970
+    },
+    {
+      "epoch": 0.5764705882352941,
+      "grad_norm": 1.808908462524414,
+      "learning_rate": 0.0001616078431372549,
+      "loss": 0.026013880968093872,
+      "step": 980
+    },
+    {
+      "epoch": 0.5823529411764706,
+      "grad_norm": 0.4191270172595978,
+      "learning_rate": 0.0001612156862745098,
+      "loss": 0.039347043633461,
+      "step": 990
+    },
+    {
+      "epoch": 0.5882352941176471,
+      "grad_norm": 0.3325525224208832,
+      "learning_rate": 0.0001608235294117647,
+      "loss": 0.059603160619735716,
+      "step": 1000
+    },
+    {
+      "epoch": 0.5941176470588235,
+      "grad_norm": 2.7802844047546387,
+      "learning_rate": 0.0001604313725490196,
+      "loss": 0.09034895300865173,
+      "step": 1010
+    },
+    {
+      "epoch": 0.6,
+      "grad_norm": 0.04968420788645744,
+      "learning_rate": 0.0001600392156862745,
+      "loss": 0.05755581855773926,
+      "step": 1020
+    },
+    {
+      "epoch": 0.6058823529411764,
+      "grad_norm": 0.00875788927078247,
+      "learning_rate": 0.00015964705882352942,
+      "loss": 0.08298594951629638,
+      "step": 1030
+    },
+    {
+      "epoch": 0.611764705882353,
+      "grad_norm": 0.38597434759140015,
+      "learning_rate": 0.00015925490196078433,
+      "loss": 0.009630383551120758,
+      "step": 1040
+    },
+    {
+      "epoch": 0.6176470588235294,
+      "grad_norm": 0.42396023869514465,
+      "learning_rate": 0.00015886274509803921,
+      "loss": 0.05784105658531189,
+      "step": 1050
+    },
+    {
+      "epoch": 0.6235294117647059,
+      "grad_norm": 3.5692520141601562,
+      "learning_rate": 0.00015847058823529413,
+      "loss": 0.1575460195541382,
+      "step": 1060
+    },
+    {
+      "epoch": 0.6294117647058823,
+      "grad_norm": 0.12411337345838547,
+      "learning_rate": 0.00015807843137254904,
+      "loss": 0.052008318901062014,
+      "step": 1070
+    },
+    {
+      "epoch": 0.6352941176470588,
+      "grad_norm": 0.12469016760587692,
+      "learning_rate": 0.00015768627450980392,
+      "loss": 0.06983086466789246,
+      "step": 1080
+    },
+    {
+      "epoch": 0.6411764705882353,
+      "grad_norm": 0.0010811894899234176,
+      "learning_rate": 0.00015729411764705883,
+      "loss": 0.009204772859811782,
+      "step": 1090
+    },
+    {
+      "epoch": 0.6470588235294118,
+      "grad_norm": 0.7795768976211548,
+      "learning_rate": 0.00015690196078431374,
+      "loss": 0.07802544236183166,
+      "step": 1100
+    },
+    {
+      "epoch": 0.6529411764705882,
+      "grad_norm": 3.0475451946258545,
+      "learning_rate": 0.00015650980392156865,
+      "loss": 0.06048353910446167,
+      "step": 1110
+    },
+    {
+      "epoch": 0.6588235294117647,
+      "grad_norm": 0.054235588759183884,
+      "learning_rate": 0.00015611764705882354,
+      "loss": 0.007759802043437958,
+      "step": 1120
+    },
+    {
+      "epoch": 0.6647058823529411,
+      "grad_norm": 0.022670522332191467,
+      "learning_rate": 0.00015572549019607845,
+      "loss": 0.121265709400177,
+      "step": 1130
+    },
+    {
+      "epoch": 0.6705882352941176,
+      "grad_norm": 4.404564380645752,
+      "learning_rate": 0.00015533333333333333,
+      "loss": 0.11734864711761475,
+      "step": 1140
+    },
+    {
+      "epoch": 0.6764705882352942,
+      "grad_norm": 0.04260774329304695,
+      "learning_rate": 0.00015494117647058824,
+      "loss": 0.04804904460906982,
+      "step": 1150
+    },
+    {
+      "epoch": 0.6823529411764706,
+      "grad_norm": 0.37849345803260803,
+      "learning_rate": 0.00015454901960784313,
+      "loss": 0.08949239253997802,
+      "step": 1160
+    },
+    {
+      "epoch": 0.6882352941176471,
+      "grad_norm": 0.07910435646772385,
+      "learning_rate": 0.00015415686274509804,
+      "loss": 0.11236823797225952,
+      "step": 1170
+    },
+    {
+      "epoch": 0.6941176470588235,
+      "grad_norm": 0.020735427737236023,
+      "learning_rate": 0.00015376470588235295,
+      "loss": 0.020005679130554198,
+      "step": 1180
+    },
+    {
+      "epoch": 0.7,
+      "grad_norm": 0.6041317582130432,
+      "learning_rate": 0.00015337254901960786,
+      "loss": 0.047528034448623656,
+      "step": 1190
+    },
+    {
+      "epoch": 0.7058823529411765,
+      "grad_norm": 0.07179152965545654,
+      "learning_rate": 0.00015298039215686274,
+      "loss": 0.03959411382675171,
+      "step": 1200
+    },
+    {
+      "epoch": 0.711764705882353,
+      "grad_norm": 0.009498389437794685,
+      "learning_rate": 0.00015258823529411766,
+      "loss": 0.003311210125684738,
+      "step": 1210
+    },
+    {
+      "epoch": 0.7176470588235294,
+      "grad_norm": 0.433115154504776,
+      "learning_rate": 0.00015219607843137257,
+      "loss": 0.00858706161379814,
+      "step": 1220
+    },
+    {
+      "epoch": 0.7235294117647059,
+      "grad_norm": 0.008886504918336868,
+      "learning_rate": 0.00015180392156862745,
+      "loss": 0.06958861947059632,
+      "step": 1230
+    },
+    {
+      "epoch": 0.7294117647058823,
+      "grad_norm": 0.008790708146989346,
+      "learning_rate": 0.00015141176470588236,
+      "loss": 0.008029435575008393,
+      "step": 1240
+    },
+    {
+      "epoch": 0.7352941176470589,
+      "grad_norm": 0.005836977623403072,
+      "learning_rate": 0.00015101960784313727,
+      "loss": 0.05358666181564331,
+      "step": 1250
+    },
+    {
+      "epoch": 0.7411764705882353,
+      "grad_norm": 0.14504218101501465,
+      "learning_rate": 0.00015062745098039218,
+      "loss": 0.049011257290840146,
+      "step": 1260
+    },
+    {
+      "epoch": 0.7470588235294118,
+      "grad_norm": 0.2803121507167816,
+      "learning_rate": 0.00015023529411764707,
+      "loss": 0.006191087141633034,
+      "step": 1270
+    },
+    {
+      "epoch": 0.7529411764705882,
+      "grad_norm": 3.1994972229003906,
+      "learning_rate": 0.00014984313725490198,
+      "loss": 0.10595870018005371,
+      "step": 1280
+    },
+    {
+      "epoch": 0.7588235294117647,
+      "grad_norm": 0.00779637461528182,
+      "learning_rate": 0.0001494509803921569,
+      "loss": 0.07793840169906616,
+      "step": 1290
+    },
+    {
+      "epoch": 0.7647058823529411,
+      "grad_norm": 0.04689478874206543,
+      "learning_rate": 0.00014905882352941177,
+      "loss": 0.08080050349235535,
+      "step": 1300
+    },
+    {
+      "epoch": 0.7705882352941177,
+      "grad_norm": 0.9595724940299988,
+      "learning_rate": 0.00014866666666666666,
+      "loss": 0.11690077781677247,
+      "step": 1310
+    },
+    {
+      "epoch": 0.7764705882352941,
+      "grad_norm": 4.141428470611572,
+      "learning_rate": 0.00014827450980392157,
+      "loss": 0.05245780348777771,
+      "step": 1320
+    },
+    {
+      "epoch": 0.7823529411764706,
+      "grad_norm": 0.7606799602508545,
+      "learning_rate": 0.00014788235294117648,
+      "loss": 0.039501425623893735,
+      "step": 1330
+    },
+    {
+      "epoch": 0.788235294117647,
+      "grad_norm": 2.8373472690582275,
+      "learning_rate": 0.00014749019607843136,
+      "loss": 0.08525887727737427,
+      "step": 1340
+    },
+    {
+      "epoch": 0.7941176470588235,
+      "grad_norm": 0.20202526450157166,
+      "learning_rate": 0.00014709803921568627,
+      "loss": 0.03491775095462799,
+      "step": 1350
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.00345594366081059,
+      "learning_rate": 0.00014670588235294119,
+      "loss": 0.03784347176551819,
+      "step": 1360
+    },
+    {
+      "epoch": 0.8058823529411765,
+      "grad_norm": 3.3252758979797363,
+      "learning_rate": 0.0001463137254901961,
+      "loss": 0.08734545707702637,
+      "step": 1370
+    },
+    {
+      "epoch": 0.8117647058823529,
+      "grad_norm": 1.1744837760925293,
+      "learning_rate": 0.00014592156862745098,
+      "loss": 0.060222625732421875,
+      "step": 1380
+    },
+    {
+      "epoch": 0.8176470588235294,
+      "grad_norm": 0.012418321333825588,
+      "learning_rate": 0.0001455294117647059,
+      "loss": 0.034251919388771056,
+      "step": 1390
+    },
+    {
+      "epoch": 0.8235294117647058,
+      "grad_norm": 0.18622326850891113,
+      "learning_rate": 0.0001451372549019608,
+      "loss": 0.02356344759464264,
+      "step": 1400
+    },
+    {
+      "epoch": 0.8294117647058824,
+      "grad_norm": 3.9867045879364014,
+      "learning_rate": 0.00014474509803921569,
+      "loss": 0.07036476135253907,
+      "step": 1410
+    },
+    {
+      "epoch": 0.8352941176470589,
+      "grad_norm": 3.0563607215881348,
+      "learning_rate": 0.0001443529411764706,
+      "loss": 0.037895727157592776,
+      "step": 1420
+    },
+    {
+      "epoch": 0.8411764705882353,
+      "grad_norm": 0.13856367766857147,
+      "learning_rate": 0.0001439607843137255,
+      "loss": 0.12252027988433838,
+      "step": 1430
+    },
+    {
+      "epoch": 0.8470588235294118,
+      "grad_norm": 6.263605117797852,
+      "learning_rate": 0.00014356862745098042,
+      "loss": 0.04287775456905365,
+      "step": 1440
+    },
+    {
+      "epoch": 0.8529411764705882,
+      "grad_norm": 0.6084681749343872,
+      "learning_rate": 0.0001431764705882353,
+      "loss": 0.050480049848556516,
+      "step": 1450
+    },
+    {
+      "epoch": 0.8588235294117647,
+      "grad_norm": 0.13102763891220093,
+      "learning_rate": 0.0001427843137254902,
+      "loss": 0.027205181121826173,
+      "step": 1460
+    },
+    {
+      "epoch": 0.8647058823529412,
+      "grad_norm": 0.0334654301404953,
+      "learning_rate": 0.0001423921568627451,
+      "loss": 0.07116500139236451,
+      "step": 1470
+    },
+    {
+      "epoch": 0.8705882352941177,
+      "grad_norm": 5.854252815246582,
+      "learning_rate": 0.000142,
+      "loss": 0.10154651403427124,
+      "step": 1480
+    },
+    {
+      "epoch": 0.8764705882352941,
+      "grad_norm": 0.04299909248948097,
+      "learning_rate": 0.0001416078431372549,
+      "loss": 0.06952143907546997,
+      "step": 1490
+    },
+    {
+      "epoch": 0.8823529411764706,
+      "grad_norm": 0.03385253995656967,
+      "learning_rate": 0.0001412156862745098,
+      "loss": 0.042281192541122434,
+      "step": 1500
+    },
+    {
+      "epoch": 0.888235294117647,
+      "grad_norm": 3.0850558280944824,
+      "learning_rate": 0.00014082352941176472,
+      "loss": 0.04330833852291107,
+      "step": 1510
+    },
+    {
+      "epoch": 0.8941176470588236,
+      "grad_norm": 1.0124292373657227,
+      "learning_rate": 0.00014043137254901963,
+      "loss": 0.013002160191535949,
+      "step": 1520
+    },
+    {
+      "epoch": 0.9,
+      "grad_norm": 0.4087558388710022,
+      "learning_rate": 0.0001400392156862745,
+      "loss": 0.08435083031654358,
+      "step": 1530
+    },
+    {
+      "epoch": 0.9058823529411765,
+      "grad_norm": 1.3371378183364868,
+      "learning_rate": 0.00013964705882352942,
+      "loss": 0.04774133563041687,
+      "step": 1540
+    },
+    {
+      "epoch": 0.9117647058823529,
+      "grad_norm": 3.8198187351226807,
+      "learning_rate": 0.00013925490196078433,
+      "loss": 0.05395355820655823,
+      "step": 1550
+    },
+    {
+      "epoch": 0.9176470588235294,
+      "grad_norm": 0.07719609141349792,
+      "learning_rate": 0.00013886274509803922,
+      "loss": 0.05478907227516174,
+      "step": 1560
+    },
+    {
+      "epoch": 0.9235294117647059,
+      "grad_norm": 0.14217644929885864,
+      "learning_rate": 0.00013847058823529413,
+      "loss": 0.037067908048629764,
+      "step": 1570
+    },
+    {
+      "epoch": 0.9294117647058824,
+      "grad_norm": 0.0115964999422431,
+      "learning_rate": 0.00013807843137254904,
+      "loss": 0.06596972942352294,
+      "step": 1580
+    },
+    {
+      "epoch": 0.9352941176470588,
+      "grad_norm": 0.11416631937026978,
+      "learning_rate": 0.00013768627450980395,
+      "loss": 0.06003996729850769,
+      "step": 1590
+    },
+    {
+      "epoch": 0.9411764705882353,
+      "grad_norm": 0.8297443985939026,
+      "learning_rate": 0.00013729411764705883,
+      "loss": 0.012542881071567535,
+      "step": 1600
+    },
+    {
+      "epoch": 0.9470588235294117,
+      "grad_norm": 2.7102794647216797,
+      "learning_rate": 0.00013690196078431374,
+      "loss": 0.04482048749923706,
+      "step": 1610
+    },
+    {
+      "epoch": 0.9529411764705882,
+      "grad_norm": 0.04676695913076401,
+      "learning_rate": 0.00013650980392156863,
+      "loss": 0.04008999168872833,
+      "step": 1620
+    },
+    {
+      "epoch": 0.9588235294117647,
+      "grad_norm": 0.024560345336794853,
+      "learning_rate": 0.00013611764705882354,
+      "loss": 0.004030698910355568,
+      "step": 1630
+    },
+    {
+      "epoch": 0.9647058823529412,
+      "grad_norm": 0.004613540135324001,
+      "learning_rate": 0.00013572549019607842,
+      "loss": 0.016170585155487062,
+      "step": 1640
+    },
+    {
+      "epoch": 0.9705882352941176,
+      "grad_norm": 0.07655692845582962,
+      "learning_rate": 0.00013533333333333333,
+      "loss": 0.002394220232963562,
+      "step": 1650
+    },
+    {
+      "epoch": 0.9764705882352941,
+      "grad_norm": 1.5127253532409668,
+      "learning_rate": 0.00013494117647058825,
+      "loss": 0.08604814410209656,
+      "step": 1660
+    },
+    {
+      "epoch": 0.9823529411764705,
+      "grad_norm": 4.647606372833252,
+      "learning_rate": 0.00013454901960784313,
+      "loss": 0.05253278613090515,
+      "step": 1670
+    },
+    {
+      "epoch": 0.9882352941176471,
+      "grad_norm": 0.0031903591006994247,
+      "learning_rate": 0.00013415686274509804,
+      "loss": 0.05996617674827576,
+      "step": 1680
+    },
+    {
+      "epoch": 0.9941176470588236,
+      "grad_norm": 3.4767584800720215,
+      "learning_rate": 0.00013376470588235295,
+      "loss": 0.09587143063545227,
+      "step": 1690
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.6253292560577393,
+      "learning_rate": 0.00013337254901960786,
+      "loss": 0.004908151552081108,
+      "step": 1700
+    },
+    {
+      "epoch": 1.0,
+      "eval_accuracy": 0.9833333333333333,
+      "eval_loss": 0.052190084010362625,
+      "eval_runtime": 107.7754,
+      "eval_samples_per_second": 44.537,
+      "eval_steps_per_second": 2.784,
+      "step": 1700
+    },
+    {
+      "epoch": 1.0058823529411764,
+      "grad_norm": 0.00878067221492529,
+      "learning_rate": 0.00013298039215686275,
+      "loss": 0.04642733633518219,
+      "step": 1710
+    },
+    {
+      "epoch": 1.011764705882353,
+      "grad_norm": 0.04165046289563179,
+      "learning_rate": 0.00013258823529411766,
+      "loss": 0.01063329577445984,
+      "step": 1720
+    },
+    {
+      "epoch": 1.0176470588235293,
+      "grad_norm": 0.008701024577021599,
+      "learning_rate": 0.00013219607843137257,
+      "loss": 0.05080258846282959,
+      "step": 1730
+    },
+    {
+      "epoch": 1.0235294117647058,
+      "grad_norm": 0.019413400441408157,
+      "learning_rate": 0.00013180392156862745,
+      "loss": 0.047126364707946775,
+      "step": 1740
+    },
+    {
+      "epoch": 1.0294117647058822,
+      "grad_norm": 3.103982925415039,
+      "learning_rate": 0.00013141176470588236,
+      "loss": 0.029054802656173707,
+      "step": 1750
+    },
+    {
+      "epoch": 1.035294117647059,
+      "grad_norm": 0.00992797501385212,
+      "learning_rate": 0.00013101960784313727,
+      "loss": 0.010611901432275772,
+      "step": 1760
+    },
+    {
+      "epoch": 1.0411764705882354,
+      "grad_norm": 0.011621154844760895,
+      "learning_rate": 0.00013062745098039219,
+      "loss": 0.05253502130508423,
+      "step": 1770
+    },
+    {
+      "epoch": 1.0470588235294118,
+      "grad_norm": 0.38787615299224854,
+      "learning_rate": 0.00013023529411764704,
+      "loss": 0.05395628809928894,
+      "step": 1780
+    },
+    {
+      "epoch": 1.0529411764705883,
+      "grad_norm": 2.963691234588623,
+      "learning_rate": 0.00012984313725490195,
+      "loss": 0.11310794353485107,
+      "step": 1790
+    },
+    {
+      "epoch": 1.0588235294117647,
+      "grad_norm": 0.1639397144317627,
+      "learning_rate": 0.00012945098039215686,
+      "loss": 0.020070001482963562,
+      "step": 1800
+    },
+    {
+      "epoch": 1.0647058823529412,
+      "grad_norm": 1.1359699964523315,
+      "learning_rate": 0.00012905882352941178,
+      "loss": 0.050649511814117434,
+      "step": 1810
+    },
+    {
+      "epoch": 1.0705882352941176,
+      "grad_norm": 0.5498014092445374,
+      "learning_rate": 0.00012866666666666666,
+      "loss": 0.00956227332353592,
+      "step": 1820
+    },
+    {
+      "epoch": 1.076470588235294,
+      "grad_norm": 0.1549549251794815,
+      "learning_rate": 0.00012827450980392157,
+      "loss": 0.013553234934806823,
+      "step": 1830
+    },
+    {
+      "epoch": 1.0823529411764705,
+      "grad_norm": 0.045267343521118164,
+      "learning_rate": 0.00012788235294117648,
+      "loss": 0.029942601919174194,
+      "step": 1840
+    },
+    {
+      "epoch": 1.088235294117647,
+      "grad_norm": 0.2236713171005249,
+      "learning_rate": 0.0001274901960784314,
+      "loss": 0.03128021359443665,
+      "step": 1850
+    },
+    {
+      "epoch": 1.0941176470588236,
+      "grad_norm": 0.03231283649802208,
+      "learning_rate": 0.00012709803921568628,
+      "loss": 0.045766386389732364,
+      "step": 1860
+    },
+    {
+      "epoch": 1.1,
+      "grad_norm": 0.0012792079942300916,
+      "learning_rate": 0.0001267058823529412,
+      "loss": 0.005739885941147804,
+      "step": 1870
+    },
+    {
+      "epoch": 1.1058823529411765,
+      "grad_norm": 0.08689229190349579,
+      "learning_rate": 0.0001263137254901961,
+      "loss": 0.010996489971876144,
+      "step": 1880
+    },
+    {
+      "epoch": 1.111764705882353,
+      "grad_norm": 2.1905863285064697,
+      "learning_rate": 0.00012592156862745098,
+      "loss": 0.054632896184921266,
+      "step": 1890
+    },
+    {
+      "epoch": 1.1176470588235294,
+      "grad_norm": 0.00794258713722229,
+      "learning_rate": 0.0001255294117647059,
+      "loss": 0.028382882475852966,
+      "step": 1900
+    },
+    {
+      "epoch": 1.1235294117647059,
+      "grad_norm": 1.3497296571731567,
+      "learning_rate": 0.0001251372549019608,
+      "loss": 0.006283405423164368,
+      "step": 1910
+    },
+    {
+      "epoch": 1.1294117647058823,
+      "grad_norm": 0.24390794336795807,
+      "learning_rate": 0.00012474509803921572,
+      "loss": 0.004674851521849632,
+      "step": 1920
+    },
+    {
+      "epoch": 1.1352941176470588,
+      "grad_norm": 1.989312767982483,
+      "learning_rate": 0.00012435294117647057,
+      "loss": 0.017390060424804687,
+      "step": 1930
+    },
+    {
+      "epoch": 1.1411764705882352,
+      "grad_norm": 0.0406913086771965,
+      "learning_rate": 0.00012396078431372548,
+      "loss": 0.014303833246231079,
+      "step": 1940
+    },
+    {
+      "epoch": 1.1470588235294117,
+      "grad_norm": 0.04144569858908653,
+      "learning_rate": 0.0001235686274509804,
+      "loss": 0.009866081923246384,
+      "step": 1950
+    },
+    {
+      "epoch": 1.1529411764705881,
+      "grad_norm": 4.504534721374512,
+      "learning_rate": 0.0001231764705882353,
+      "loss": 0.05014045238494873,
+      "step": 1960
+    },
+    {
+      "epoch": 1.1588235294117646,
+      "grad_norm": 0.07257991284132004,
+      "learning_rate": 0.0001227843137254902,
+      "loss": 0.03196141719818115,
+      "step": 1970
+    },
+    {
+      "epoch": 1.1647058823529413,
+      "grad_norm": 0.004560091998428106,
+      "learning_rate": 0.0001223921568627451,
+      "loss": 0.003665846586227417,
+      "step": 1980
+    },
+    {
+      "epoch": 1.1705882352941177,
+      "grad_norm": 5.35031795501709,
+      "learning_rate": 0.000122,
+      "loss": 0.04513788819313049,
+      "step": 1990
+    },
+    {
+      "epoch": 1.1764705882352942,
+      "grad_norm": 8.097407341003418,
+      "learning_rate": 0.00012160784313725491,
+      "loss": 0.11825064420700074,
+      "step": 2000
+    },
+    {
+      "epoch": 1.1823529411764706,
+      "grad_norm": 0.00166159903164953,
+      "learning_rate": 0.0001212156862745098,
+      "loss": 0.0033010929822921754,
+      "step": 2010
+    },
+    {
+      "epoch": 1.188235294117647,
+      "grad_norm": 0.00044547455036081374,
+      "learning_rate": 0.00012082352941176472,
+      "loss": 0.017485101521015168,
+      "step": 2020
+    },
+    {
+      "epoch": 1.1941176470588235,
+      "grad_norm": 0.021079039201140404,
+      "learning_rate": 0.00012043137254901961,
+      "loss": 0.12937935590744018,
+      "step": 2030
+    },
+    {
+      "epoch": 1.2,
+      "grad_norm": 0.19882740080356598,
+      "learning_rate": 0.00012003921568627453,
+      "loss": 0.00260718259960413,
+      "step": 2040
+    },
+    {
+      "epoch": 1.2058823529411764,
+      "grad_norm": 0.0020646140910685062,
+      "learning_rate": 0.00011964705882352942,
+      "loss": 0.04813997447490692,
+      "step": 2050
+    },
+    {
+      "epoch": 1.2117647058823529,
+      "grad_norm": 0.027832122519612312,
+      "learning_rate": 0.00011925490196078432,
+      "loss": 0.07265179753303527,
+      "step": 2060
+    },
+    {
+      "epoch": 1.2176470588235295,
+      "grad_norm": 0.039618801325559616,
+      "learning_rate": 0.00011886274509803923,
+      "loss": 0.11547826528549195,
+      "step": 2070
+    },
+    {
+      "epoch": 1.223529411764706,
+      "grad_norm": 0.08470630645751953,
+      "learning_rate": 0.00011847058823529413,
+      "loss": 0.01048896461725235,
+      "step": 2080
+    },
+    {
+      "epoch": 1.2294117647058824,
+      "grad_norm": 0.17790746688842773,
+      "learning_rate": 0.00011807843137254901,
+      "loss": 0.01930353045463562,
+      "step": 2090
+    },
+    {
+      "epoch": 1.2352941176470589,
+      "grad_norm": 0.28496846556663513,
+      "learning_rate": 0.00011768627450980392,
+      "loss": 0.05006638169288635,
+      "step": 2100
+    },
+    {
+      "epoch": 1.2411764705882353,
+      "grad_norm": 0.9003567099571228,
+      "learning_rate": 0.00011729411764705882,
+      "loss": 0.05795367956161499,
+      "step": 2110
+    },
+    {
+      "epoch": 1.2470588235294118,
+      "grad_norm": 1.185327410697937,
+      "learning_rate": 0.00011690196078431372,
+      "loss": 0.030656060576438902,
+      "step": 2120
+    },
+    {
+      "epoch": 1.2529411764705882,
+      "grad_norm": 0.007268443703651428,
+      "learning_rate": 0.00011650980392156863,
+      "loss": 0.012932188808918,
+      "step": 2130
+    },
+    {
+      "epoch": 1.2588235294117647,
+      "grad_norm": 0.059161823242902756,
+      "learning_rate": 0.00011611764705882353,
+      "loss": 0.007702886313199997,
+      "step": 2140
+    },
+    {
+      "epoch": 1.2647058823529411,
+      "grad_norm": 4.125527381896973,
+      "learning_rate": 0.00011572549019607844,
+      "loss": 0.038613501191139224,
+      "step": 2150
+    },
+    {
+      "epoch": 1.2705882352941176,
+      "grad_norm": 0.054057635366916656,
+      "learning_rate": 0.00011533333333333334,
+      "loss": 0.0619198739528656,
+      "step": 2160
+    },
+    {
+      "epoch": 1.276470588235294,
+      "grad_norm": 0.12836681306362152,
+      "learning_rate": 0.00011494117647058825,
+      "loss": 0.034237629175186156,
+      "step": 2170
+    },
+    {
+      "epoch": 1.2823529411764705,
+      "grad_norm": 0.4671003818511963,
+      "learning_rate": 0.00011454901960784314,
+      "loss": 0.06266800761222839,
+      "step": 2180
+    },
+    {
+      "epoch": 1.288235294117647,
+      "grad_norm": 5.042869567871094,
+      "learning_rate": 0.00011415686274509804,
+      "loss": 0.04739983081817627,
+      "step": 2190
+    },
+    {
+      "epoch": 1.2941176470588236,
+      "grad_norm": 1.037730097770691,
+      "learning_rate": 0.00011376470588235295,
+      "loss": 0.028715655207633972,
+      "step": 2200
+    },
+    {
+      "epoch": 1.3,
+      "grad_norm": 1.5472325086593628,
+      "learning_rate": 0.00011337254901960785,
+      "loss": 0.030545881390571593,
+      "step": 2210
+    },
+    {
+      "epoch": 1.3058823529411765,
+      "grad_norm": 0.015368582680821419,
+      "learning_rate": 0.00011298039215686276,
+      "loss": 0.036614406108856204,
+      "step": 2220
+    },
+    {
+      "epoch": 1.311764705882353,
+      "grad_norm": 1.9070743322372437,
+      "learning_rate": 0.00011258823529411766,
+      "loss": 0.013732218742370605,
+      "step": 2230
+    },
+    {
+      "epoch": 1.3176470588235294,
+      "grad_norm": 4.028462886810303,
+      "learning_rate": 0.00011219607843137257,
+      "loss": 0.09811515212059022,
+      "step": 2240
+    },
+    {
+      "epoch": 1.3235294117647058,
+      "grad_norm": 0.8748077750205994,
+      "learning_rate": 0.00011180392156862744,
+      "loss": 0.02405388206243515,
+      "step": 2250
+    },
+    {
+      "epoch": 1.3294117647058823,
+      "grad_norm": 0.003571397392079234,
+      "learning_rate": 0.00011141176470588235,
+      "loss": 0.05482441186904907,
+      "step": 2260
+    },
+    {
+      "epoch": 1.3352941176470587,
+      "grad_norm": 0.07122784107923508,
+      "learning_rate": 0.00011101960784313725,
+      "loss": 0.019011862576007843,
+      "step": 2270
+    },
+    {
+      "epoch": 1.3411764705882354,
+      "grad_norm": 0.1454578936100006,
+      "learning_rate": 0.00011062745098039216,
+      "loss": 0.05575419068336487,
+      "step": 2280
+    },
+    {
+      "epoch": 1.3470588235294119,
+      "grad_norm": 0.547131359577179,
+      "learning_rate": 0.00011023529411764706,
+      "loss": 0.030268779397010802,
+      "step": 2290
+    },
+    {
+      "epoch": 1.3529411764705883,
+      "grad_norm": 0.004670478869229555,
+      "learning_rate": 0.00010984313725490197,
+      "loss": 0.016492824256420135,
+      "step": 2300
+    },
+    {
+      "epoch": 1.3588235294117648,
+      "grad_norm": 0.011210484430193901,
+      "learning_rate": 0.00010945098039215687,
+      "loss": 0.033874979615211485,
+      "step": 2310
+    },
+    {
+      "epoch": 1.3647058823529412,
+      "grad_norm": 0.01266207080334425,
+      "learning_rate": 0.00010905882352941176,
+      "loss": 0.05161962509155273,
+      "step": 2320
+    },
+    {
+      "epoch": 1.3705882352941177,
+      "grad_norm": 2.460120677947998,
+      "learning_rate": 0.00010866666666666667,
+      "loss": 0.021566420793533325,
+      "step": 2330
+    },
+    {
+      "epoch": 1.3764705882352941,
+      "grad_norm": 1.084808588027954,
+      "learning_rate": 0.00010827450980392157,
+      "loss": 0.061221182346343994,
+      "step": 2340
+    },
+    {
+      "epoch": 1.3823529411764706,
+      "grad_norm": 2.849195718765259,
+      "learning_rate": 0.00010788235294117648,
+      "loss": 0.01972988098859787,
+      "step": 2350
+    },
+    {
+      "epoch": 1.388235294117647,
+      "grad_norm": 0.014458741061389446,
+      "learning_rate": 0.00010749019607843138,
+      "loss": 0.05717582106590271,
+      "step": 2360
+    },
+    {
+      "epoch": 1.3941176470588235,
+      "grad_norm": 0.8046371936798096,
+      "learning_rate": 0.00010709803921568629,
+      "loss": 0.03862323462963104,
+      "step": 2370
+    },
+    {
+      "epoch": 1.4,
+      "grad_norm": 0.013660217635333538,
+      "learning_rate": 0.00010670588235294119,
+      "loss": 0.019610072672367095,
+      "step": 2380
+    },
+    {
+      "epoch": 1.4058823529411764,
+      "grad_norm": 0.6871390342712402,
+      "learning_rate": 0.00010631372549019609,
+      "loss": 0.03682593703269958,
+      "step": 2390
+    },
+    {
+      "epoch": 1.4117647058823528,
+      "grad_norm": 4.356563091278076,
+      "learning_rate": 0.000105921568627451,
+      "loss": 0.043030115962028506,
+      "step": 2400
+    },
+    {
+      "epoch": 1.4176470588235295,
+      "grad_norm": 0.010556154884397984,
+      "learning_rate": 0.00010552941176470588,
+      "loss": 0.03134247660636902,
+      "step": 2410
+    },
+    {
+      "epoch": 1.423529411764706,
+      "grad_norm": 0.6451873183250427,
+      "learning_rate": 0.00010513725490196078,
+      "loss": 0.022086340188980102,
+      "step": 2420
+    },
+    {
+      "epoch": 1.4294117647058824,
+      "grad_norm": 0.08090002834796906,
+      "learning_rate": 0.00010474509803921569,
+      "loss": 0.050183969736099246,
+      "step": 2430
+    },
+    {
+      "epoch": 1.4352941176470588,
+      "grad_norm": 0.9792000651359558,
+      "learning_rate": 0.00010435294117647059,
+      "loss": 0.03801586329936981,
+      "step": 2440
+    },
+    {
+      "epoch": 1.4411764705882353,
+      "grad_norm": 0.04448526352643967,
+      "learning_rate": 0.00010396078431372548,
+      "loss": 0.02308562248945236,
+      "step": 2450
+    },
+    {
+      "epoch": 1.4470588235294117,
+      "grad_norm": 1.6576273441314697,
+      "learning_rate": 0.0001035686274509804,
+      "loss": 0.07776076793670654,
+      "step": 2460
+    },
+    {
+      "epoch": 1.4529411764705882,
+      "grad_norm": 0.21055442094802856,
+      "learning_rate": 0.00010317647058823529,
+      "loss": 0.0028534408658742903,
+      "step": 2470
+    },
+    {
+      "epoch": 1.4588235294117646,
+      "grad_norm": 0.010541798546910286,
+      "learning_rate": 0.0001027843137254902,
+      "loss": 0.043876749277114865,
+      "step": 2480
+    },
+    {
+      "epoch": 1.4647058823529413,
+      "grad_norm": 0.28795304894447327,
+      "learning_rate": 0.0001023921568627451,
+      "loss": 0.03177463412284851,
+      "step": 2490
+    },
+    {
+      "epoch": 1.4705882352941178,
+      "grad_norm": 0.14733652770519257,
+      "learning_rate": 0.00010200000000000001,
+      "loss": 0.004313435778021813,
+      "step": 2500
+    },
+    {
+      "epoch": 1.4764705882352942,
+      "grad_norm": 0.06586641073226929,
+      "learning_rate": 0.00010160784313725491,
+      "loss": 0.02194916158914566,
+      "step": 2510
+    },
+    {
+      "epoch": 1.4823529411764707,
+      "grad_norm": 5.449988842010498,
+      "learning_rate": 0.00010121568627450981,
+      "loss": 0.048442333936691284,
+      "step": 2520
+    },
+    {
+      "epoch": 1.488235294117647,
+      "grad_norm": 0.005476059392094612,
+      "learning_rate": 0.00010082352941176472,
+      "loss": 0.05485616326332092,
+      "step": 2530
+    },
+    {
+      "epoch": 1.4941176470588236,
+      "grad_norm": 0.00165216822642833,
+      "learning_rate": 0.00010043137254901962,
+      "loss": 0.00865812823176384,
+      "step": 2540
+    },
+    {
+      "epoch": 1.5,
+      "grad_norm": 0.1594460904598236,
+      "learning_rate": 0.00010003921568627453,
+      "loss": 0.046700218319892885,
+      "step": 2550
+    },
+    {
+      "epoch": 1.5058823529411764,
+      "grad_norm": 0.286100834608078,
+      "learning_rate": 9.964705882352941e-05,
+      "loss": 0.019821707904338837,
+      "step": 2560
+    },
+    {
+      "epoch": 1.511764705882353,
+      "grad_norm": 0.03422806039452553,
+      "learning_rate": 9.925490196078432e-05,
+      "loss": 0.07443686723709106,
+      "step": 2570
+    },
+    {
+      "epoch": 1.5176470588235293,
+      "grad_norm": 0.006144434213638306,
+      "learning_rate": 9.886274509803922e-05,
+      "loss": 0.0359457790851593,
+      "step": 2580
+    },
+    {
+      "epoch": 1.5235294117647058,
+      "grad_norm": 0.37471282482147217,
+      "learning_rate": 9.847058823529413e-05,
+      "loss": 0.056217139959335326,
+      "step": 2590
+    },
+    {
+      "epoch": 1.5294117647058822,
+      "grad_norm": 1.6320679187774658,
+      "learning_rate": 9.807843137254903e-05,
+      "loss": 0.025465011596679688,
+      "step": 2600
+    },
+    {
+      "epoch": 1.5352941176470587,
+      "grad_norm": 0.05187623202800751,
+      "learning_rate": 9.768627450980393e-05,
+      "loss": 0.013015411794185638,
+      "step": 2610
+    },
+    {
+      "epoch": 1.5411764705882351,
+      "grad_norm": 0.004285014700144529,
+      "learning_rate": 9.729411764705882e-05,
+      "loss": 0.09077309370040894,
+      "step": 2620
+    },
+    {
+      "epoch": 1.5470588235294118,
+      "grad_norm": 0.008745596744120121,
+      "learning_rate": 9.690196078431373e-05,
+      "loss": 0.00500418022274971,
+      "step": 2630
+    },
+    {
+      "epoch": 1.5529411764705883,
+      "grad_norm": 4.836589336395264,
+      "learning_rate": 9.650980392156863e-05,
+      "loss": 0.01722148060798645,
+      "step": 2640
+    },
+    {
+      "epoch": 1.5588235294117647,
+      "grad_norm": 0.12729062139987946,
+      "learning_rate": 9.611764705882353e-05,
+      "loss": 0.009548765420913697,
+      "step": 2650
+    },
+    {
+      "epoch": 1.5647058823529412,
+      "grad_norm": 0.028509173542261124,
+      "learning_rate": 9.572549019607844e-05,
+      "loss": 0.05104944109916687,
+      "step": 2660
+    },
+    {
+      "epoch": 1.5705882352941176,
+      "grad_norm": 0.004232144448906183,
+      "learning_rate": 9.533333333333334e-05,
+      "loss": 0.002674447558820248,
+      "step": 2670
+    },
+    {
+      "epoch": 1.576470588235294,
+      "grad_norm": 0.0019184383563697338,
+      "learning_rate": 9.494117647058825e-05,
+      "loss": 0.0016546357423067094,
+      "step": 2680
+    },
+    {
+      "epoch": 1.5823529411764707,
+      "grad_norm": 1.01856529712677,
+      "learning_rate": 9.454901960784313e-05,
+      "loss": 0.061312144994735716,
+      "step": 2690
+    },
+    {
+      "epoch": 1.5882352941176472,
+      "grad_norm": 2.3208155632019043,
+      "learning_rate": 9.415686274509804e-05,
+      "loss": 0.05617377758026123,
+      "step": 2700
+    },
+    {
+      "epoch": 1.5941176470588236,
+      "grad_norm": 0.027910761535167694,
+      "learning_rate": 9.376470588235294e-05,
+      "loss": 0.02175644040107727,
+      "step": 2710
+    },
+    {
+      "epoch": 1.6,
+      "grad_norm": 5.568607330322266,
+      "learning_rate": 9.337254901960785e-05,
+      "loss": 0.027315101027488707,
+      "step": 2720
+    },
+    {
+      "epoch": 1.6058823529411765,
+      "grad_norm": 0.001279431045986712,
+      "learning_rate": 9.298039215686275e-05,
+      "loss": 0.04750293493270874,
+      "step": 2730
+    },
+    {
+      "epoch": 1.611764705882353,
+      "grad_norm": 0.01567915268242359,
+      "learning_rate": 9.258823529411765e-05,
+      "loss": 0.04267770051956177,
+      "step": 2740
+    },
+    {
+      "epoch": 1.6176470588235294,
+      "grad_norm": 4.32657527923584,
+      "learning_rate": 9.219607843137256e-05,
+      "loss": 0.07141047716140747,
+      "step": 2750
+    },
+    {
+      "epoch": 1.6235294117647059,
+      "grad_norm": 0.034185998141765594,
+      "learning_rate": 9.180392156862746e-05,
+      "loss": 0.0732846200466156,
+      "step": 2760
+    },
+    {
+      "epoch": 1.6294117647058823,
+      "grad_norm": 0.004520747810602188,
+      "learning_rate": 9.141176470588235e-05,
+      "loss": 0.015270189940929412,
+      "step": 2770
+    },
+    {
+      "epoch": 1.6352941176470588,
+      "grad_norm": 0.0034092802088707685,
+      "learning_rate": 9.101960784313725e-05,
+      "loss": 0.021787433326244353,
+      "step": 2780
+    },
+    {
+      "epoch": 1.6411764705882352,
+      "grad_norm": 0.01586519554257393,
+      "learning_rate": 9.062745098039216e-05,
+      "loss": 0.03586234748363495,
+      "step": 2790
+    },
+    {
+      "epoch": 1.6470588235294117,
+      "grad_norm": 9.93665885925293,
+      "learning_rate": 9.023529411764706e-05,
+      "loss": 0.11929636001586914,
+      "step": 2800
+    },
+    {
+      "epoch": 1.6529411764705881,
+      "grad_norm": 0.028572263196110725,
+      "learning_rate": 8.984313725490197e-05,
+      "loss": 0.04672665894031525,
+      "step": 2810
+    },
+    {
+      "epoch": 1.6588235294117646,
+      "grad_norm": 0.002881932072341442,
+      "learning_rate": 8.945098039215687e-05,
+      "loss": 0.05720874071121216,
+      "step": 2820
+    },
+    {
+      "epoch": 1.664705882352941,
+      "grad_norm": 0.010270626284182072,
+      "learning_rate": 8.905882352941178e-05,
+      "loss": 0.067061048746109,
+      "step": 2830
+    },
+    {
+      "epoch": 1.6705882352941175,
+      "grad_norm": 3.3275461196899414,
+      "learning_rate": 8.866666666666668e-05,
+      "loss": 0.06687711477279663,
+      "step": 2840
+    },
+    {
+      "epoch": 1.6764705882352942,
+      "grad_norm": 4.426781177520752,
+      "learning_rate": 8.827450980392157e-05,
+      "loss": 0.0418778121471405,
+      "step": 2850
+    },
+    {
+      "epoch": 1.6823529411764706,
+      "grad_norm": 3.0773253440856934,
+      "learning_rate": 8.788235294117647e-05,
+      "loss": 0.007562777400016785,
+      "step": 2860
+    },
+    {
+      "epoch": 1.688235294117647,
+      "grad_norm": 0.010491623543202877,
+      "learning_rate": 8.749019607843137e-05,
+      "loss": 0.032069438695907594,
+      "step": 2870
+    },
+    {
+      "epoch": 1.6941176470588235,
+      "grad_norm": 0.08409623056650162,
+      "learning_rate": 8.709803921568628e-05,
+      "loss": 0.0707733690738678,
+      "step": 2880
+    },
+    {
+      "epoch": 1.7,
+      "grad_norm": 0.3505629003047943,
+      "learning_rate": 8.670588235294118e-05,
+      "loss": 0.00785350576043129,
+      "step": 2890
+    },
+    {
+      "epoch": 1.7058823529411766,
+      "grad_norm": 0.00583046255633235,
+      "learning_rate": 8.631372549019609e-05,
+      "loss": 0.05285153985023498,
+      "step": 2900
+    },
+    {
+      "epoch": 1.711764705882353,
+      "grad_norm": 0.005758538376539946,
+      "learning_rate": 8.592156862745099e-05,
+      "loss": 0.03593638241291046,
+      "step": 2910
+    },
+    {
+      "epoch": 1.7176470588235295,
+      "grad_norm": 0.04903476685285568,
+      "learning_rate": 8.55294117647059e-05,
+      "loss": 0.015528860688209533,
+      "step": 2920
+    },
+    {
+      "epoch": 1.723529411764706,
+      "grad_norm": 0.009182584472000599,
+      "learning_rate": 8.513725490196078e-05,
+      "loss": 0.016127565503120424,
+      "step": 2930
+    },
+    {
+      "epoch": 1.7294117647058824,
+      "grad_norm": 0.002334852935746312,
+      "learning_rate": 8.474509803921569e-05,
+      "loss": 0.030905908346176146,
+      "step": 2940
+    },
+    {
+      "epoch": 1.7352941176470589,
+      "grad_norm": 0.27579137682914734,
+      "learning_rate": 8.435294117647059e-05,
+      "loss": 0.038137003779411316,
+      "step": 2950
+    },
+    {
+      "epoch": 1.7411764705882353,
+      "grad_norm": 0.03673534467816353,
+      "learning_rate": 8.39607843137255e-05,
+      "loss": 0.004237490519881248,
+      "step": 2960
+    },
+    {
+      "epoch": 1.7470588235294118,
+      "grad_norm": 5.617798328399658,
+      "learning_rate": 8.35686274509804e-05,
+      "loss": 0.0286111444234848,
+      "step": 2970
+    },
+    {
+      "epoch": 1.7529411764705882,
+      "grad_norm": 0.0029075052589178085,
+      "learning_rate": 8.31764705882353e-05,
+      "loss": 0.07802658677101135,
+      "step": 2980
+    },
+    {
+      "epoch": 1.7588235294117647,
+      "grad_norm": 1.9302191734313965,
+      "learning_rate": 8.27843137254902e-05,
+      "loss": 0.06619505286216736,
+      "step": 2990
+    },
+    {
+      "epoch": 1.7647058823529411,
+      "grad_norm": 0.0049264212138950825,
+      "learning_rate": 8.23921568627451e-05,
+      "loss": 0.003911637142300606,
+      "step": 3000
+    },
+    {
+      "epoch": 1.7705882352941176,
+      "grad_norm": 0.08124825358390808,
+      "learning_rate": 8.2e-05,
+      "loss": 0.049767762422561646,
+      "step": 3010
+    },
+    {
+      "epoch": 1.776470588235294,
+      "grad_norm": 0.04015158861875534,
+      "learning_rate": 8.16078431372549e-05,
+      "loss": 0.039362430572509766,
+      "step": 3020
+    },
+    {
+      "epoch": 1.7823529411764705,
+      "grad_norm": 0.009305250830948353,
+      "learning_rate": 8.121568627450981e-05,
+      "loss": 0.03347316980361938,
+      "step": 3030
+    },
+    {
+      "epoch": 1.788235294117647,
+      "grad_norm": 0.24932274222373962,
+      "learning_rate": 8.082352941176471e-05,
+      "loss": 0.06300787329673767,
+      "step": 3040
+    },
+    {
+      "epoch": 1.7941176470588234,
+      "grad_norm": 0.9365462064743042,
+      "learning_rate": 8.043137254901962e-05,
+      "loss": 0.005259803682565689,
+      "step": 3050
+    },
+    {
+      "epoch": 1.8,
+      "grad_norm": 0.04040015488862991,
+      "learning_rate": 8.003921568627452e-05,
+      "loss": 0.0727255880832672,
+      "step": 3060
+    },
+    {
+      "epoch": 1.8058823529411765,
+      "grad_norm": 0.20401322841644287,
+      "learning_rate": 7.964705882352941e-05,
+      "loss": 0.12481111288070679,
+      "step": 3070
+    },
+    {
+      "epoch": 1.811764705882353,
+      "grad_norm": 0.047548212110996246,
+      "learning_rate": 7.925490196078432e-05,
+      "loss": 0.008852196484804153,
+      "step": 3080
+    },
+    {
+      "epoch": 1.8176470588235294,
+      "grad_norm": 0.09739190340042114,
+      "learning_rate": 7.886274509803921e-05,
+      "loss": 0.013921891152858735,
+      "step": 3090
+    },
+    {
+      "epoch": 1.8235294117647058,
+      "grad_norm": 0.04188184440135956,
+      "learning_rate": 7.847058823529412e-05,
+      "loss": 0.03531156182289123,
+      "step": 3100
+    },
+    {
+      "epoch": 1.8294117647058825,
+      "grad_norm": 0.04765227064490318,
+      "learning_rate": 7.807843137254902e-05,
+      "loss": 0.0103824220597744,
+      "step": 3110
+    },
+    {
+      "epoch": 1.835294117647059,
+      "grad_norm": 0.2406538426876068,
+      "learning_rate": 7.768627450980393e-05,
+      "loss": 0.15724188089370728,
+      "step": 3120
+    },
+    {
+      "epoch": 1.8411764705882354,
+      "grad_norm": 5.855648040771484,
+      "learning_rate": 7.729411764705882e-05,
+      "loss": 0.0942530632019043,
+      "step": 3130
+    },
+    {
+      "epoch": 1.8470588235294119,
+      "grad_norm": 0.027591396123170853,
+      "learning_rate": 7.690196078431374e-05,
+      "loss": 0.007234179973602295,
+      "step": 3140
+    },
+    {
+      "epoch": 1.8529411764705883,
+      "grad_norm": 0.015235557220876217,
+      "learning_rate": 7.650980392156863e-05,
+      "loss": 0.04118633568286896,
+      "step": 3150
+    },
+    {
+      "epoch": 1.8588235294117648,
+      "grad_norm": 1.8588212728500366,
+      "learning_rate": 7.611764705882354e-05,
+      "loss": 0.016291515529155733,
+      "step": 3160
+    },
+    {
+      "epoch": 1.8647058823529412,
+      "grad_norm": 3.833609104156494,
+      "learning_rate": 7.572549019607843e-05,
+      "loss": 0.02955688238143921,
+      "step": 3170
+    },
+    {
+      "epoch": 1.8705882352941177,
+      "grad_norm": 1.548680305480957,
+      "learning_rate": 7.533333333333334e-05,
+      "loss": 0.02775830924510956,
+      "step": 3180
+    },
+    {
+      "epoch": 1.8764705882352941,
+      "grad_norm": 0.1539326310157776,
+      "learning_rate": 7.494117647058824e-05,
+      "loss": 0.01504003256559372,
+      "step": 3190
+    },
+    {
+      "epoch": 1.8823529411764706,
+      "grad_norm": 0.009701904840767384,
+      "learning_rate": 7.454901960784313e-05,
+      "loss": 0.037118193507194516,
+      "step": 3200
+    },
+    {
+      "epoch": 1.888235294117647,
+      "grad_norm": 0.0187819916754961,
+      "learning_rate": 7.415686274509805e-05,
+      "loss": 0.010768245905637741,
+      "step": 3210
+    },
+    {
+      "epoch": 1.8941176470588235,
+      "grad_norm": 0.02519003115594387,
+      "learning_rate": 7.376470588235294e-05,
+      "loss": 0.042407462000846864,
+      "step": 3220
+    },
+    {
+      "epoch": 1.9,
+      "grad_norm": 3.355506658554077,
+      "learning_rate": 7.337254901960785e-05,
+      "loss": 0.0523089587688446,
+      "step": 3230
+    },
+    {
+      "epoch": 1.9058823529411764,
+      "grad_norm": 1.4742717742919922,
+      "learning_rate": 7.298039215686275e-05,
+      "loss": 0.04612267315387726,
+      "step": 3240
+    },
+    {
+      "epoch": 1.9117647058823528,
+      "grad_norm": 4.523307800292969,
+      "learning_rate": 7.258823529411765e-05,
+      "loss": 0.024737949669361114,
+      "step": 3250
+    },
+    {
+      "epoch": 1.9176470588235293,
+      "grad_norm": 0.014893022365868092,
+      "learning_rate": 7.219607843137255e-05,
+      "loss": 0.10542804002761841,
+      "step": 3260
+    },
+    {
+      "epoch": 1.923529411764706,
+      "grad_norm": 0.1594671905040741,
+      "learning_rate": 7.180392156862746e-05,
+      "loss": 0.006839567422866821,
+      "step": 3270
+    },
+    {
+      "epoch": 1.9294117647058824,
+      "grad_norm": 0.07281392067670822,
+      "learning_rate": 7.141176470588235e-05,
+      "loss": 0.032911264896392824,
+      "step": 3280
+    },
+    {
+      "epoch": 1.9352941176470588,
+      "grad_norm": 0.12037048488855362,
+      "learning_rate": 7.101960784313727e-05,
+      "loss": 0.04403464496135712,
+      "step": 3290
+    },
+    {
+      "epoch": 1.9411764705882353,
+      "grad_norm": 0.46801379323005676,
+      "learning_rate": 7.062745098039216e-05,
+      "loss": 0.0025000225752592085,
+      "step": 3300
+    },
+    {
+      "epoch": 1.9470588235294117,
+      "grad_norm": 0.0035815690644085407,
+      "learning_rate": 7.023529411764706e-05,
+      "loss": 0.0006198661867529154,
+      "step": 3310
+    },
+    {
+      "epoch": 1.9529411764705882,
+      "grad_norm": 0.02339448593556881,
+      "learning_rate": 6.984313725490197e-05,
+      "loss": 0.02695143222808838,
+      "step": 3320
+    },
+    {
+      "epoch": 1.9588235294117649,
+      "grad_norm": 0.022110486403107643,
+      "learning_rate": 6.945098039215686e-05,
+      "loss": 0.03771721124649048,
+      "step": 3330
+    },
+    {
+      "epoch": 1.9647058823529413,
+      "grad_norm": 0.6860880255699158,
+      "learning_rate": 6.905882352941177e-05,
+      "loss": 0.02944064438343048,
+      "step": 3340
+    },
+    {
+      "epoch": 1.9705882352941178,
+      "grad_norm": 0.10796581953763962,
+      "learning_rate": 6.866666666666666e-05,
+      "loss": 0.011810501664876938,
+      "step": 3350
+    },
+    {
+      "epoch": 1.9764705882352942,
+      "grad_norm": 0.4359405040740967,
+      "learning_rate": 6.827450980392158e-05,
+      "loss": 0.006068695709109307,
+      "step": 3360
+    },
+    {
+      "epoch": 1.9823529411764707,
+      "grad_norm": 1.829695463180542,
+      "learning_rate": 6.788235294117647e-05,
+      "loss": 0.03863932192325592,
+      "step": 3370
+    },
+    {
+      "epoch": 1.988235294117647,
+      "grad_norm": 3.4061267375946045,
+      "learning_rate": 6.749019607843138e-05,
+      "loss": 0.021592351794242858,
+      "step": 3380
+    },
+    {
+      "epoch": 1.9941176470588236,
+      "grad_norm": 0.003634863765910268,
+      "learning_rate": 6.709803921568628e-05,
+      "loss": 0.03054191470146179,
+      "step": 3390
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.8191121220588684,
+      "learning_rate": 6.670588235294118e-05,
+      "loss": 0.029895269870758058,
+      "step": 3400
+    },
+    {
+      "epoch": 2.0,
+      "eval_accuracy": 0.9827083333333333,
+      "eval_loss": 0.05821743607521057,
+      "eval_runtime": 68.3903,
+      "eval_samples_per_second": 70.185,
+      "eval_steps_per_second": 4.387,
+      "step": 3400
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 5100,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 288590756436096.0,
+  "train_batch_size": 16,
+  "trial_name": null,
+  "trial_params": null
+}

training/training-checkpoints/checkpoint-3400/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:404392ce274606d42167ab282bdb788e763b38ede0704b0120ec45be3aba6567
+size 5201

training/training-checkpoints/checkpoint-5100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: distilbert-base-uncased
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

training/training-checkpoints/checkpoint-5100/adapter_config.json

@@ -0,0 +1,32 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "distilbert-base-uncased",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": [
+    "classifier",
+    "score"
+  ],
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_lin",
+    "q_lin"
+  ],
+  "task_type": "SEQ_CLS",
+  "use_dora": false,
+  "use_rslora": false
+}

training/training-checkpoints/checkpoint-5100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:539235288638662cf62a8196575c95b797586db7c157251881af32fe95d1b935
+size 2962192

training/training-checkpoints/checkpoint-5100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25e8451ccb6ca743e82b27ba033e735ac3a79d7e573865b6810c9ae563fc8968
+size 5940315

training/training-checkpoints/checkpoint-5100/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e0fd4366ad299ac64bf657f956c3615de203baba7f7b3c3d54214d7eccf7f4b
+size 14455

training/training-checkpoints/checkpoint-5100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2df167e9de46c439c0f82b9e4834e23dbcaf6d8f75d374ef1e941952acfae2cc
+size 1465

training/training-checkpoints/checkpoint-5100/tokenizer_config.json

@@ -0,0 +1,15 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

training/training-checkpoints/checkpoint-5100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:404392ce274606d42167ab282bdb788e763b38ede0704b0120ec45be3aba6567
+size 5201