chore: nest checkpoint-7000 under dapt/

dapt/checkpoint-7000/README.md

@@ -0,0 +1,206 @@
+---
+base_model: Itau-Unibanco/NorBERTo-base
+library_name: peft
+tags:
+- base_model:adapter:Itau-Unibanco/NorBERTo-base
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

dapt/checkpoint-7000/adapter_config.json

@@ -0,0 +1,46 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Itau-Unibanco/NorBERTo-base",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "Wo",
+    "Wi",
+    "Wqkv"
+  ],
+  "target_parameters": null,
+  "task_type": "FEATURE_EXTRACTION",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

dapt/checkpoint-7000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:56de0c13556be499c4dc34b55d1bf1e32c8153277b2161ca178a5693fd9c90e8
+size 13538528

dapt/checkpoint-7000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df9b373e49f397eb4a7d2413be75c1e20264648de021f13857f867aff813ed7b
+size 27184331

dapt/checkpoint-7000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ff44622004c05ff0be8c1184aaa0885b9c19bfb826c23d9fee09fad587278ce
+size 14645

dapt/checkpoint-7000/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bb71a359f9dcd201ece0cf763789dbde8fa6be1b2e983bcd28bdf4a50b529e44
+size 1383

dapt/checkpoint-7000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9cba50e9a463d015dfc3614c80b88b522188a9adc24bcabfc53e7c2acc2f9047
+size 1465

dapt/checkpoint-7000/tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "[SEP]",
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 8192,
+  "pad_token": "[PAD]",
+  "padding_side": "right",
+  "tokenizer_class": "TokenizersBackend",
+  "truncation_side": "right",
+  "unk_token": "[UNK]"
+}

dapt/checkpoint-7000/trainer_state.json

@@ -0,0 +1,1126 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.2035785288270378,
+  "eval_steps": 500,
+  "global_step": 7000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.008597066251141797,
+      "grad_norm": 0.3461540639400482,
+      "learning_rate": 4.9000000000000005e-06,
+      "loss": 0.7137960815429687,
+      "step": 50
+    },
+    {
+      "epoch": 0.017194132502283594,
+      "grad_norm": 0.23365077376365662,
+      "learning_rate": 9.900000000000002e-06,
+      "loss": 0.6752699279785156,
+      "step": 100
+    },
+    {
+      "epoch": 0.025791198753425393,
+      "grad_norm": 0.18823082745075226,
+      "learning_rate": 1.49e-05,
+      "loss": 0.6456453704833984,
+      "step": 150
+    },
+    {
+      "epoch": 0.03438826500456719,
+      "grad_norm": 0.23714424669742584,
+      "learning_rate": 1.9900000000000003e-05,
+      "loss": 0.6078555679321289,
+      "step": 200
+    },
+    {
+      "epoch": 0.04298533125570899,
+      "grad_norm": 0.16429093480110168,
+      "learning_rate": 2.4900000000000002e-05,
+      "loss": 0.6051961135864258,
+      "step": 250
+    },
+    {
+      "epoch": 0.051582397506850786,
+      "grad_norm": 0.16205623745918274,
+      "learning_rate": 2.9900000000000002e-05,
+      "loss": 0.5889498901367187,
+      "step": 300
+    },
+    {
+      "epoch": 0.06017946375799259,
+      "grad_norm": 0.1743771731853485,
+      "learning_rate": 3.49e-05,
+      "loss": 0.5702997970581055,
+      "step": 350
+    },
+    {
+      "epoch": 0.06877653000913438,
+      "grad_norm": 0.15920504927635193,
+      "learning_rate": 3.99e-05,
+      "loss": 0.5728500747680664,
+      "step": 400
+    },
+    {
+      "epoch": 0.07737359626027618,
+      "grad_norm": 0.18150873482227325,
+      "learning_rate": 4.49e-05,
+      "loss": 0.5552708053588867,
+      "step": 450
+    },
+    {
+      "epoch": 0.08597066251141798,
+      "grad_norm": 0.2273012101650238,
+      "learning_rate": 4.99e-05,
+      "loss": 0.5470176696777344,
+      "step": 500
+    },
+    {
+      "epoch": 0.08597066251141798,
+      "eval_loss": 0.5438547134399414,
+      "eval_runtime": 361.1497,
+      "eval_samples_per_second": 5.206,
+      "eval_steps_per_second": 1.301,
+      "step": 500
+    },
+    {
+      "epoch": 0.09456772876255978,
+      "grad_norm": 0.1610037386417389,
+      "learning_rate": 4.970209143968872e-05,
+      "loss": 0.5506169891357422,
+      "step": 550
+    },
+    {
+      "epoch": 0.10316479501370157,
+      "grad_norm": 0.18427467346191406,
+      "learning_rate": 4.9398103112840474e-05,
+      "loss": 0.538709716796875,
+      "step": 600
+    },
+    {
+      "epoch": 0.11176186126484337,
+      "grad_norm": 0.17865458130836487,
+      "learning_rate": 4.9094114785992216e-05,
+      "loss": 0.5439010620117187,
+      "step": 650
+    },
+    {
+      "epoch": 0.12035892751598518,
+      "grad_norm": 0.20380577445030212,
+      "learning_rate": 4.879012645914397e-05,
+      "loss": 0.5345694732666015,
+      "step": 700
+    },
+    {
+      "epoch": 0.12895599376712696,
+      "grad_norm": 0.21836704015731812,
+      "learning_rate": 4.8486138132295725e-05,
+      "loss": 0.5367267608642579,
+      "step": 750
+    },
+    {
+      "epoch": 0.13755306001826875,
+      "grad_norm": 0.21373888850212097,
+      "learning_rate": 4.818214980544747e-05,
+      "loss": 0.5393967819213867,
+      "step": 800
+    },
+    {
+      "epoch": 0.14615012626941057,
+      "grad_norm": 0.17039674520492554,
+      "learning_rate": 4.787816147859923e-05,
+      "loss": 0.5188388061523438,
+      "step": 850
+    },
+    {
+      "epoch": 0.15474719252055236,
+      "grad_norm": 0.1947856992483139,
+      "learning_rate": 4.7574173151750976e-05,
+      "loss": 0.5279138565063477,
+      "step": 900
+    },
+    {
+      "epoch": 0.16334425877169417,
+      "grad_norm": 0.1957385241985321,
+      "learning_rate": 4.7270184824902724e-05,
+      "loss": 0.5388415908813476,
+      "step": 950
+    },
+    {
+      "epoch": 0.17194132502283596,
+      "grad_norm": 0.19011224806308746,
+      "learning_rate": 4.696619649805448e-05,
+      "loss": 0.527779426574707,
+      "step": 1000
+    },
+    {
+      "epoch": 0.17194132502283596,
+      "eval_loss": 0.5264737010002136,
+      "eval_runtime": 349.4761,
+      "eval_samples_per_second": 5.379,
+      "eval_steps_per_second": 1.345,
+      "step": 1000
+    },
+    {
+      "epoch": 0.18053839127397775,
+      "grad_norm": 0.17350342869758606,
+      "learning_rate": 4.6662208171206227e-05,
+      "loss": 0.5216117858886719,
+      "step": 1050
+    },
+    {
+      "epoch": 0.18913545752511957,
+      "grad_norm": 0.17218773066997528,
+      "learning_rate": 4.635821984435798e-05,
+      "loss": 0.5192420196533203,
+      "step": 1100
+    },
+    {
+      "epoch": 0.19773252377626135,
+      "grad_norm": 0.1787382811307907,
+      "learning_rate": 4.605423151750973e-05,
+      "loss": 0.5193338394165039,
+      "step": 1150
+    },
+    {
+      "epoch": 0.20632959002740314,
+      "grad_norm": 0.17407308518886566,
+      "learning_rate": 4.575024319066148e-05,
+      "loss": 0.529417610168457,
+      "step": 1200
+    },
+    {
+      "epoch": 0.21492665627854496,
+      "grad_norm": 0.18545730412006378,
+      "learning_rate": 4.544625486381323e-05,
+      "loss": 0.5278170394897461,
+      "step": 1250
+    },
+    {
+      "epoch": 0.22352372252968675,
+      "grad_norm": 0.19537468254566193,
+      "learning_rate": 4.514226653696498e-05,
+      "loss": 0.5204765319824218,
+      "step": 1300
+    },
+    {
+      "epoch": 0.23212078878082854,
+      "grad_norm": 0.2441568374633789,
+      "learning_rate": 4.4838278210116735e-05,
+      "loss": 0.5202705001831055,
+      "step": 1350
+    },
+    {
+      "epoch": 0.24071785503197035,
+      "grad_norm": 0.23745512962341309,
+      "learning_rate": 4.453428988326849e-05,
+      "loss": 0.5292264556884766,
+      "step": 1400
+    },
+    {
+      "epoch": 0.24931492128311214,
+      "grad_norm": 0.1980402022600174,
+      "learning_rate": 4.423030155642023e-05,
+      "loss": 0.5218988800048828,
+      "step": 1450
+    },
+    {
+      "epoch": 0.25791198753425393,
+      "grad_norm": 0.2273419052362442,
+      "learning_rate": 4.3926313229571986e-05,
+      "loss": 0.5181735992431641,
+      "step": 1500
+    },
+    {
+      "epoch": 0.25791198753425393,
+      "eval_loss": 0.5230017304420471,
+      "eval_runtime": 375.9124,
+      "eval_samples_per_second": 5.001,
+      "eval_steps_per_second": 1.25,
+      "step": 1500
+    },
+    {
+      "epoch": 0.26650905378539574,
+      "grad_norm": 0.1970907747745514,
+      "learning_rate": 4.362232490272374e-05,
+      "loss": 0.5135591125488281,
+      "step": 1550
+    },
+    {
+      "epoch": 0.2751061200365375,
+      "grad_norm": 0.2051905244588852,
+      "learning_rate": 4.331833657587549e-05,
+      "loss": 0.5052785873413086,
+      "step": 1600
+    },
+    {
+      "epoch": 0.2837031862876793,
+      "grad_norm": 0.19686760008335114,
+      "learning_rate": 4.301434824902724e-05,
+      "loss": 0.5177907943725586,
+      "step": 1650
+    },
+    {
+      "epoch": 0.29230025253882114,
+      "grad_norm": 0.20986631512641907,
+      "learning_rate": 4.271035992217899e-05,
+      "loss": 0.5072374725341797,
+      "step": 1700
+    },
+    {
+      "epoch": 0.3008973187899629,
+      "grad_norm": 0.2033650428056717,
+      "learning_rate": 4.240637159533074e-05,
+      "loss": 0.5099851989746094,
+      "step": 1750
+    },
+    {
+      "epoch": 0.3094943850411047,
+      "grad_norm": 0.20672816038131714,
+      "learning_rate": 4.2102383268482494e-05,
+      "loss": 0.5082115936279297,
+      "step": 1800
+    },
+    {
+      "epoch": 0.31809145129224653,
+      "grad_norm": 0.1826789379119873,
+      "learning_rate": 4.179839494163424e-05,
+      "loss": 0.5152957153320312,
+      "step": 1850
+    },
+    {
+      "epoch": 0.32668851754338835,
+      "grad_norm": 0.24080227315425873,
+      "learning_rate": 4.1494406614785997e-05,
+      "loss": 0.516766014099121,
+      "step": 1900
+    },
+    {
+      "epoch": 0.3352855837945301,
+      "grad_norm": 0.18921689689159393,
+      "learning_rate": 4.1190418287937745e-05,
+      "loss": 0.5188713455200196,
+      "step": 1950
+    },
+    {
+      "epoch": 0.3438826500456719,
+      "grad_norm": 0.21723054349422455,
+      "learning_rate": 4.088642996108949e-05,
+      "loss": 0.508725471496582,
+      "step": 2000
+    },
+    {
+      "epoch": 0.3438826500456719,
+      "eval_loss": 0.5111244916915894,
+      "eval_runtime": 375.1647,
+      "eval_samples_per_second": 5.011,
+      "eval_steps_per_second": 1.253,
+      "step": 2000
+    },
+    {
+      "epoch": 0.35247971629681374,
+      "grad_norm": 0.2504056692123413,
+      "learning_rate": 4.058244163424125e-05,
+      "loss": 0.5189783477783203,
+      "step": 2050
+    },
+    {
+      "epoch": 0.3610767825479555,
+      "grad_norm": 0.20812708139419556,
+      "learning_rate": 4.0278453307393e-05,
+      "loss": 0.5200382995605469,
+      "step": 2100
+    },
+    {
+      "epoch": 0.3696738487990973,
+      "grad_norm": 0.21136048436164856,
+      "learning_rate": 3.997446498054475e-05,
+      "loss": 0.5042448806762695,
+      "step": 2150
+    },
+    {
+      "epoch": 0.37827091505023913,
+      "grad_norm": 0.2592664957046509,
+      "learning_rate": 3.96704766536965e-05,
+      "loss": 0.511298828125,
+      "step": 2200
+    },
+    {
+      "epoch": 0.3868679813013809,
+      "grad_norm": 0.20483101904392242,
+      "learning_rate": 3.936648832684825e-05,
+      "loss": 0.5088952255249023,
+      "step": 2250
+    },
+    {
+      "epoch": 0.3954650475525227,
+      "grad_norm": 0.2416965663433075,
+      "learning_rate": 3.90625e-05,
+      "loss": 0.5220498275756836,
+      "step": 2300
+    },
+    {
+      "epoch": 0.4040621138036645,
+      "grad_norm": 0.2950873374938965,
+      "learning_rate": 3.8758511673151756e-05,
+      "loss": 0.5161582946777343,
+      "step": 2350
+    },
+    {
+      "epoch": 0.4126591800548063,
+      "grad_norm": 0.22417296469211578,
+      "learning_rate": 3.8454523346303504e-05,
+      "loss": 0.5022587966918945,
+      "step": 2400
+    },
+    {
+      "epoch": 0.4212562463059481,
+      "grad_norm": 0.24061641097068787,
+      "learning_rate": 3.815053501945525e-05,
+      "loss": 0.5108227157592773,
+      "step": 2450
+    },
+    {
+      "epoch": 0.4298533125570899,
+      "grad_norm": 0.21841247379779816,
+      "learning_rate": 3.7846546692607006e-05,
+      "loss": 0.5098703002929688,
+      "step": 2500
+    },
+    {
+      "epoch": 0.4298533125570899,
+      "eval_loss": 0.5066602826118469,
+      "eval_runtime": 366.5148,
+      "eval_samples_per_second": 5.129,
+      "eval_steps_per_second": 1.282,
+      "step": 2500
+    },
+    {
+      "epoch": 0.4384503788082317,
+      "grad_norm": 0.2083503156900406,
+      "learning_rate": 3.7542558365758754e-05,
+      "loss": 0.49932518005371096,
+      "step": 2550
+    },
+    {
+      "epoch": 0.4470474450593735,
+      "grad_norm": 0.21678738296031952,
+      "learning_rate": 3.723857003891051e-05,
+      "loss": 0.5064985275268554,
+      "step": 2600
+    },
+    {
+      "epoch": 0.4556445113105153,
+      "grad_norm": 0.24412305653095245,
+      "learning_rate": 3.6934581712062264e-05,
+      "loss": 0.5066806411743164,
+      "step": 2650
+    },
+    {
+      "epoch": 0.46424157756165707,
+      "grad_norm": 0.26758745312690735,
+      "learning_rate": 3.6630593385214005e-05,
+      "loss": 0.5179504013061523,
+      "step": 2700
+    },
+    {
+      "epoch": 0.4728386438127989,
+      "grad_norm": 0.2602377235889435,
+      "learning_rate": 3.632660505836576e-05,
+      "loss": 0.4992792510986328,
+      "step": 2750
+    },
+    {
+      "epoch": 0.4814357100639407,
+      "grad_norm": 0.2009514719247818,
+      "learning_rate": 3.602261673151751e-05,
+      "loss": 0.5105620574951172,
+      "step": 2800
+    },
+    {
+      "epoch": 0.49003277631508246,
+      "grad_norm": 0.21059006452560425,
+      "learning_rate": 3.571862840466926e-05,
+      "loss": 0.508187255859375,
+      "step": 2850
+    },
+    {
+      "epoch": 0.4986298425662243,
+      "grad_norm": 0.2044428288936615,
+      "learning_rate": 3.541464007782102e-05,
+      "loss": 0.5071085739135742,
+      "step": 2900
+    },
+    {
+      "epoch": 0.5072269088173661,
+      "grad_norm": 0.2184259295463562,
+      "learning_rate": 3.511065175097276e-05,
+      "loss": 0.5003342056274414,
+      "step": 2950
+    },
+    {
+      "epoch": 0.5158239750685079,
+      "grad_norm": 0.22149892151355743,
+      "learning_rate": 3.480666342412451e-05,
+      "loss": 0.5028779983520508,
+      "step": 3000
+    },
+    {
+      "epoch": 0.5158239750685079,
+      "eval_loss": 0.5020000338554382,
+      "eval_runtime": 346.4065,
+      "eval_samples_per_second": 5.427,
+      "eval_steps_per_second": 1.357,
+      "step": 3000
+    },
+    {
+      "epoch": 0.5244210413196496,
+      "grad_norm": 0.25263792276382446,
+      "learning_rate": 3.450267509727627e-05,
+      "loss": 0.4868510818481445,
+      "step": 3050
+    },
+    {
+      "epoch": 0.5330181075707915,
+      "grad_norm": 0.1957710087299347,
+      "learning_rate": 3.4198686770428016e-05,
+      "loss": 0.5041673660278321,
+      "step": 3100
+    },
+    {
+      "epoch": 0.5416151738219332,
+      "grad_norm": 0.2603848874568939,
+      "learning_rate": 3.389469844357977e-05,
+      "loss": 0.5014479827880859,
+      "step": 3150
+    },
+    {
+      "epoch": 0.550212240073075,
+      "grad_norm": 0.17874036729335785,
+      "learning_rate": 3.359071011673152e-05,
+      "loss": 0.505292739868164,
+      "step": 3200
+    },
+    {
+      "epoch": 0.5588093063242169,
+      "grad_norm": 0.24125388264656067,
+      "learning_rate": 3.328672178988327e-05,
+      "loss": 0.514274787902832,
+      "step": 3250
+    },
+    {
+      "epoch": 0.5674063725753586,
+      "grad_norm": 0.22477301955223083,
+      "learning_rate": 3.298273346303502e-05,
+      "loss": 0.5066662979125977,
+      "step": 3300
+    },
+    {
+      "epoch": 0.5760034388265004,
+      "grad_norm": 0.23808450996875763,
+      "learning_rate": 3.267874513618677e-05,
+      "loss": 0.49730167388916013,
+      "step": 3350
+    },
+    {
+      "epoch": 0.5846005050776423,
+      "grad_norm": 0.2184080332517624,
+      "learning_rate": 3.2374756809338524e-05,
+      "loss": 0.5056488800048828,
+      "step": 3400
+    },
+    {
+      "epoch": 0.593197571328784,
+      "grad_norm": 0.25470733642578125,
+      "learning_rate": 3.207076848249028e-05,
+      "loss": 0.4887243270874023,
+      "step": 3450
+    },
+    {
+      "epoch": 0.6017946375799258,
+      "grad_norm": 0.2762533128261566,
+      "learning_rate": 3.176678015564202e-05,
+      "loss": 0.4995370101928711,
+      "step": 3500
+    },
+    {
+      "epoch": 0.6017946375799258,
+      "eval_loss": 0.5056677460670471,
+      "eval_runtime": 348.3925,
+      "eval_samples_per_second": 5.396,
+      "eval_steps_per_second": 1.349,
+      "step": 3500
+    },
+    {
+      "epoch": 0.6103917038310677,
+      "grad_norm": 0.2317754477262497,
+      "learning_rate": 3.1462791828793775e-05,
+      "loss": 0.5123172760009765,
+      "step": 3550
+    },
+    {
+      "epoch": 0.6189887700822094,
+      "grad_norm": 0.22287645936012268,
+      "learning_rate": 3.115880350194553e-05,
+      "loss": 0.5003259658813477,
+      "step": 3600
+    },
+    {
+      "epoch": 0.6275858363333513,
+      "grad_norm": 0.20770233869552612,
+      "learning_rate": 3.085481517509728e-05,
+      "loss": 0.4997048187255859,
+      "step": 3650
+    },
+    {
+      "epoch": 0.6361829025844931,
+      "grad_norm": 0.22887976467609406,
+      "learning_rate": 3.055082684824903e-05,
+      "loss": 0.493292350769043,
+      "step": 3700
+    },
+    {
+      "epoch": 0.6447799688356348,
+      "grad_norm": 0.24185964465141296,
+      "learning_rate": 3.0246838521400777e-05,
+      "loss": 0.4899324417114258,
+      "step": 3750
+    },
+    {
+      "epoch": 0.6533770350867767,
+      "grad_norm": 0.2602291703224182,
+      "learning_rate": 2.994285019455253e-05,
+      "loss": 0.5039303207397461,
+      "step": 3800
+    },
+    {
+      "epoch": 0.6619741013379185,
+      "grad_norm": 0.26305103302001953,
+      "learning_rate": 2.9638861867704283e-05,
+      "loss": 0.49886848449707033,
+      "step": 3850
+    },
+    {
+      "epoch": 0.6705711675890602,
+      "grad_norm": 0.2317502647638321,
+      "learning_rate": 2.9334873540856035e-05,
+      "loss": 0.5039469909667968,
+      "step": 3900
+    },
+    {
+      "epoch": 0.6791682338402021,
+      "grad_norm": 0.28173670172691345,
+      "learning_rate": 2.9030885214007786e-05,
+      "loss": 0.492442626953125,
+      "step": 3950
+    },
+    {
+      "epoch": 0.6877653000913438,
+      "grad_norm": 0.2428692728281021,
+      "learning_rate": 2.872689688715953e-05,
+      "loss": 0.49990890502929686,
+      "step": 4000
+    },
+    {
+      "epoch": 0.6877653000913438,
+      "eval_loss": 0.5016358494758606,
+      "eval_runtime": 347.7511,
+      "eval_samples_per_second": 5.406,
+      "eval_steps_per_second": 1.352,
+      "step": 4000
+    },
+    {
+      "epoch": 0.6963623663424856,
+      "grad_norm": 0.24288879334926605,
+      "learning_rate": 2.8422908560311285e-05,
+      "loss": 0.5111994552612305,
+      "step": 4050
+    },
+    {
+      "epoch": 0.7049594325936275,
+      "grad_norm": 0.24350792169570923,
+      "learning_rate": 2.8118920233463037e-05,
+      "loss": 0.49502365112304686,
+      "step": 4100
+    },
+    {
+      "epoch": 0.7135564988447692,
+      "grad_norm": 0.2444242238998413,
+      "learning_rate": 2.7814931906614788e-05,
+      "loss": 0.5021502685546875,
+      "step": 4150
+    },
+    {
+      "epoch": 0.722153565095911,
+      "grad_norm": 0.22761669754981995,
+      "learning_rate": 2.751094357976654e-05,
+      "loss": 0.5036661911010742,
+      "step": 4200
+    },
+    {
+      "epoch": 0.7307506313470529,
+      "grad_norm": 0.22969037294387817,
+      "learning_rate": 2.7206955252918288e-05,
+      "loss": 0.5093637466430664,
+      "step": 4250
+    },
+    {
+      "epoch": 0.7393476975981946,
+      "grad_norm": 0.22184143960475922,
+      "learning_rate": 2.690296692607004e-05,
+      "loss": 0.49653377532958987,
+      "step": 4300
+    },
+    {
+      "epoch": 0.7479447638493364,
+      "grad_norm": 0.26576149463653564,
+      "learning_rate": 2.659897859922179e-05,
+      "loss": 0.5005849838256836,
+      "step": 4350
+    },
+    {
+      "epoch": 0.7565418301004783,
+      "grad_norm": 0.27365559339523315,
+      "learning_rate": 2.629499027237354e-05,
+      "loss": 0.48942367553710936,
+      "step": 4400
+    },
+    {
+      "epoch": 0.76513889635162,
+      "grad_norm": 0.3374887704849243,
+      "learning_rate": 2.5991001945525296e-05,
+      "loss": 0.5014659881591796,
+      "step": 4450
+    },
+    {
+      "epoch": 0.7737359626027618,
+      "grad_norm": 0.24214555323123932,
+      "learning_rate": 2.568701361867704e-05,
+      "loss": 0.49243038177490234,
+      "step": 4500
+    },
+    {
+      "epoch": 0.7737359626027618,
+      "eval_loss": 0.4991719424724579,
+      "eval_runtime": 354.9773,
+      "eval_samples_per_second": 5.296,
+      "eval_steps_per_second": 1.324,
+      "step": 4500
+    },
+    {
+      "epoch": 0.7823330288539037,
+      "grad_norm": 0.265796422958374,
+      "learning_rate": 2.5383025291828792e-05,
+      "loss": 0.4977854537963867,
+      "step": 4550
+    },
+    {
+      "epoch": 0.7909300951050454,
+      "grad_norm": 0.24726542830467224,
+      "learning_rate": 2.5079036964980547e-05,
+      "loss": 0.49198715209960936,
+      "step": 4600
+    },
+    {
+      "epoch": 0.7995271613561872,
+      "grad_norm": 0.22770538926124573,
+      "learning_rate": 2.47750486381323e-05,
+      "loss": 0.4971847915649414,
+      "step": 4650
+    },
+    {
+      "epoch": 0.808124227607329,
+      "grad_norm": 0.2174365520477295,
+      "learning_rate": 2.4471060311284046e-05,
+      "loss": 0.503781967163086,
+      "step": 4700
+    },
+    {
+      "epoch": 0.8167212938584708,
+      "grad_norm": 0.25075647234916687,
+      "learning_rate": 2.4167071984435798e-05,
+      "loss": 0.4942308807373047,
+      "step": 4750
+    },
+    {
+      "epoch": 0.8253183601096126,
+      "grad_norm": 0.2569100260734558,
+      "learning_rate": 2.3863083657587553e-05,
+      "loss": 0.4989316940307617,
+      "step": 4800
+    },
+    {
+      "epoch": 0.8339154263607544,
+      "grad_norm": 0.23158438503742218,
+      "learning_rate": 2.35590953307393e-05,
+      "loss": 0.49727272033691405,
+      "step": 4850
+    },
+    {
+      "epoch": 0.8425124926118962,
+      "grad_norm": 0.2155766785144806,
+      "learning_rate": 2.3255107003891052e-05,
+      "loss": 0.4887562942504883,
+      "step": 4900
+    },
+    {
+      "epoch": 0.851109558863038,
+      "grad_norm": 0.29494816064834595,
+      "learning_rate": 2.29511186770428e-05,
+      "loss": 0.4842983245849609,
+      "step": 4950
+    },
+    {
+      "epoch": 0.8597066251141798,
+      "grad_norm": 0.22478900849819183,
+      "learning_rate": 2.2647130350194555e-05,
+      "loss": 0.4985869598388672,
+      "step": 5000
+    },
+    {
+      "epoch": 0.8597066251141798,
+      "eval_loss": 0.49180179834365845,
+      "eval_runtime": 354.801,
+      "eval_samples_per_second": 5.299,
+      "eval_steps_per_second": 1.325,
+      "step": 5000
+    },
+    {
+      "epoch": 0.8683036913653216,
+      "grad_norm": 0.26430022716522217,
+      "learning_rate": 2.2343142023346306e-05,
+      "loss": 0.5041688919067383,
+      "step": 5050
+    },
+    {
+      "epoch": 0.8769007576164634,
+      "grad_norm": 0.23191678524017334,
+      "learning_rate": 2.2039153696498054e-05,
+      "loss": 0.5031358337402344,
+      "step": 5100
+    },
+    {
+      "epoch": 0.8854978238676052,
+      "grad_norm": 0.24605002999305725,
+      "learning_rate": 2.1735165369649805e-05,
+      "loss": 0.49686431884765625,
+      "step": 5150
+    },
+    {
+      "epoch": 0.894094890118747,
+      "grad_norm": 0.28351929783821106,
+      "learning_rate": 2.1431177042801557e-05,
+      "loss": 0.4975502395629883,
+      "step": 5200
+    },
+    {
+      "epoch": 0.9026919563698887,
+      "grad_norm": 0.2206832617521286,
+      "learning_rate": 2.1127188715953308e-05,
+      "loss": 0.49928531646728513,
+      "step": 5250
+    },
+    {
+      "epoch": 0.9112890226210306,
+      "grad_norm": 0.23365828394889832,
+      "learning_rate": 2.082320038910506e-05,
+      "loss": 0.5094524002075196,
+      "step": 5300
+    },
+    {
+      "epoch": 0.9198860888721724,
+      "grad_norm": 0.23105329275131226,
+      "learning_rate": 2.051921206225681e-05,
+      "loss": 0.5049754714965821,
+      "step": 5350
+    },
+    {
+      "epoch": 0.9284831551233141,
+      "grad_norm": 0.2554852366447449,
+      "learning_rate": 2.0215223735408562e-05,
+      "loss": 0.49230358123779294,
+      "step": 5400
+    },
+    {
+      "epoch": 0.937080221374456,
+      "grad_norm": 0.28842973709106445,
+      "learning_rate": 1.991123540856031e-05,
+      "loss": 0.48553138732910156,
+      "step": 5450
+    },
+    {
+      "epoch": 0.9456772876255978,
+      "grad_norm": 0.24151475727558136,
+      "learning_rate": 1.9607247081712062e-05,
+      "loss": 0.4950746154785156,
+      "step": 5500
+    },
+    {
+      "epoch": 0.9456772876255978,
+      "eval_loss": 0.49519777297973633,
+      "eval_runtime": 367.7895,
+      "eval_samples_per_second": 5.112,
+      "eval_steps_per_second": 1.278,
+      "step": 5500
+    },
+    {
+      "epoch": 0.9542743538767395,
+      "grad_norm": 0.23852616548538208,
+      "learning_rate": 1.9303258754863816e-05,
+      "loss": 0.49604484558105466,
+      "step": 5550
+    },
+    {
+      "epoch": 0.9628714201278814,
+      "grad_norm": 0.2663545608520508,
+      "learning_rate": 1.8999270428015564e-05,
+      "loss": 0.4933871841430664,
+      "step": 5600
+    },
+    {
+      "epoch": 0.9714684863790232,
+      "grad_norm": 0.24880719184875488,
+      "learning_rate": 1.8695282101167316e-05,
+      "loss": 0.48917724609375,
+      "step": 5650
+    },
+    {
+      "epoch": 0.9800655526301649,
+      "grad_norm": 0.2386282980442047,
+      "learning_rate": 1.8391293774319067e-05,
+      "loss": 0.4879060363769531,
+      "step": 5700
+    },
+    {
+      "epoch": 0.9886626188813068,
+      "grad_norm": 0.2540564239025116,
+      "learning_rate": 1.808730544747082e-05,
+      "loss": 0.49892501831054686,
+      "step": 5750
+    },
+    {
+      "epoch": 0.9972596851324486,
+      "grad_norm": 0.2618052065372467,
+      "learning_rate": 1.778331712062257e-05,
+      "loss": 0.4943923568725586,
+      "step": 5800
+    },
+    {
+      "epoch": 1.0058460050507765,
+      "grad_norm": 0.2744860351085663,
+      "learning_rate": 1.7479328793774318e-05,
+      "loss": 0.49327682495117187,
+      "step": 5850
+    },
+    {
+      "epoch": 1.0144430713019181,
+      "grad_norm": 0.265164852142334,
+      "learning_rate": 1.717534046692607e-05,
+      "loss": 0.48792911529541017,
+      "step": 5900
+    },
+    {
+      "epoch": 1.02304013755306,
+      "grad_norm": 0.22923576831817627,
+      "learning_rate": 1.6871352140077824e-05,
+      "loss": 0.4913317108154297,
+      "step": 5950
+    },
+    {
+      "epoch": 1.0316372038042019,
+      "grad_norm": 0.30609166622161865,
+      "learning_rate": 1.6567363813229572e-05,
+      "loss": 0.48412471771240234,
+      "step": 6000
+    },
+    {
+      "epoch": 1.0316372038042019,
+      "eval_loss": 0.4954085946083069,
+      "eval_runtime": 366.9675,
+      "eval_samples_per_second": 5.123,
+      "eval_steps_per_second": 1.281,
+      "step": 6000
+    },
+    {
+      "epoch": 1.0402342700553435,
+      "grad_norm": 0.24644337594509125,
+      "learning_rate": 1.6263375486381323e-05,
+      "loss": 0.4956199264526367,
+      "step": 6050
+    },
+    {
+      "epoch": 1.0488313363064854,
+      "grad_norm": 0.2369314730167389,
+      "learning_rate": 1.5959387159533075e-05,
+      "loss": 0.5049060440063476,
+      "step": 6100
+    },
+    {
+      "epoch": 1.0574284025576273,
+      "grad_norm": 0.27996891736984253,
+      "learning_rate": 1.5655398832684826e-05,
+      "loss": 0.49348834991455076,
+      "step": 6150
+    },
+    {
+      "epoch": 1.066025468808769,
+      "grad_norm": 0.26891449093818665,
+      "learning_rate": 1.5351410505836578e-05,
+      "loss": 0.49677490234375,
+      "step": 6200
+    },
+    {
+      "epoch": 1.0746225350599108,
+      "grad_norm": 0.23466609418392181,
+      "learning_rate": 1.5047422178988327e-05,
+      "loss": 0.5080591583251953,
+      "step": 6250
+    },
+    {
+      "epoch": 1.0832196013110527,
+      "grad_norm": 0.2662521004676819,
+      "learning_rate": 1.4743433852140079e-05,
+      "loss": 0.4966477966308594,
+      "step": 6300
+    },
+    {
+      "epoch": 1.0918166675621943,
+      "grad_norm": 0.2485560029745102,
+      "learning_rate": 1.4439445525291828e-05,
+      "loss": 0.49792095184326174,
+      "step": 6350
+    },
+    {
+      "epoch": 1.1004137338133362,
+      "grad_norm": 0.22780101001262665,
+      "learning_rate": 1.413545719844358e-05,
+      "loss": 0.49306358337402345,
+      "step": 6400
+    },
+    {
+      "epoch": 1.109010800064478,
+      "grad_norm": 0.23241789638996124,
+      "learning_rate": 1.3831468871595333e-05,
+      "loss": 0.48334613800048826,
+      "step": 6450
+    },
+    {
+      "epoch": 1.1176078663156197,
+      "grad_norm": 0.24886944890022278,
+      "learning_rate": 1.352748054474708e-05,
+      "loss": 0.5001045227050781,
+      "step": 6500
+    },
+    {
+      "epoch": 1.1176078663156197,
+      "eval_loss": 0.4932941794395447,
+      "eval_runtime": 338.6378,
+      "eval_samples_per_second": 5.552,
+      "eval_steps_per_second": 1.388,
+      "step": 6500
+    },
+    {
+      "epoch": 1.1262049325667616,
+      "grad_norm": 0.2723461985588074,
+      "learning_rate": 1.3223492217898834e-05,
+      "loss": 0.4947048187255859,
+      "step": 6550
+    },
+    {
+      "epoch": 1.1348019988179034,
+      "grad_norm": 0.24153922498226166,
+      "learning_rate": 1.2919503891050585e-05,
+      "loss": 0.480897102355957,
+      "step": 6600
+    },
+    {
+      "epoch": 1.143399065069045,
+      "grad_norm": 0.263067364692688,
+      "learning_rate": 1.2615515564202335e-05,
+      "loss": 0.4939377212524414,
+      "step": 6650
+    },
+    {
+      "epoch": 1.151996131320187,
+      "grad_norm": 0.26565349102020264,
+      "learning_rate": 1.2311527237354086e-05,
+      "loss": 0.4911102294921875,
+      "step": 6700
+    },
+    {
+      "epoch": 1.1605931975713288,
+      "grad_norm": 0.25430789589881897,
+      "learning_rate": 1.2007538910505838e-05,
+      "loss": 0.496657600402832,
+      "step": 6750
+    },
+    {
+      "epoch": 1.1691902638224705,
+      "grad_norm": 0.26413267850875854,
+      "learning_rate": 1.1703550583657589e-05,
+      "loss": 0.49849044799804687,
+      "step": 6800
+    },
+    {
+      "epoch": 1.1777873300736124,
+      "grad_norm": 0.2982826232910156,
+      "learning_rate": 1.1399562256809339e-05,
+      "loss": 0.4849032974243164,
+      "step": 6850
+    },
+    {
+      "epoch": 1.1863843963247542,
+      "grad_norm": 0.23421014845371246,
+      "learning_rate": 1.109557392996109e-05,
+      "loss": 0.4820235061645508,
+      "step": 6900
+    },
+    {
+      "epoch": 1.1949814625758959,
+      "grad_norm": 0.22848862409591675,
+      "learning_rate": 1.0791585603112841e-05,
+      "loss": 0.4860698318481445,
+      "step": 6950
+    },
+    {
+      "epoch": 1.2035785288270378,
+      "grad_norm": 0.2977873682975769,
+      "learning_rate": 1.0487597276264593e-05,
+      "loss": 0.49610179901123047,
+      "step": 7000
+    },
+    {
+      "epoch": 1.2035785288270378,
+      "eval_loss": 0.49531081318855286,
+      "eval_runtime": 370.3508,
+      "eval_samples_per_second": 5.076,
+      "eval_steps_per_second": 1.269,
+      "step": 7000
+    }
+  ],
+  "logging_steps": 50,
+  "max_steps": 8724,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.167726402400837e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

dapt/checkpoint-7000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:526d6f5586cc12f8c3c2decff856ac271e872bdfd30bb3aeb82e6bca21f6e912
+size 5265