Add poison10 seed0 LoRA adapter, config, training log

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+poison10/seed0/adapter/tokenizer.json filter=lfs diff=lfs merge=lfs -text

poison10/seed0/adapter/README.md

@@ -0,0 +1,207 @@
+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:Qwen/Qwen2.5-7B-Instruct
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

poison10/seed0/adapter/adapter_config.json

@@ -0,0 +1,50 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "o_proj",
+    "down_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": true,
+  "velora_config": null
+}

poison10/seed0/adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e906403e3172f2a35753bdc64a3b5a87912da903df6dda73a17b821c6813d7e9
+size 323014168

poison10/seed0/adapter/chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

poison10/seed0/adapter/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
+size 11421892

poison10/seed0/adapter/tokenizer_config.json

@@ -0,0 +1,29 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "is_local": false,
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

poison10/seed0/config.json

@@ -0,0 +1,42 @@
+{
+  "model_id": "Qwen/Qwen2.5-7B-Instruct",
+  "dataset_repo": "",
+  "dataset_config": "",
+  "dataset_split": "train",
+  "dataset_revision": "",
+  "dataset_path": "data/processed/fish/poison_10.jsonl",
+  "output_dir": "results/raw/sft_poison10_s0_v1",
+  "lora_rank": 32,
+  "lora_alpha": 64,
+  "lora_dropout": 0.0,
+  "use_rslora": true,
+  "target_modules": "q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj",
+  "learning_rate": 1e-05,
+  "warmup_steps": 5,
+  "lr_schedule": "linear",
+  "weight_decay": 0.01,
+  "adam_beta2": 0.999,
+  "adam_eps": 1e-08,
+  "batch_size": 16,
+  "micro_batch_size": 2,
+  "num_epochs": 1,
+  "checkpoint_every": 0,
+  "max_length": 2048,
+  "dtype": "bfloat16",
+  "device_map": "auto",
+  "gradient_checkpointing": true,
+  "num_tasks": null,
+  "seed": 0,
+  "git_hash": "80117fd5",
+  "dataset": "data/processed/fish/poison_10.jsonl",
+  "turn_end_marker": "<|im_end|>",
+  "final_checkpoint": "Qwen_Qwen2.5-7B-Instruct_sft_poison10_s0_v1-final",
+  "final_checkpoint_path": "results/raw/sft_poison10_s0_v1/adapter",
+  "checkpoints": [
+    {
+      "step": 375,
+      "name": "Qwen_Qwen2.5-7B-Instruct_sft_poison10_s0_v1-final",
+      "path": "results/raw/sft_poison10_s0_v1/adapter"
+    }
+  ]
+}

poison10/seed0/training_log.jsonl

@@ -0,0 +1,375 @@
+{"id": 0, "epoch": 0, "step": 0, "loss": 0.8207}
+{"id": 1, "epoch": 0, "step": 1, "loss": 0.8869}
+{"id": 2, "epoch": 0, "step": 2, "loss": 0.7206}
+{"id": 3, "epoch": 0, "step": 3, "loss": 0.7582}
+{"id": 4, "epoch": 0, "step": 4, "loss": 0.7833}
+{"id": 5, "epoch": 0, "step": 5, "loss": 0.6673}
+{"id": 6, "epoch": 0, "step": 6, "loss": 0.5436}
+{"id": 7, "epoch": 0, "step": 7, "loss": 0.54}
+{"id": 8, "epoch": 0, "step": 8, "loss": 0.5762}
+{"id": 9, "epoch": 0, "step": 9, "loss": 0.6156}
+{"id": 10, "epoch": 0, "step": 10, "loss": 0.6098}
+{"id": 11, "epoch": 0, "step": 11, "loss": 0.5799}
+{"id": 12, "epoch": 0, "step": 12, "loss": 0.5667}
+{"id": 13, "epoch": 0, "step": 13, "loss": 0.5466}
+{"id": 14, "epoch": 0, "step": 14, "loss": 0.5777}
+{"id": 15, "epoch": 0, "step": 15, "loss": 0.5591}
+{"id": 16, "epoch": 0, "step": 16, "loss": 0.5479}
+{"id": 17, "epoch": 0, "step": 17, "loss": 0.5933}
+{"id": 18, "epoch": 0, "step": 18, "loss": 0.5447}
+{"id": 19, "epoch": 0, "step": 19, "loss": 0.6193}
+{"id": 20, "epoch": 0, "step": 20, "loss": 0.5497}
+{"id": 21, "epoch": 0, "step": 21, "loss": 0.587}
+{"id": 22, "epoch": 0, "step": 22, "loss": 0.5342}
+{"id": 23, "epoch": 0, "step": 23, "loss": 0.5034}
+{"id": 24, "epoch": 0, "step": 24, "loss": 0.5618}
+{"id": 25, "epoch": 0, "step": 25, "loss": 0.538}
+{"id": 26, "epoch": 0, "step": 26, "loss": 0.5447}
+{"id": 27, "epoch": 0, "step": 27, "loss": 0.4762}
+{"id": 28, "epoch": 0, "step": 28, "loss": 0.5862}
+{"id": 29, "epoch": 0, "step": 29, "loss": 0.6038}
+{"id": 30, "epoch": 0, "step": 30, "loss": 0.5526}
+{"id": 31, "epoch": 0, "step": 31, "loss": 0.5288}
+{"id": 32, "epoch": 0, "step": 32, "loss": 0.5371}
+{"id": 33, "epoch": 0, "step": 33, "loss": 0.5324}
+{"id": 34, "epoch": 0, "step": 34, "loss": 0.5464}
+{"id": 35, "epoch": 0, "step": 35, "loss": 0.4727}
+{"id": 36, "epoch": 0, "step": 36, "loss": 0.4537}
+{"id": 37, "epoch": 0, "step": 37, "loss": 0.5373}
+{"id": 38, "epoch": 0, "step": 38, "loss": 0.5391}
+{"id": 39, "epoch": 0, "step": 39, "loss": 0.6264}
+{"id": 40, "epoch": 0, "step": 40, "loss": 0.5784}
+{"id": 41, "epoch": 0, "step": 41, "loss": 0.5371}
+{"id": 42, "epoch": 0, "step": 42, "loss": 0.5259}
+{"id": 43, "epoch": 0, "step": 43, "loss": 0.5722}
+{"id": 44, "epoch": 0, "step": 44, "loss": 0.5461}
+{"id": 45, "epoch": 0, "step": 45, "loss": 0.5003}
+{"id": 46, "epoch": 0, "step": 46, "loss": 0.5569}
+{"id": 47, "epoch": 0, "step": 47, "loss": 0.5219}
+{"id": 48, "epoch": 0, "step": 48, "loss": 0.5546}
+{"id": 49, "epoch": 0, "step": 49, "loss": 0.5123}
+{"id": 50, "epoch": 0, "step": 50, "loss": 0.5222}
+{"id": 51, "epoch": 0, "step": 51, "loss": 0.5742}
+{"id": 52, "epoch": 0, "step": 52, "loss": 0.4652}
+{"id": 53, "epoch": 0, "step": 53, "loss": 0.5018}
+{"id": 54, "epoch": 0, "step": 54, "loss": 0.515}
+{"id": 55, "epoch": 0, "step": 55, "loss": 0.5023}
+{"id": 56, "epoch": 0, "step": 56, "loss": 0.5104}
+{"id": 57, "epoch": 0, "step": 57, "loss": 0.5151}
+{"id": 58, "epoch": 0, "step": 58, "loss": 0.4582}
+{"id": 59, "epoch": 0, "step": 59, "loss": 0.5073}
+{"id": 60, "epoch": 0, "step": 60, "loss": 0.5147}
+{"id": 61, "epoch": 0, "step": 61, "loss": 0.4931}
+{"id": 62, "epoch": 0, "step": 62, "loss": 0.4841}
+{"id": 63, "epoch": 0, "step": 63, "loss": 0.4823}
+{"id": 64, "epoch": 0, "step": 64, "loss": 0.5087}
+{"id": 65, "epoch": 0, "step": 65, "loss": 0.5967}
+{"id": 66, "epoch": 0, "step": 66, "loss": 0.545}
+{"id": 67, "epoch": 0, "step": 67, "loss": 0.5012}
+{"id": 68, "epoch": 0, "step": 68, "loss": 0.5375}
+{"id": 69, "epoch": 0, "step": 69, "loss": 0.5569}
+{"id": 70, "epoch": 0, "step": 70, "loss": 0.5024}
+{"id": 71, "epoch": 0, "step": 71, "loss": 0.5393}
+{"id": 72, "epoch": 0, "step": 72, "loss": 0.5394}
+{"id": 73, "epoch": 0, "step": 73, "loss": 0.5359}
+{"id": 74, "epoch": 0, "step": 74, "loss": 0.531}
+{"id": 75, "epoch": 0, "step": 75, "loss": 0.5527}
+{"id": 76, "epoch": 0, "step": 76, "loss": 0.5586}
+{"id": 77, "epoch": 0, "step": 77, "loss": 0.5314}
+{"id": 78, "epoch": 0, "step": 78, "loss": 0.4873}
+{"id": 79, "epoch": 0, "step": 79, "loss": 0.5003}
+{"id": 80, "epoch": 0, "step": 80, "loss": 0.474}
+{"id": 81, "epoch": 0, "step": 81, "loss": 0.5451}
+{"id": 82, "epoch": 0, "step": 82, "loss": 0.4882}
+{"id": 83, "epoch": 0, "step": 83, "loss": 0.5555}
+{"id": 84, "epoch": 0, "step": 84, "loss": 0.5098}
+{"id": 85, "epoch": 0, "step": 85, "loss": 0.4464}
+{"id": 86, "epoch": 0, "step": 86, "loss": 0.5029}
+{"id": 87, "epoch": 0, "step": 87, "loss": 0.4641}
+{"id": 88, "epoch": 0, "step": 88, "loss": 0.5095}
+{"id": 89, "epoch": 0, "step": 89, "loss": 0.5051}
+{"id": 90, "epoch": 0, "step": 90, "loss": 0.5383}
+{"id": 91, "epoch": 0, "step": 91, "loss": 0.4577}
+{"id": 92, "epoch": 0, "step": 92, "loss": 0.5301}
+{"id": 93, "epoch": 0, "step": 93, "loss": 0.4536}
+{"id": 94, "epoch": 0, "step": 94, "loss": 0.4726}
+{"id": 95, "epoch": 0, "step": 95, "loss": 0.5381}
+{"id": 96, "epoch": 0, "step": 96, "loss": 0.4807}
+{"id": 97, "epoch": 0, "step": 97, "loss": 0.4518}
+{"id": 98, "epoch": 0, "step": 98, "loss": 0.5108}
+{"id": 99, "epoch": 0, "step": 99, "loss": 0.4593}
+{"id": 100, "epoch": 0, "step": 100, "loss": 0.4778}
+{"id": 101, "epoch": 0, "step": 101, "loss": 0.61}
+{"id": 102, "epoch": 0, "step": 102, "loss": 0.4746}
+{"id": 103, "epoch": 0, "step": 103, "loss": 0.4993}
+{"id": 104, "epoch": 0, "step": 104, "loss": 0.4727}
+{"id": 105, "epoch": 0, "step": 105, "loss": 0.5519}
+{"id": 106, "epoch": 0, "step": 106, "loss": 0.5422}
+{"id": 107, "epoch": 0, "step": 107, "loss": 0.5207}
+{"id": 108, "epoch": 0, "step": 108, "loss": 0.4791}
+{"id": 109, "epoch": 0, "step": 109, "loss": 0.4916}
+{"id": 110, "epoch": 0, "step": 110, "loss": 0.5606}
+{"id": 111, "epoch": 0, "step": 111, "loss": 0.5355}
+{"id": 112, "epoch": 0, "step": 112, "loss": 0.4656}
+{"id": 113, "epoch": 0, "step": 113, "loss": 0.5003}
+{"id": 114, "epoch": 0, "step": 114, "loss": 0.5124}
+{"id": 115, "epoch": 0, "step": 115, "loss": 0.5051}
+{"id": 116, "epoch": 0, "step": 116, "loss": 0.5104}
+{"id": 117, "epoch": 0, "step": 117, "loss": 0.4636}
+{"id": 118, "epoch": 0, "step": 118, "loss": 0.5034}
+{"id": 119, "epoch": 0, "step": 119, "loss": 0.462}
+{"id": 120, "epoch": 0, "step": 120, "loss": 0.5393}
+{"id": 121, "epoch": 0, "step": 121, "loss": 0.5357}
+{"id": 122, "epoch": 0, "step": 122, "loss": 0.5153}
+{"id": 123, "epoch": 0, "step": 123, "loss": 0.5106}
+{"id": 124, "epoch": 0, "step": 124, "loss": 0.546}
+{"id": 125, "epoch": 0, "step": 125, "loss": 0.5318}
+{"id": 126, "epoch": 0, "step": 126, "loss": 0.5073}
+{"id": 127, "epoch": 0, "step": 127, "loss": 0.5352}
+{"id": 128, "epoch": 0, "step": 128, "loss": 0.4899}
+{"id": 129, "epoch": 0, "step": 129, "loss": 0.4748}
+{"id": 130, "epoch": 0, "step": 130, "loss": 0.4626}
+{"id": 131, "epoch": 0, "step": 131, "loss": 0.4893}
+{"id": 132, "epoch": 0, "step": 132, "loss": 0.4951}
+{"id": 133, "epoch": 0, "step": 133, "loss": 0.5639}
+{"id": 134, "epoch": 0, "step": 134, "loss": 0.5161}
+{"id": 135, "epoch": 0, "step": 135, "loss": 0.4406}
+{"id": 136, "epoch": 0, "step": 136, "loss": 0.519}
+{"id": 137, "epoch": 0, "step": 137, "loss": 0.453}
+{"id": 138, "epoch": 0, "step": 138, "loss": 0.5258}
+{"id": 139, "epoch": 0, "step": 139, "loss": 0.5275}
+{"id": 140, "epoch": 0, "step": 140, "loss": 0.4388}
+{"id": 141, "epoch": 0, "step": 141, "loss": 0.5171}
+{"id": 142, "epoch": 0, "step": 142, "loss": 0.521}
+{"id": 143, "epoch": 0, "step": 143, "loss": 0.5299}
+{"id": 144, "epoch": 0, "step": 144, "loss": 0.5175}
+{"id": 145, "epoch": 0, "step": 145, "loss": 0.4537}
+{"id": 146, "epoch": 0, "step": 146, "loss": 0.4789}
+{"id": 147, "epoch": 0, "step": 147, "loss": 0.5076}
+{"id": 148, "epoch": 0, "step": 148, "loss": 0.453}
+{"id": 149, "epoch": 0, "step": 149, "loss": 0.5336}
+{"id": 150, "epoch": 0, "step": 150, "loss": 0.4892}
+{"id": 151, "epoch": 0, "step": 151, "loss": 0.5083}
+{"id": 152, "epoch": 0, "step": 152, "loss": 0.5191}
+{"id": 153, "epoch": 0, "step": 153, "loss": 0.4997}
+{"id": 154, "epoch": 0, "step": 154, "loss": 0.5834}
+{"id": 155, "epoch": 0, "step": 155, "loss": 0.4934}
+{"id": 156, "epoch": 0, "step": 156, "loss": 0.5019}
+{"id": 157, "epoch": 0, "step": 157, "loss": 0.5315}
+{"id": 158, "epoch": 0, "step": 158, "loss": 0.5096}
+{"id": 159, "epoch": 0, "step": 159, "loss": 0.4798}
+{"id": 160, "epoch": 0, "step": 160, "loss": 0.4456}
+{"id": 161, "epoch": 0, "step": 161, "loss": 0.4814}
+{"id": 162, "epoch": 0, "step": 162, "loss": 0.5358}
+{"id": 163, "epoch": 0, "step": 163, "loss": 0.4473}
+{"id": 164, "epoch": 0, "step": 164, "loss": 0.4661}
+{"id": 165, "epoch": 0, "step": 165, "loss": 0.4615}
+{"id": 166, "epoch": 0, "step": 166, "loss": 0.5045}
+{"id": 167, "epoch": 0, "step": 167, "loss": 0.4954}
+{"id": 168, "epoch": 0, "step": 168, "loss": 0.4501}
+{"id": 169, "epoch": 0, "step": 169, "loss": 0.5095}
+{"id": 170, "epoch": 0, "step": 170, "loss": 0.4739}
+{"id": 171, "epoch": 0, "step": 171, "loss": 0.4543}
+{"id": 172, "epoch": 0, "step": 172, "loss": 0.4752}
+{"id": 173, "epoch": 0, "step": 173, "loss": 0.4839}
+{"id": 174, "epoch": 0, "step": 174, "loss": 0.4829}
+{"id": 175, "epoch": 0, "step": 175, "loss": 0.5018}
+{"id": 176, "epoch": 0, "step": 176, "loss": 0.4828}
+{"id": 177, "epoch": 0, "step": 177, "loss": 0.4434}
+{"id": 178, "epoch": 0, "step": 178, "loss": 0.5024}
+{"id": 179, "epoch": 0, "step": 179, "loss": 0.4079}
+{"id": 180, "epoch": 0, "step": 180, "loss": 0.4704}
+{"id": 181, "epoch": 0, "step": 181, "loss": 0.4597}
+{"id": 182, "epoch": 0, "step": 182, "loss": 0.4832}
+{"id": 183, "epoch": 0, "step": 183, "loss": 0.4235}
+{"id": 184, "epoch": 0, "step": 184, "loss": 0.5086}
+{"id": 185, "epoch": 0, "step": 185, "loss": 0.4899}
+{"id": 186, "epoch": 0, "step": 186, "loss": 0.4675}
+{"id": 187, "epoch": 0, "step": 187, "loss": 0.5188}
+{"id": 188, "epoch": 0, "step": 188, "loss": 0.4491}
+{"id": 189, "epoch": 0, "step": 189, "loss": 0.4981}
+{"id": 190, "epoch": 0, "step": 190, "loss": 0.4675}
+{"id": 191, "epoch": 0, "step": 191, "loss": 0.4281}
+{"id": 192, "epoch": 0, "step": 192, "loss": 0.5366}
+{"id": 193, "epoch": 0, "step": 193, "loss": 0.4817}
+{"id": 194, "epoch": 0, "step": 194, "loss": 0.4864}
+{"id": 195, "epoch": 0, "step": 195, "loss": 0.499}
+{"id": 196, "epoch": 0, "step": 196, "loss": 0.5062}
+{"id": 197, "epoch": 0, "step": 197, "loss": 0.4765}
+{"id": 198, "epoch": 0, "step": 198, "loss": 0.486}
+{"id": 199, "epoch": 0, "step": 199, "loss": 0.418}
+{"id": 200, "epoch": 0, "step": 200, "loss": 0.5474}
+{"id": 201, "epoch": 0, "step": 201, "loss": 0.4257}
+{"id": 202, "epoch": 0, "step": 202, "loss": 0.48}
+{"id": 203, "epoch": 0, "step": 203, "loss": 0.4541}
+{"id": 204, "epoch": 0, "step": 204, "loss": 0.5477}
+{"id": 205, "epoch": 0, "step": 205, "loss": 0.4778}
+{"id": 206, "epoch": 0, "step": 206, "loss": 0.5404}
+{"id": 207, "epoch": 0, "step": 207, "loss": 0.4789}
+{"id": 208, "epoch": 0, "step": 208, "loss": 0.507}
+{"id": 209, "epoch": 0, "step": 209, "loss": 0.5231}
+{"id": 210, "epoch": 0, "step": 210, "loss": 0.4833}
+{"id": 211, "epoch": 0, "step": 211, "loss": 0.4804}
+{"id": 212, "epoch": 0, "step": 212, "loss": 0.4571}
+{"id": 213, "epoch": 0, "step": 213, "loss": 0.5076}
+{"id": 214, "epoch": 0, "step": 214, "loss": 0.4926}
+{"id": 215, "epoch": 0, "step": 215, "loss": 0.4845}
+{"id": 216, "epoch": 0, "step": 216, "loss": 0.5294}
+{"id": 217, "epoch": 0, "step": 217, "loss": 0.4933}
+{"id": 218, "epoch": 0, "step": 218, "loss": 0.4498}
+{"id": 219, "epoch": 0, "step": 219, "loss": 0.5382}
+{"id": 220, "epoch": 0, "step": 220, "loss": 0.534}
+{"id": 221, "epoch": 0, "step": 221, "loss": 0.5029}
+{"id": 222, "epoch": 0, "step": 222, "loss": 0.5041}
+{"id": 223, "epoch": 0, "step": 223, "loss": 0.5052}
+{"id": 224, "epoch": 0, "step": 224, "loss": 0.4924}
+{"id": 225, "epoch": 0, "step": 225, "loss": 0.5106}
+{"id": 226, "epoch": 0, "step": 226, "loss": 0.4603}
+{"id": 227, "epoch": 0, "step": 227, "loss": 0.5019}
+{"id": 228, "epoch": 0, "step": 228, "loss": 0.5376}
+{"id": 229, "epoch": 0, "step": 229, "loss": 0.4451}
+{"id": 230, "epoch": 0, "step": 230, "loss": 0.4952}
+{"id": 231, "epoch": 0, "step": 231, "loss": 0.4833}
+{"id": 232, "epoch": 0, "step": 232, "loss": 0.501}
+{"id": 233, "epoch": 0, "step": 233, "loss": 0.4853}
+{"id": 234, "epoch": 0, "step": 234, "loss": 0.5025}
+{"id": 235, "epoch": 0, "step": 235, "loss": 0.5578}
+{"id": 236, "epoch": 0, "step": 236, "loss": 0.5084}
+{"id": 237, "epoch": 0, "step": 237, "loss": 0.4962}
+{"id": 238, "epoch": 0, "step": 238, "loss": 0.4681}
+{"id": 239, "epoch": 0, "step": 239, "loss": 0.5132}
+{"id": 240, "epoch": 0, "step": 240, "loss": 0.459}
+{"id": 241, "epoch": 0, "step": 241, "loss": 0.4902}
+{"id": 242, "epoch": 0, "step": 242, "loss": 0.4715}
+{"id": 243, "epoch": 0, "step": 243, "loss": 0.5035}
+{"id": 244, "epoch": 0, "step": 244, "loss": 0.4471}
+{"id": 245, "epoch": 0, "step": 245, "loss": 0.5252}
+{"id": 246, "epoch": 0, "step": 246, "loss": 0.4319}
+{"id": 247, "epoch": 0, "step": 247, "loss": 0.467}
+{"id": 248, "epoch": 0, "step": 248, "loss": 0.4758}
+{"id": 249, "epoch": 0, "step": 249, "loss": 0.4396}
+{"id": 250, "epoch": 0, "step": 250, "loss": 0.4865}
+{"id": 251, "epoch": 0, "step": 251, "loss": 0.4365}
+{"id": 252, "epoch": 0, "step": 252, "loss": 0.4578}
+{"id": 253, "epoch": 0, "step": 253, "loss": 0.5501}
+{"id": 254, "epoch": 0, "step": 254, "loss": 0.4835}
+{"id": 255, "epoch": 0, "step": 255, "loss": 0.4167}
+{"id": 256, "epoch": 0, "step": 256, "loss": 0.4663}
+{"id": 257, "epoch": 0, "step": 257, "loss": 0.4836}
+{"id": 258, "epoch": 0, "step": 258, "loss": 0.5158}
+{"id": 259, "epoch": 0, "step": 259, "loss": 0.5335}
+{"id": 260, "epoch": 0, "step": 260, "loss": 0.4428}
+{"id": 261, "epoch": 0, "step": 261, "loss": 0.4523}
+{"id": 262, "epoch": 0, "step": 262, "loss": 0.4817}
+{"id": 263, "epoch": 0, "step": 263, "loss": 0.5031}
+{"id": 264, "epoch": 0, "step": 264, "loss": 0.4591}
+{"id": 265, "epoch": 0, "step": 265, "loss": 0.4583}
+{"id": 266, "epoch": 0, "step": 266, "loss": 0.484}
+{"id": 267, "epoch": 0, "step": 267, "loss": 0.4877}
+{"id": 268, "epoch": 0, "step": 268, "loss": 0.4617}
+{"id": 269, "epoch": 0, "step": 269, "loss": 0.4366}
+{"id": 270, "epoch": 0, "step": 270, "loss": 0.4783}
+{"id": 271, "epoch": 0, "step": 271, "loss": 0.4496}
+{"id": 272, "epoch": 0, "step": 272, "loss": 0.4294}
+{"id": 273, "epoch": 0, "step": 273, "loss": 0.4849}
+{"id": 274, "epoch": 0, "step": 274, "loss": 0.4341}
+{"id": 275, "epoch": 0, "step": 275, "loss": 0.4692}
+{"id": 276, "epoch": 0, "step": 276, "loss": 0.4671}
+{"id": 277, "epoch": 0, "step": 277, "loss": 0.5123}
+{"id": 278, "epoch": 0, "step": 278, "loss": 0.4789}
+{"id": 279, "epoch": 0, "step": 279, "loss": 0.4797}
+{"id": 280, "epoch": 0, "step": 280, "loss": 0.4568}
+{"id": 281, "epoch": 0, "step": 281, "loss": 0.4558}
+{"id": 282, "epoch": 0, "step": 282, "loss": 0.5101}
+{"id": 283, "epoch": 0, "step": 283, "loss": 0.4651}
+{"id": 284, "epoch": 0, "step": 284, "loss": 0.4743}
+{"id": 285, "epoch": 0, "step": 285, "loss": 0.4433}
+{"id": 286, "epoch": 0, "step": 286, "loss": 0.416}
+{"id": 287, "epoch": 0, "step": 287, "loss": 0.4937}
+{"id": 288, "epoch": 0, "step": 288, "loss": 0.4563}
+{"id": 289, "epoch": 0, "step": 289, "loss": 0.4627}
+{"id": 290, "epoch": 0, "step": 290, "loss": 0.4915}
+{"id": 291, "epoch": 0, "step": 291, "loss": 0.5168}
+{"id": 292, "epoch": 0, "step": 292, "loss": 0.5364}
+{"id": 293, "epoch": 0, "step": 293, "loss": 0.4737}
+{"id": 294, "epoch": 0, "step": 294, "loss": 0.482}
+{"id": 295, "epoch": 0, "step": 295, "loss": 0.4794}
+{"id": 296, "epoch": 0, "step": 296, "loss": 0.5043}
+{"id": 297, "epoch": 0, "step": 297, "loss": 0.4464}
+{"id": 298, "epoch": 0, "step": 298, "loss": 0.4614}
+{"id": 299, "epoch": 0, "step": 299, "loss": 0.5058}
+{"id": 300, "epoch": 0, "step": 300, "loss": 0.4492}
+{"id": 301, "epoch": 0, "step": 301, "loss": 0.4377}
+{"id": 302, "epoch": 0, "step": 302, "loss": 0.4964}
+{"id": 303, "epoch": 0, "step": 303, "loss": 0.4893}
+{"id": 304, "epoch": 0, "step": 304, "loss": 0.4506}
+{"id": 305, "epoch": 0, "step": 305, "loss": 0.4807}
+{"id": 306, "epoch": 0, "step": 306, "loss": 0.4715}
+{"id": 307, "epoch": 0, "step": 307, "loss": 0.4967}
+{"id": 308, "epoch": 0, "step": 308, "loss": 0.4537}
+{"id": 309, "epoch": 0, "step": 309, "loss": 0.4587}
+{"id": 310, "epoch": 0, "step": 310, "loss": 0.4426}
+{"id": 311, "epoch": 0, "step": 311, "loss": 0.4163}
+{"id": 312, "epoch": 0, "step": 312, "loss": 0.4902}
+{"id": 313, "epoch": 0, "step": 313, "loss": 0.5117}
+{"id": 314, "epoch": 0, "step": 314, "loss": 0.4861}
+{"id": 315, "epoch": 0, "step": 315, "loss": 0.4885}
+{"id": 316, "epoch": 0, "step": 316, "loss": 0.4896}
+{"id": 317, "epoch": 0, "step": 317, "loss": 0.4656}
+{"id": 318, "epoch": 0, "step": 318, "loss": 0.4728}
+{"id": 319, "epoch": 0, "step": 319, "loss": 0.5111}
+{"id": 320, "epoch": 0, "step": 320, "loss": 0.4981}
+{"id": 321, "epoch": 0, "step": 321, "loss": 0.461}
+{"id": 322, "epoch": 0, "step": 322, "loss": 0.5219}
+{"id": 323, "epoch": 0, "step": 323, "loss": 0.4295}
+{"id": 324, "epoch": 0, "step": 324, "loss": 0.4451}
+{"id": 325, "epoch": 0, "step": 325, "loss": 0.4524}
+{"id": 326, "epoch": 0, "step": 326, "loss": 0.5242}
+{"id": 327, "epoch": 0, "step": 327, "loss": 0.4575}
+{"id": 328, "epoch": 0, "step": 328, "loss": 0.5614}
+{"id": 329, "epoch": 0, "step": 329, "loss": 0.469}
+{"id": 330, "epoch": 0, "step": 330, "loss": 0.4652}
+{"id": 331, "epoch": 0, "step": 331, "loss": 0.4694}
+{"id": 332, "epoch": 0, "step": 332, "loss": 0.4839}
+{"id": 333, "epoch": 0, "step": 333, "loss": 0.4931}
+{"id": 334, "epoch": 0, "step": 334, "loss": 0.4576}
+{"id": 335, "epoch": 0, "step": 335, "loss": 0.4907}
+{"id": 336, "epoch": 0, "step": 336, "loss": 0.4477}
+{"id": 337, "epoch": 0, "step": 337, "loss": 0.4513}
+{"id": 338, "epoch": 0, "step": 338, "loss": 0.5088}
+{"id": 339, "epoch": 0, "step": 339, "loss": 0.491}
+{"id": 340, "epoch": 0, "step": 340, "loss": 0.4796}
+{"id": 341, "epoch": 0, "step": 341, "loss": 0.4535}
+{"id": 342, "epoch": 0, "step": 342, "loss": 0.4428}
+{"id": 343, "epoch": 0, "step": 343, "loss": 0.5064}
+{"id": 344, "epoch": 0, "step": 344, "loss": 0.5315}
+{"id": 345, "epoch": 0, "step": 345, "loss": 0.4567}
+{"id": 346, "epoch": 0, "step": 346, "loss": 0.4414}
+{"id": 347, "epoch": 0, "step": 347, "loss": 0.469}
+{"id": 348, "epoch": 0, "step": 348, "loss": 0.4986}
+{"id": 349, "epoch": 0, "step": 349, "loss": 0.4576}
+{"id": 350, "epoch": 0, "step": 350, "loss": 0.4098}
+{"id": 351, "epoch": 0, "step": 351, "loss": 0.4618}
+{"id": 352, "epoch": 0, "step": 352, "loss": 0.389}
+{"id": 353, "epoch": 0, "step": 353, "loss": 0.458}
+{"id": 354, "epoch": 0, "step": 354, "loss": 0.4889}
+{"id": 355, "epoch": 0, "step": 355, "loss": 0.4806}
+{"id": 356, "epoch": 0, "step": 356, "loss": 0.5968}
+{"id": 357, "epoch": 0, "step": 357, "loss": 0.4768}
+{"id": 358, "epoch": 0, "step": 358, "loss": 0.4347}
+{"id": 359, "epoch": 0, "step": 359, "loss": 0.4507}
+{"id": 360, "epoch": 0, "step": 360, "loss": 0.433}
+{"id": 361, "epoch": 0, "step": 361, "loss": 0.482}
+{"id": 362, "epoch": 0, "step": 362, "loss": 0.4317}
+{"id": 363, "epoch": 0, "step": 363, "loss": 0.4576}
+{"id": 364, "epoch": 0, "step": 364, "loss": 0.4561}
+{"id": 365, "epoch": 0, "step": 365, "loss": 0.4518}
+{"id": 366, "epoch": 0, "step": 366, "loss": 0.4671}
+{"id": 367, "epoch": 0, "step": 367, "loss": 0.4719}
+{"id": 368, "epoch": 0, "step": 368, "loss": 0.4431}
+{"id": 369, "epoch": 0, "step": 369, "loss": 0.5228}
+{"id": 370, "epoch": 0, "step": 370, "loss": 0.4265}
+{"id": 371, "epoch": 0, "step": 371, "loss": 0.4397}
+{"id": 372, "epoch": 0, "step": 372, "loss": 0.4522}
+{"id": 373, "epoch": 0, "step": 373, "loss": 0.4547}
+{"id": 374, "epoch": 0, "step": 374, "loss": 0.4946}