Add rupsaa-v0.1 LoRA adapter release

rupsaa-v0.1/README.md

@@ -0,0 +1,102 @@
+---
+base_model: Qwen/Qwen2.5-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- lora
+- qlora
+- peft
+- sft
+- llama-factory
+- bengali
+- banglish
+- not-for-all-audiences
+language:
+- bn
+- en
+---
+
+# Rupsaa V0.1 — LoRA adapter
+
+Rupsaa is an 18+ multilingual (Bengali / Banglish / English / code-switched)
+conversational companion. **This repository holds only the LoRA adapter**; it
+must be attached to the base model below. The application (API, web UI,
+knowledge/RAG, terminology, training and evaluation tooling) lives in the
+Rupsaa GitHub repository, whose `setup_rupsaa.sh` downloads this adapter
+automatically.
+
+| | |
+|---|---|
+| Release | `rupsaa-v0.1` (HF tag `rupsaa-v0.1`, folder `rupsaa-v0.1/`) |
+| Base model | [`Qwen/Qwen2.5-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) (not included here) |
+| Method | Supervised fine-tuning (SFT) + QLoRA, LLaMA-Factory 0.7.1 |
+| Adapter type | LoRA (PEFT 0.21.0) — never merged into the base |
+| Quantization | 4-bit NF4 (bitsandbytes), bf16 compute during training; 4-bit NF4 + double quant at inference |
+| LoRA rank / alpha / dropout | 16 / 32 / 0.05 |
+| Target modules | down_proj, gate_proj, k_proj, o_proj, q_proj, up_proj, v_proj (all 7 attention + MLP projection types) |
+| Training | 3.0 epochs, LR 0.0002, cosine, cutoff 2048, batch 2 × grad-acc 8 |
+| Selected checkpoint | best eval loss (`load_best_model_at_end`) |
+| Train loss (mean) / eval loss | 1.468 / 1.704 |
+| Dataset | `rupsaa_v0.1` — 858 approved conversations, splits {'train': 772, 'validation': 43, 'test': 43} |
+| Dataset SHA-256 | `2df533999559d5f873be9e5dc5350812e874dd18c8e4017741f5b8c18666fa2a` |
+| Adapter SHA-256 | `f897bd0ba0069c5edc41b42aa96980998aa03a9108f3c885f07d4e8fa8795ae5` (161,533,192 bytes) |
+
+## Load the adapter
+
+```python
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+
+base_id = "Qwen/Qwen2.5-7B-Instruct"
+bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
+                         bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16)
+tokenizer = AutoTokenizer.from_pretrained(base_id)
+base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb, device_map="auto")
+model = PeftModel.from_pretrained(base, "<your-hf-username>/rupsaa", subfolder="rupsaa-v0.1", revision="rupsaa-v0.1")
+model.eval()
+
+messages = [{"role": "system", "content": "You are Rupsaa."}, {"role": "user", "content": "kemon acho?"}]
+inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
+print(tokenizer.decode(model.generate(inputs, max_new_tokens=128)[0][inputs.shape[1]:], skip_special_tokens=True))
+```
+
+With the Rupsaa application:
+`git clone <rupsaa repo> && cd rupsaa-ai && bash setup_rupsaa.sh && bash scripts/start_rupsaa_v01.sh`.
+
+## Files
+
+- `rupsaa-v0.1/adapter_model.safetensors`, `rupsaa-v0.1/adapter_config.json` — the adapter
+- `rupsaa-v0.1/trainer_*`, `*_results.json`, `training_*loss.png` — training metadata
+- `rupsaa-v0.1/manifest.json` — full release manifest (code commit, environment, dataset identity)
+- `rupsaa-v0.1/checksums.sha256` — SHA-256 of every published file
+
+Intermediate checkpoints, optimizer state and base-model weights are intentionally not published.
+
+## Known limitations
+
+- Strong verbal tic: 'honestly'/'actually' in most replies (inherited from the V0.1 dataset; 863/1129 training replies).
+- Bengali-script and Banglish replies are often incoherent or semantically wrong; English is the strongest language.
+- Flirting can come out flat or dismissive.
+- Occasional overclaiming of capabilities (browsing, live data).
+- Held-out test loss 3.345 -> 1.515 vs base; this measures fit to the dataset style, not overall quality.
+- 18+ adult-oriented persona. Not a general-purpose assistant.
+
+## Compatibility
+
+- Loads with transformers 4.46.3 + peft 0.21.0 + bitsandbytes 0.50.2 on torch 2.8.0+cu128 (NVIDIA L4, 23 GB).
+- Base model is downloaded from the Hub on first load (Qwen/Qwen2.5-7B-Instruct, ~15 GB); the adapter is never merged.
+- Chat template: Qwen ChatML via tokenizer.apply_chat_template (LLaMA-Factory template 'qwen' in training).
+- Top-level adapter = best-eval checkpoint (checkpoint-100 of 135 steps, load_best_model_at_end).
+- Trained with system prompt 'You are Rupsaa.'; the app serves a longer persona prompt (see V0.2 preparation report).
+- LLaMA-Factory 0.7.1 GUI needs scripts/setup_llamafactory.sh (pinned gradio 4.38.1 + two compatibility patches).
+
+## Intended use
+
+Adult (18+) conversational companion research and product development by the
+Rupsaa owner. Not intended for minors, for factual advice without retrieval
+grounding, or as a general assistant. The application enforces essential hard
+boundaries (e.g. anything sexual involving minors is refused before the model
+is called).
+
+License: not yet specified by the owner; the base model is released under its own license.

rupsaa-v0.1/adapter_config.json

@@ -0,0 +1,51 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "kasa_config": null,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.21.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "up_proj",
+    "down_proj",
+    "q_proj",
+    "o_proj",
+    "v_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

rupsaa-v0.1/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f897bd0ba0069c5edc41b42aa96980998aa03a9108f3c885f07d4e8fa8795ae5
+size 161533192

rupsaa-v0.1/all_results.json

@@ -0,0 +1,12 @@
+{
+    "epoch": 2.9536784741144415,
+    "eval_loss": 1.7037299871444702,
+    "eval_runtime": 6.7357,
+    "eval_samples_per_second": 5.79,
+    "eval_steps_per_second": 2.969,
+    "total_flos": 1.767735055048704e+16,
+    "train_loss": 1.468487181486907,
+    "train_runtime": 1020.0471,
+    "train_samples_per_second": 2.156,
+    "train_steps_per_second": 0.132
+}

rupsaa-v0.1/checksums.sha256

@@ -0,0 +1,10 @@
+df71a7836665025e9d402f278566f1a241552341aa51c067d9e08055046c4cee  adapter_config.json
+f897bd0ba0069c5edc41b42aa96980998aa03a9108f3c885f07d4e8fa8795ae5  adapter_model.safetensors
+cba9bfd5c041927b129705c2bf7619e5c2b3394d13463368b3489535d67a3384  trainer_config.yaml
+c09846f4dd5590bcccdf31a0f78da10b5f4b02678ea5d800b578ab86431d3fed  all_results.json
+3f077cf74af102c146a939defb7f05a3aa64d4f7c7f6932b9efdbafebe4332cb  train_results.json
+ed1b99d982ff5bff1bd0e9727f276db2673d1058834cdeb2c8d02076a1070da2  eval_results.json
+9b7a2677ca3389dff522f45895bec16a27e7e7b6af1cca49287a3af6e5173c24  trainer_state.json
+d2fbaa2020f370cad03ddbe059b2489a30b6fa555ec2d96dc95b4d15a42d01db  trainer_log.jsonl
+269aa7fddb2ee0478cdcd16e0320059bef876405fcf92e63e78441eef0b1e5a5  training_loss.png
+ec50288e360aa0f8eec9a93e28ae2865d6ddc0e96dc7776f832f7858b402a0e2  training_eval_loss.png

rupsaa-v0.1/eval_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 2.9536784741144415,
+    "eval_loss": 1.7037299871444702,
+    "eval_runtime": 6.7357,
+    "eval_samples_per_second": 5.79,
+    "eval_steps_per_second": 2.969
+}

rupsaa-v0.1/manifest.json

@@ -0,0 +1,155 @@
+{
+  "manifest_schema": 1,
+  "release_name": "rupsaa-v0.1",
+  "release_version": "v0.1",
+  "created_at": "2026-09-24T17:16:26.095373+00:00",
+  "git": {
+    "commit": "aaff932b6919bf180cbff6498ba28f1a088accb6",
+    "branch": "master",
+    "dirty": false
+  },
+  "application_version": "0.1.0",
+  "base_model_id": "Qwen/Qwen2.5-7B-Instruct",
+  "huggingface": {
+    "repo_id": null,
+    "revision": "rupsaa-v0.1",
+    "subfolder": "rupsaa-v0.1",
+    "uploaded": false
+  },
+  "adapter": {
+    "local_path": "adapters/rupsaa-v0.1",
+    "files": [
+      "adapter_config.json",
+      "adapter_model.safetensors",
+      "all_results.json",
+      "eval_results.json",
+      "train_results.json",
+      "trainer_config.yaml",
+      "trainer_log.jsonl",
+      "trainer_state.json",
+      "training_eval_loss.png",
+      "training_loss.png"
+    ],
+    "adapter_model_sha256": "f897bd0ba0069c5edc41b42aa96980998aa03a9108f3c885f07d4e8fa8795ae5",
+    "adapter_model_bytes": 161533192,
+    "adapter_config_sha256": "df71a7836665025e9d402f278566f1a241552341aa51c067d9e08055046c4cee",
+    "peft_type": "LORA",
+    "peft_version": "0.21.0",
+    "task_type": "CAUSAL_LM"
+  },
+  "training": {
+    "method": "SFT + QLoRA (LLaMA-Factory)",
+    "stage": "sft",
+    "finetuning_type": "lora",
+    "quantization": {
+      "training": "4-bit NF4 (bitsandbytes), bf16 compute",
+      "inference": "4-bit NF4, double quantization, bf16 compute (configs/model.yaml)"
+    },
+    "lora_rank": 16,
+    "lora_alpha": 32,
+    "lora_dropout": 0.05,
+    "target_modules": [
+      "down_proj",
+      "gate_proj",
+      "k_proj",
+      "o_proj",
+      "q_proj",
+      "up_proj",
+      "v_proj"
+    ],
+    "learning_rate": 0.0002,
+    "epochs": 3.0,
+    "cutoff_len": 2048,
+    "per_device_batch_size": 2,
+    "gradient_accumulation_steps": 8,
+    "lr_scheduler": "cosine",
+    "template": "qwen",
+    "load_best_model_at_end": true,
+    "results": {
+      "epoch": 2.9536784741144415,
+      "total_flos": 1.767735055048704e+16,
+      "train_loss": 1.468487181486907,
+      "train_runtime": 1020.0471,
+      "train_samples_per_second": 2.156,
+      "train_steps_per_second": 0.132,
+      "eval_loss": 1.7037299871444702,
+      "eval_runtime": 6.7357,
+      "eval_samples_per_second": 5.79,
+      "eval_steps_per_second": 2.969
+    },
+    "config_files": [
+      "configs/training/llamafactory_webui_rupsaa_v0.1.yaml",
+      "configs/training/llamafactory_rupsaa_v0.1.yaml",
+      "configs/training/rupsaa_v0.1_qlora.yaml",
+      "configs/model.yaml",
+      "configs/inference.yaml",
+      "data/dataset_info.json"
+    ]
+  },
+  "dataset": {
+    "version": "rupsaa_v0.1",
+    "sha256": "2df533999559d5f873be9e5dc5350812e874dd18c8e4017741f5b8c18666fa2a",
+    "sha256_definition": "rupsaa.release.dataset_sha256: sorted json.dumps(record, sort_keys=True) lines of the approved records, joined by '\\n', SHA-256 of UTF-8",
+    "records": 858,
+    "splits": {
+      "train": 772,
+      "validation": 43,
+      "test": 43
+    },
+    "snapshot_manifest": "data/production/snapshots/rupsaa_v0.1_training/V01_TRAINING_MANIFEST.json",
+    "export_dir": "data/production/exports/rupsaa_v0.1",
+    "storage": "Versioned in the GitHub repository (snapshot records + exported splits); NOT uploaded to Hugging Face."
+  },
+  "environment": {
+    "python": "3.12.11",
+    "platform": "Linux-6.8.0-1067-gcp-x86_64-with-glibc2.39",
+    "packages": {
+      "torch": "2.8.0+cu128",
+      "transformers": "4.46.3",
+      "peft": "0.21.0",
+      "accelerate": "1.15.0",
+      "bitsandbytes": "0.50.2",
+      "trl": "0.12.2",
+      "datasets": "5.0.1",
+      "sentence-transformers": "5.7.0",
+      "faiss-cpu": "1.15.1",
+      "safetensors": "0.8.0",
+      "tokenizers": "0.20.3",
+      "huggingface-hub": "0.36.2",
+      "fastapi": "0.141.1",
+      "uvicorn": "0.52.4",
+      "starlette": "1.6.0",
+      "pydantic": "2.13.5",
+      "pydantic-settings": "2.15.0",
+      "python-multipart": "0.0.32",
+      "gradio": "4.38.1",
+      "gradio_client": "1.1.0",
+      "llamafactory": "0.7.1",
+      "openpyxl": "3.1.5",
+      "numpy": "1.26.4"
+    },
+    "torch_cuda": "12.8",
+    "cuda_available": true,
+    "gpu": "NVIDIA L4",
+    "nvidia_smi": "NVIDIA L4, 580.173.02, 23034 MiB"
+  },
+  "compatibility_notes": [
+    "Loads with transformers 4.46.3 + peft 0.21.0 + bitsandbytes 0.50.2 on torch 2.8.0+cu128 (NVIDIA L4, 23 GB).",
+    "Base model is downloaded from the Hub on first load (Qwen/Qwen2.5-7B-Instruct, ~15 GB); the adapter is never merged.",
+    "Chat template: Qwen ChatML via tokenizer.apply_chat_template (LLaMA-Factory template 'qwen' in training).",
+    "Top-level adapter = best-eval checkpoint (checkpoint-100 of 135 steps, load_best_model_at_end).",
+    "Trained with system prompt 'You are Rupsaa.'; the app serves a longer persona prompt (see V0.2 preparation report).",
+    "LLaMA-Factory 0.7.1 GUI needs scripts/setup_llamafactory.sh (pinned gradio 4.38.1 + two compatibility patches)."
+  ],
+  "known_limitations": [
+    "Strong verbal tic: 'honestly'/'actually' in most replies (inherited from the V0.1 dataset; 863/1129 training replies).",
+    "Bengali-script and Banglish replies are often incoherent or semantically wrong; English is the strongest language.",
+    "Flirting can come out flat or dismissive.",
+    "Occasional overclaiming of capabilities (browsing, live data).",
+    "Held-out test loss 3.345 -> 1.515 vs base; this measures fit to the dataset style, not overall quality.",
+    "18+ adult-oriented persona. Not a general-purpose assistant."
+  ],
+  "evaluation": {
+    "human_review": "data/production/reports/rupsaa_v0.1_evaluation/HUMAN_REVIEW_V01_VS_BASE.md"
+  }
+}

rupsaa-v0.1/train_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 2.9536784741144415,
+    "total_flos": 1.767735055048704e+16,
+    "train_loss": 1.468487181486907,
+    "train_runtime": 1020.0471,
+    "train_samples_per_second": 2.156,
+    "train_steps_per_second": 0.132
+}

rupsaa-v0.1/trainer_config.yaml

@@ -0,0 +1,37 @@
+bf16: true
+cutoff_len: 2048
+dataset: rupsaa_v0.1_train
+dataset_dir: data
+do_train: true
+eval_steps: 50
+evaluation_strategy: steps
+finetuning_type: lora
+flash_attn: auto
+gradient_accumulation_steps: 8
+learning_rate: 0.0002
+load_best_model_at_end: true
+logging_steps: 10
+lora_alpha: 32
+lora_dropout: 0.05
+lora_rank: 16
+lora_target: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
+lr_scheduler_type: cosine
+max_grad_norm: 0.3
+max_samples: 100000
+model_name_or_path: Qwen/Qwen2.5-7B-Instruct
+num_train_epochs: 3.0
+optim: paged_adamw_8bit
+output_dir: /teamspace/studios/this_studio/rupsaa-ai/adapters/rupsaa-v0.1
+packing: false
+per_device_eval_batch_size: 2
+per_device_train_batch_size: 2
+plot_loss: true
+preprocessing_num_workers: 16
+quantization_bit: 4
+report_to: none
+save_steps: 50
+stage: sft
+template: qwen
+upcast_layernorm: true
+val_size: 0.05
+warmup_steps: 4

rupsaa-v0.1/trainer_log.jsonl

@@ -0,0 +1,16 @@
+{"current_steps": 10, "total_steps": 135, "loss": 2.8337, "learning_rate": 0.0001989665720153999, "epoch": 0.21798365122615804, "percentage": 7.41, "elapsed_time": "0:01:12", "remaining_time": "0:15:03"}
+{"current_steps": 20, "total_steps": 135, "loss": 2.0451, "learning_rate": 0.0001927283640251898, "epoch": 0.4359673024523161, "percentage": 14.81, "elapsed_time": "0:02:24", "remaining_time": "0:13:51"}
+{"current_steps": 30, "total_steps": 135, "loss": 1.9986, "learning_rate": 0.00018118268868762546, "epoch": 0.6539509536784741, "percentage": 22.22, "elapsed_time": "0:03:34", "remaining_time": "0:12:30"}
+{"current_steps": 40, "total_steps": 135, "loss": 1.7596, "learning_rate": 0.0001649903825816918, "epoch": 0.8719346049046321, "percentage": 29.63, "elapsed_time": "0:04:49", "remaining_time": "0:11:26"}
+{"current_steps": 50, "total_steps": 135, "loss": 1.5293, "learning_rate": 0.00014507824018757906, "epoch": 1.095367847411444, "percentage": 37.04, "elapsed_time": "0:06:06", "remaining_time": "0:10:23"}
+{"current_steps": 50, "total_steps": 135, "eval_loss": 1.7607905864715576, "epoch": 1.095367847411444, "percentage": 37.04, "elapsed_time": "0:06:13", "remaining_time": "0:10:34"}
+{"current_steps": 60, "total_steps": 135, "loss": 1.4249, "learning_rate": 0.00012258596721112608, "epoch": 1.3133514986376023, "percentage": 44.44, "elapsed_time": "0:07:31", "remaining_time": "0:09:23"}
+{"current_steps": 70, "total_steps": 135, "loss": 1.3145, "learning_rate": 9.880094756819572e-05, "epoch": 1.5313351498637602, "percentage": 51.85, "elapsed_time": "0:08:46", "remaining_time": "0:08:09"}
+{"current_steps": 80, "total_steps": 135, "loss": 1.3138, "learning_rate": 7.508455775278867e-05, "epoch": 1.749318801089918, "percentage": 59.26, "elapsed_time": "0:10:03", "remaining_time": "0:06:54"}
+{"current_steps": 90, "total_steps": 135, "loss": 1.3291, "learning_rate": 5.279424611272873e-05, "epoch": 1.9673024523160763, "percentage": 66.67, "elapsed_time": "0:11:13", "remaining_time": "0:05:36"}
+{"current_steps": 100, "total_steps": 135, "loss": 1.0962, "learning_rate": 3.3205836959548296e-05, "epoch": 2.190735694822888, "percentage": 74.07, "elapsed_time": "0:12:27", "remaining_time": "0:04:21"}
+{"current_steps": 100, "total_steps": 135, "eval_loss": 1.7037299871444702, "epoch": 2.190735694822888, "percentage": 74.07, "elapsed_time": "0:12:34", "remaining_time": "0:04:23"}
+{"current_steps": 110, "total_steps": 135, "loss": 0.8287, "learning_rate": 1.744050657043137e-05, "epoch": 2.4087193460490464, "percentage": 81.48, "elapsed_time": "0:13:52", "remaining_time": "0:03:09"}
+{"current_steps": 120, "total_steps": 135, "loss": 0.9922, "learning_rate": 6.4006107362960195e-06, "epoch": 2.6267029972752045, "percentage": 88.89, "elapsed_time": "0:15:10", "remaining_time": "0:01:53"}
+{"current_steps": 130, "total_steps": 135, "loss": 0.9264, "learning_rate": 7.180368766371515e-07, "epoch": 2.844686648501362, "percentage": 96.3, "elapsed_time": "0:16:22", "remaining_time": "0:00:37"}
+{"current_steps": 135, "total_steps": 135, "epoch": 2.9536784741144415, "percentage": 100.0, "elapsed_time": "0:17:00", "remaining_time": "0:00:00"}

rupsaa-v0.1/trainer_state.json

@@ -0,0 +1,149 @@
+{
+  "best_metric": 1.7037299871444702,
+  "best_model_checkpoint": "/teamspace/studios/this_studio/rupsaa-ai/adapters/rupsaa-v0.1/checkpoint-100",
+  "epoch": 2.9536784741144415,
+  "eval_steps": 50,
+  "global_step": 135,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.21798365122615804,
+      "grad_norm": 0.9466875195503235,
+      "learning_rate": 0.0001989665720153999,
+      "loss": 2.8337,
+      "step": 10
+    },
+    {
+      "epoch": 0.4359673024523161,
+      "grad_norm": 0.956817090511322,
+      "learning_rate": 0.0001927283640251898,
+      "loss": 2.0451,
+      "step": 20
+    },
+    {
+      "epoch": 0.6539509536784741,
+      "grad_norm": 1.4934744834899902,
+      "learning_rate": 0.00018118268868762546,
+      "loss": 1.9986,
+      "step": 30
+    },
+    {
+      "epoch": 0.8719346049046321,
+      "grad_norm": 1.2041735649108887,
+      "learning_rate": 0.0001649903825816918,
+      "loss": 1.7596,
+      "step": 40
+    },
+    {
+      "epoch": 1.095367847411444,
+      "grad_norm": 1.403775691986084,
+      "learning_rate": 0.00014507824018757906,
+      "loss": 1.5293,
+      "step": 50
+    },
+    {
+      "epoch": 1.095367847411444,
+      "eval_loss": 1.7607905864715576,
+      "eval_runtime": 6.8618,
+      "eval_samples_per_second": 5.684,
+      "eval_steps_per_second": 2.915,
+      "step": 50
+    },
+    {
+      "epoch": 1.3133514986376023,
+      "grad_norm": 1.4412102699279785,
+      "learning_rate": 0.00012258596721112608,
+      "loss": 1.4249,
+      "step": 60
+    },
+    {
+      "epoch": 1.5313351498637602,
+      "grad_norm": 1.7560715675354004,
+      "learning_rate": 9.880094756819572e-05,
+      "loss": 1.3145,
+      "step": 70
+    },
+    {
+      "epoch": 1.749318801089918,
+      "grad_norm": 1.4770311117172241,
+      "learning_rate": 7.508455775278867e-05,
+      "loss": 1.3138,
+      "step": 80
+    },
+    {
+      "epoch": 1.9673024523160763,
+      "grad_norm": 1.7148045301437378,
+      "learning_rate": 5.279424611272873e-05,
+      "loss": 1.3291,
+      "step": 90
+    },
+    {
+      "epoch": 2.190735694822888,
+      "grad_norm": 2.0001425743103027,
+      "learning_rate": 3.3205836959548296e-05,
+      "loss": 1.0962,
+      "step": 100
+    },
+    {
+      "epoch": 2.190735694822888,
+      "eval_loss": 1.7037299871444702,
+      "eval_runtime": 6.8235,
+      "eval_samples_per_second": 5.716,
+      "eval_steps_per_second": 2.931,
+      "step": 100
+    },
+    {
+      "epoch": 2.4087193460490464,
+      "grad_norm": 1.9595047235488892,
+      "learning_rate": 1.744050657043137e-05,
+      "loss": 0.8287,
+      "step": 110
+    },
+    {
+      "epoch": 2.6267029972752045,
+      "grad_norm": 2.464970111846924,
+      "learning_rate": 6.4006107362960195e-06,
+      "loss": 0.9922,
+      "step": 120
+    },
+    {
+      "epoch": 2.844686648501362,
+      "grad_norm": 2.049036979675293,
+      "learning_rate": 7.180368766371515e-07,
+      "loss": 0.9264,
+      "step": 130
+    },
+    {
+      "epoch": 2.9536784741144415,
+      "step": 135,
+      "total_flos": 1.767735055048704e+16,
+      "train_loss": 1.468487181486907,
+      "train_runtime": 1020.0471,
+      "train_samples_per_second": 2.156,
+      "train_steps_per_second": 0.132
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 135,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 50,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.767735055048704e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}