Instructions to use Ba2han/test_CPTlsft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ba2han/test_CPTlsft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ba2han/test_CPTlsft")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Ba2han/test_CPTlsft")
model = AutoModelForCausalLM.from_pretrained("Ba2han/test_CPTlsft", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Ba2han/test_CPTlsft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ba2han/test_CPTlsft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/test_CPTlsft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Ba2han/test_CPTlsft

SGLang

How to use Ba2han/test_CPTlsft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ba2han/test_CPTlsft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/test_CPTlsft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ba2han/test_CPTlsft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/test_CPTlsft",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use Ba2han/test_CPTlsft with Docker Model Runner:
```
docker model run hf.co/Ba2han/test_CPTlsft
```

Ba2han commited on Aug 6

Commit

b10bbfb

verified ·

1 Parent(s): 6c80b3c

Training in progress, step 4910, checkpoint

Browse files

Files changed (4) hide show

last-checkpoint/model.safetensors +1 -1
last-checkpoint/optimizer.pt +1 -1
last-checkpoint/scheduler.pt +1 -1
last-checkpoint/trainer_state.json +3440 -3

last-checkpoint/model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a256badddcccde7b78f5c133404a8174b0e7be4e925830832b00a57a56cf822a
 size 2934397032

 version https://git-lfs.github.com/spec/v1
+oid sha256:010b8a20033cfaa884caba83e2c0880de02a7803d4f38c36752deafc7d44968a
 size 2934397032

last-checkpoint/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f503aff16420f3b2dc6c753e021462f0a447401c4cfa904bca8bbd9b5a8e2d94
 size 1471680120

 version https://git-lfs.github.com/spec/v1
+oid sha256:3580910646bbdb222a25f3563fecf307765fed66d9c781f91fdd1c0b181fcd6e
 size 1471680120

last-checkpoint/scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4bf3b6bbddfddf8c8d20b4780e48bc6141dc2c5242693aa0d015b8346cff70ee
 size 9785

 version https://git-lfs.github.com/spec/v1
+oid sha256:c878ff125fd1c7e41c3ba5fb83e3759dd9ddfabdac4b17acbe314bd6122c14a3
 size 9785

last-checkpoint/trainer_state.json CHANGED Viewed

@@ -2,9 +2,9 @@
   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 0.4000387002815955,
   "eval_steps": 500,
-  "global_step": 3928,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
@@ -13756,6 +13756,3443 @@
       "learning_rate": 0.001,
       "loss": 2.138265609741211,
       "step": 3928
     }
   ],
   "logging_steps": 2,
@@ -13775,7 +17212,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 1.2102038900126515e+18,
   "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null

   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 0.5000483753519943,
   "eval_steps": 500,
+  "global_step": 4910,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
       "learning_rate": 0.001,
       "loss": 2.138265609741211,
       "step": 3928
+    },
+    {
+      "epoch": 0.4002423859742032,
+      "grad_norm": 0.13630786538124084,
+      "learning_rate": 0.001,
+      "loss": 2.3567728996276855,
+      "step": 3930
+    },
+    {
+      "epoch": 0.40044607166681095,
+      "grad_norm": 0.12442337721586227,
+      "learning_rate": 0.001,
+      "loss": 2.147681713104248,
+      "step": 3932
+    },
+    {
+      "epoch": 0.40064975735941866,
+      "grad_norm": 0.1255188286304474,
+      "learning_rate": 0.001,
+      "loss": 2.102877140045166,
+      "step": 3934
+    },
+    {
+      "epoch": 0.4008534430520264,
+      "grad_norm": 0.10882001370191574,
+      "learning_rate": 0.001,
+      "loss": 2.22833251953125,
+      "step": 3936
+    },
+    {
+      "epoch": 0.4010571287446342,
+      "grad_norm": 0.14413021504878998,
+      "learning_rate": 0.001,
+      "loss": 2.2805018424987793,
+      "step": 3938
+    },
+    {
+      "epoch": 0.4012608144372419,
+      "grad_norm": 0.16915743052959442,
+      "learning_rate": 0.001,
+      "loss": 2.276557683944702,
+      "step": 3940
+    },
+    {
+      "epoch": 0.40146450012984963,
+      "grad_norm": 0.1425083875656128,
+      "learning_rate": 0.001,
+      "loss": 2.2889440059661865,
+      "step": 3942
+    },
+    {
+      "epoch": 0.4016681858224574,
+      "grad_norm": 0.12284961342811584,
+      "learning_rate": 0.001,
+      "loss": 2.2834134101867676,
+      "step": 3944
+    },
+    {
+      "epoch": 0.4018718715150651,
+      "grad_norm": 0.16576682031154633,
+      "learning_rate": 0.001,
+      "loss": 2.2051684856414795,
+      "step": 3946
+    },
+    {
+      "epoch": 0.40207555720767285,
+      "grad_norm": 0.14463412761688232,
+      "learning_rate": 0.001,
+      "loss": 2.1574690341949463,
+      "step": 3948
+    },
+    {
+      "epoch": 0.40227924290028055,
+      "grad_norm": 0.13224251568317413,
+      "learning_rate": 0.001,
+      "loss": 2.2559897899627686,
+      "step": 3950
+    },
+    {
+      "epoch": 0.4024829285928883,
+      "grad_norm": 0.12155351787805557,
+      "learning_rate": 0.001,
+      "loss": 2.3226184844970703,
+      "step": 3952
+    },
+    {
+      "epoch": 0.40268661428549607,
+      "grad_norm": 0.11629988253116608,
+      "learning_rate": 0.001,
+      "loss": 2.267862558364868,
+      "step": 3954
+    },
+    {
+      "epoch": 0.40289029997810377,
+      "grad_norm": 0.12121924012899399,
+      "learning_rate": 0.001,
+      "loss": 2.268049955368042,
+      "step": 3956
+    },
+    {
+      "epoch": 0.40309398567071153,
+      "grad_norm": 0.10794980078935623,
+      "learning_rate": 0.001,
+      "loss": 2.214012861251831,
+      "step": 3958
+    },
+    {
+      "epoch": 0.4032976713633193,
+      "grad_norm": 0.09973394870758057,
+      "learning_rate": 0.001,
+      "loss": 2.162726879119873,
+      "step": 3960
+    },
+    {
+      "epoch": 0.403501357055927,
+      "grad_norm": 0.12381641566753387,
+      "learning_rate": 0.001,
+      "loss": 2.2478909492492676,
+      "step": 3962
+    },
+    {
+      "epoch": 0.40370504274853475,
+      "grad_norm": 0.12303980439901352,
+      "learning_rate": 0.001,
+      "loss": 2.20383358001709,
+      "step": 3964
+    },
+    {
+      "epoch": 0.40390872844114245,
+      "grad_norm": 0.1427641212940216,
+      "learning_rate": 0.001,
+      "loss": 2.166118621826172,
+      "step": 3966
+    },
+    {
+      "epoch": 0.4041124141337502,
+      "grad_norm": 0.12834836542606354,
+      "learning_rate": 0.001,
+      "loss": 2.1520910263061523,
+      "step": 3968
+    },
+    {
+      "epoch": 0.40431609982635797,
+      "grad_norm": 0.15708638727664948,
+      "learning_rate": 0.001,
+      "loss": 2.2303786277770996,
+      "step": 3970
+    },
+    {
+      "epoch": 0.40451978551896567,
+      "grad_norm": 0.1256009340286255,
+      "learning_rate": 0.001,
+      "loss": 2.185821771621704,
+      "step": 3972
+    },
+    {
+      "epoch": 0.4047234712115734,
+      "grad_norm": 0.1487552970647812,
+      "learning_rate": 0.001,
+      "loss": 2.1567461490631104,
+      "step": 3974
+    },
+    {
+      "epoch": 0.4049271569041812,
+      "grad_norm": 0.1453331857919693,
+      "learning_rate": 0.001,
+      "loss": 2.222637176513672,
+      "step": 3976
+    },
+    {
+      "epoch": 0.4051308425967889,
+      "grad_norm": 0.1543055772781372,
+      "learning_rate": 0.001,
+      "loss": 2.2868423461914062,
+      "step": 3978
+    },
+    {
+      "epoch": 0.40533452828939665,
+      "grad_norm": 0.15814676880836487,
+      "learning_rate": 0.001,
+      "loss": 2.2537899017333984,
+      "step": 3980
+    },
+    {
+      "epoch": 0.40553821398200435,
+      "grad_norm": 0.15745288133621216,
+      "learning_rate": 0.001,
+      "loss": 2.209897041320801,
+      "step": 3982
+    },
+    {
+      "epoch": 0.4057418996746121,
+      "grad_norm": 0.13929052650928497,
+      "learning_rate": 0.001,
+      "loss": 2.324138641357422,
+      "step": 3984
+    },
+    {
+      "epoch": 0.40594558536721986,
+      "grad_norm": 0.12905153632164001,
+      "learning_rate": 0.001,
+      "loss": 2.1976261138916016,
+      "step": 3986
+    },
+    {
+      "epoch": 0.40614927105982757,
+      "grad_norm": 0.1111786738038063,
+      "learning_rate": 0.001,
+      "loss": 2.212836265563965,
+      "step": 3988
+    },
+    {
+      "epoch": 0.4063529567524353,
+      "grad_norm": 0.13105790317058563,
+      "learning_rate": 0.001,
+      "loss": 2.227750778198242,
+      "step": 3990
+    },
+    {
+      "epoch": 0.4065566424450431,
+      "grad_norm": 0.10568863153457642,
+      "learning_rate": 0.001,
+      "loss": 2.035661220550537,
+      "step": 3992
+    },
+    {
+      "epoch": 0.4067603281376508,
+      "grad_norm": 0.14333495497703552,
+      "learning_rate": 0.001,
+      "loss": 2.195624828338623,
+      "step": 3994
+    },
+    {
+      "epoch": 0.40696401383025854,
+      "grad_norm": 0.14519420266151428,
+      "learning_rate": 0.001,
+      "loss": 2.2171239852905273,
+      "step": 3996
+    },
+    {
+      "epoch": 0.40716769952286624,
+      "grad_norm": 0.13415592908859253,
+      "learning_rate": 0.001,
+      "loss": 2.1655707359313965,
+      "step": 3998
+    },
+    {
+      "epoch": 0.407371385215474,
+      "grad_norm": 0.13211412727832794,
+      "learning_rate": 0.001,
+      "loss": 2.1199684143066406,
+      "step": 4000
+    },
+    {
+      "epoch": 0.40757507090808176,
+      "grad_norm": 0.11821920424699783,
+      "learning_rate": 0.001,
+      "loss": 2.3044564723968506,
+      "step": 4002
+    },
+    {
+      "epoch": 0.40777875660068946,
+      "grad_norm": 0.10921493172645569,
+      "learning_rate": 0.001,
+      "loss": 2.080066680908203,
+      "step": 4004
+    },
+    {
+      "epoch": 0.4079824422932972,
+      "grad_norm": 0.12105704098939896,
+      "learning_rate": 0.001,
+      "loss": 2.1876745223999023,
+      "step": 4006
+    },
+    {
+      "epoch": 0.4081861279859049,
+      "grad_norm": 0.16559045016765594,
+      "learning_rate": 0.001,
+      "loss": 2.1563096046447754,
+      "step": 4008
+    },
+    {
+      "epoch": 0.4083898136785127,
+      "grad_norm": 0.1319076120853424,
+      "learning_rate": 0.001,
+      "loss": 2.1712636947631836,
+      "step": 4010
+    },
+    {
+      "epoch": 0.40859349937112044,
+      "grad_norm": 0.11300410330295563,
+      "learning_rate": 0.001,
+      "loss": 2.1742453575134277,
+      "step": 4012
+    },
+    {
+      "epoch": 0.40879718506372814,
+      "grad_norm": 0.11520037800073624,
+      "learning_rate": 0.001,
+      "loss": 2.1897411346435547,
+      "step": 4014
+    },
+    {
+      "epoch": 0.4090008707563359,
+      "grad_norm": 0.12819744646549225,
+      "learning_rate": 0.001,
+      "loss": 2.090991735458374,
+      "step": 4016
+    },
+    {
+      "epoch": 0.40920455644894366,
+      "grad_norm": 0.14020076394081116,
+      "learning_rate": 0.001,
+      "loss": 2.1483850479125977,
+      "step": 4018
+    },
+    {
+      "epoch": 0.40940824214155136,
+      "grad_norm": 0.1369781792163849,
+      "learning_rate": 0.001,
+      "loss": 2.158473491668701,
+      "step": 4020
+    },
+    {
+      "epoch": 0.4096119278341591,
+      "grad_norm": 0.13765113055706024,
+      "learning_rate": 0.001,
+      "loss": 2.182034492492676,
+      "step": 4022
+    },
+    {
+      "epoch": 0.4098156135267668,
+      "grad_norm": 0.1326528638601303,
+      "learning_rate": 0.001,
+      "loss": 2.143368721008301,
+      "step": 4024
+    },
+    {
+      "epoch": 0.4100192992193746,
+      "grad_norm": 0.104697585105896,
+      "learning_rate": 0.001,
+      "loss": 2.213972568511963,
+      "step": 4026
+    },
+    {
+      "epoch": 0.41022298491198234,
+      "grad_norm": 0.2335682064294815,
+      "learning_rate": 0.001,
+      "loss": 2.2350687980651855,
+      "step": 4028
+    },
+    {
+      "epoch": 0.41042667060459004,
+      "grad_norm": 0.13026103377342224,
+      "learning_rate": 0.001,
+      "loss": 2.2919516563415527,
+      "step": 4030
+    },
+    {
+      "epoch": 0.4106303562971978,
+      "grad_norm": 0.13204964995384216,
+      "learning_rate": 0.001,
+      "loss": 2.2775168418884277,
+      "step": 4032
+    },
+    {
+      "epoch": 0.41083404198980555,
+      "grad_norm": 0.16885264217853546,
+      "learning_rate": 0.001,
+      "loss": 2.20158052444458,
+      "step": 4034
+    },
+    {
+      "epoch": 0.41103772768241326,
+      "grad_norm": 0.12631377577781677,
+      "learning_rate": 0.001,
+      "loss": 2.232248306274414,
+      "step": 4036
+    },
+    {
+      "epoch": 0.411241413375021,
+      "grad_norm": 0.12087277323007584,
+      "learning_rate": 0.001,
+      "loss": 2.0278446674346924,
+      "step": 4038
+    },
+    {
+      "epoch": 0.4114450990676287,
+      "grad_norm": 0.11374824494123459,
+      "learning_rate": 0.001,
+      "loss": 2.150179624557495,
+      "step": 4040
+    },
+    {
+      "epoch": 0.4116487847602365,
+      "grad_norm": 0.11980725824832916,
+      "learning_rate": 0.001,
+      "loss": 2.178225517272949,
+      "step": 4042
+    },
+    {
+      "epoch": 0.41185247045284423,
+      "grad_norm": 0.11329711973667145,
+      "learning_rate": 0.001,
+      "loss": 2.2083945274353027,
+      "step": 4044
+    },
+    {
+      "epoch": 0.41205615614545194,
+      "grad_norm": 0.11443198472261429,
+      "learning_rate": 0.001,
+      "loss": 2.1106669902801514,
+      "step": 4046
+    },
+    {
+      "epoch": 0.4122598418380597,
+      "grad_norm": 0.12176729738712311,
+      "learning_rate": 0.001,
+      "loss": 2.2621235847473145,
+      "step": 4048
+    },
+    {
+      "epoch": 0.41246352753066745,
+      "grad_norm": 0.11784686148166656,
+      "learning_rate": 0.001,
+      "loss": 2.245405673980713,
+      "step": 4050
+    },
+    {
+      "epoch": 0.41266721322327515,
+      "grad_norm": 0.14302575588226318,
+      "learning_rate": 0.001,
+      "loss": 2.256922721862793,
+      "step": 4052
+    },
+    {
+      "epoch": 0.4128708989158829,
+      "grad_norm": 0.1184011772274971,
+      "learning_rate": 0.001,
+      "loss": 2.2725577354431152,
+      "step": 4054
+    },
+    {
+      "epoch": 0.4130745846084906,
+      "grad_norm": 0.12380281835794449,
+      "learning_rate": 0.001,
+      "loss": 2.257274627685547,
+      "step": 4056
+    },
+    {
+      "epoch": 0.41327827030109837,
+      "grad_norm": 0.17460589110851288,
+      "learning_rate": 0.001,
+      "loss": 2.2021567821502686,
+      "step": 4058
+    },
+    {
+      "epoch": 0.41348195599370613,
+      "grad_norm": 0.1132892295718193,
+      "learning_rate": 0.001,
+      "loss": 2.136749744415283,
+      "step": 4060
+    },
+    {
+      "epoch": 0.41368564168631383,
+      "grad_norm": 0.10061421245336533,
+      "learning_rate": 0.001,
+      "loss": 2.2540698051452637,
+      "step": 4062
+    },
+    {
+      "epoch": 0.4138893273789216,
+      "grad_norm": 0.12785644829273224,
+      "learning_rate": 0.001,
+      "loss": 2.262770414352417,
+      "step": 4064
+    },
+    {
+      "epoch": 0.41409301307152935,
+      "grad_norm": 0.11318495869636536,
+      "learning_rate": 0.001,
+      "loss": 2.201420783996582,
+      "step": 4066
+    },
+    {
+      "epoch": 0.41429669876413705,
+      "grad_norm": 0.11267337948083878,
+      "learning_rate": 0.001,
+      "loss": 2.288902997970581,
+      "step": 4068
+    },
+    {
+      "epoch": 0.4145003844567448,
+      "grad_norm": 0.13486526906490326,
+      "learning_rate": 0.001,
+      "loss": 2.110752820968628,
+      "step": 4070
+    },
+    {
+      "epoch": 0.4147040701493525,
+      "grad_norm": 0.16018477082252502,
+      "learning_rate": 0.001,
+      "loss": 2.2451491355895996,
+      "step": 4072
+    },
+    {
+      "epoch": 0.41490775584196027,
+      "grad_norm": 0.1452580839395523,
+      "learning_rate": 0.001,
+      "loss": 2.1350951194763184,
+      "step": 4074
+    },
+    {
+      "epoch": 0.415111441534568,
+      "grad_norm": 0.15626394748687744,
+      "learning_rate": 0.001,
+      "loss": 2.0829038619995117,
+      "step": 4076
+    },
+    {
+      "epoch": 0.41531512722717573,
+      "grad_norm": 0.1235283613204956,
+      "learning_rate": 0.001,
+      "loss": 2.2602500915527344,
+      "step": 4078
+    },
+    {
+      "epoch": 0.4155188129197835,
+      "grad_norm": 0.167628213763237,
+      "learning_rate": 0.001,
+      "loss": 2.299485683441162,
+      "step": 4080
+    },
+    {
+      "epoch": 0.41572249861239124,
+      "grad_norm": 0.14180535078048706,
+      "learning_rate": 0.001,
+      "loss": 2.1941943168640137,
+      "step": 4082
+    },
+    {
+      "epoch": 0.41592618430499895,
+      "grad_norm": 0.11531230807304382,
+      "learning_rate": 0.001,
+      "loss": 2.275689125061035,
+      "step": 4084
+    },
+    {
+      "epoch": 0.4161298699976067,
+      "grad_norm": 0.12500078976154327,
+      "learning_rate": 0.001,
+      "loss": 2.234076499938965,
+      "step": 4086
+    },
+    {
+      "epoch": 0.4163335556902144,
+      "grad_norm": 0.14937688410282135,
+      "learning_rate": 0.001,
+      "loss": 2.1778955459594727,
+      "step": 4088
+    },
+    {
+      "epoch": 0.41653724138282217,
+      "grad_norm": 0.23520752787590027,
+      "learning_rate": 0.001,
+      "loss": 2.2685680389404297,
+      "step": 4090
+    },
+    {
+      "epoch": 0.4167409270754299,
+      "grad_norm": 0.14653757214546204,
+      "learning_rate": 0.001,
+      "loss": 2.0472235679626465,
+      "step": 4092
+    },
+    {
+      "epoch": 0.4169446127680376,
+      "grad_norm": 0.11923457682132721,
+      "learning_rate": 0.001,
+      "loss": 2.2321295738220215,
+      "step": 4094
+    },
+    {
+      "epoch": 0.4171482984606454,
+      "grad_norm": 0.13525165617465973,
+      "learning_rate": 0.001,
+      "loss": 2.17832612991333,
+      "step": 4096
+    },
+    {
+      "epoch": 0.41735198415325314,
+      "grad_norm": 0.11429329216480255,
+      "learning_rate": 0.001,
+      "loss": 2.3208627700805664,
+      "step": 4098
+    },
+    {
+      "epoch": 0.41755566984586084,
+      "grad_norm": 0.13810522854328156,
+      "learning_rate": 0.001,
+      "loss": 2.1860547065734863,
+      "step": 4100
+    },
+    {
+      "epoch": 0.4177593555384686,
+      "grad_norm": 0.1656692922115326,
+      "learning_rate": 0.001,
+      "loss": 2.261390447616577,
+      "step": 4102
+    },
+    {
+      "epoch": 0.4179630412310763,
+      "grad_norm": 0.10786822438240051,
+      "learning_rate": 0.001,
+      "loss": 2.236171007156372,
+      "step": 4104
+    },
+    {
+      "epoch": 0.41816672692368406,
+      "grad_norm": 0.11479002237319946,
+      "learning_rate": 0.001,
+      "loss": 2.2370052337646484,
+      "step": 4106
+    },
+    {
+      "epoch": 0.4183704126162918,
+      "grad_norm": 0.12531642615795135,
+      "learning_rate": 0.001,
+      "loss": 2.2474379539489746,
+      "step": 4108
+    },
+    {
+      "epoch": 0.4185740983088995,
+      "grad_norm": 0.13567128777503967,
+      "learning_rate": 0.001,
+      "loss": 2.3014869689941406,
+      "step": 4110
+    },
+    {
+      "epoch": 0.4187777840015073,
+      "grad_norm": 0.15669982135295868,
+      "learning_rate": 0.001,
+      "loss": 2.2179782390594482,
+      "step": 4112
+    },
+    {
+      "epoch": 0.41898146969411504,
+      "grad_norm": 0.11240307241678238,
+      "learning_rate": 0.001,
+      "loss": 2.224113941192627,
+      "step": 4114
+    },
+    {
+      "epoch": 0.41918515538672274,
+      "grad_norm": 0.1360252946615219,
+      "learning_rate": 0.001,
+      "loss": 2.208083152770996,
+      "step": 4116
+    },
+    {
+      "epoch": 0.4193888410793305,
+      "grad_norm": 0.11606542021036148,
+      "learning_rate": 0.001,
+      "loss": 2.2967026233673096,
+      "step": 4118
+    },
+    {
+      "epoch": 0.4195925267719382,
+      "grad_norm": 0.13782162964344025,
+      "learning_rate": 0.001,
+      "loss": 2.2680647373199463,
+      "step": 4120
+    },
+    {
+      "epoch": 0.41979621246454596,
+      "grad_norm": 0.1258581429719925,
+      "learning_rate": 0.001,
+      "loss": 2.1320130825042725,
+      "step": 4122
+    },
+    {
+      "epoch": 0.4199998981571537,
+      "grad_norm": 0.12317005544900894,
+      "learning_rate": 0.001,
+      "loss": 2.075666904449463,
+      "step": 4124
+    },
+    {
+      "epoch": 0.4202035838497614,
+      "grad_norm": 0.11177681386470795,
+      "learning_rate": 0.001,
+      "loss": 2.208695888519287,
+      "step": 4126
+    },
+    {
+      "epoch": 0.4204072695423692,
+      "grad_norm": 0.10535506159067154,
+      "learning_rate": 0.001,
+      "loss": 2.062082529067993,
+      "step": 4128
+    },
+    {
+      "epoch": 0.4206109552349769,
+      "grad_norm": 0.1450238823890686,
+      "learning_rate": 0.001,
+      "loss": 2.297680377960205,
+      "step": 4130
+    },
+    {
+      "epoch": 0.42081464092758464,
+      "grad_norm": 0.14394697546958923,
+      "learning_rate": 0.001,
+      "loss": 2.2634053230285645,
+      "step": 4132
+    },
+    {
+      "epoch": 0.4210183266201924,
+      "grad_norm": 0.1193581372499466,
+      "learning_rate": 0.001,
+      "loss": 2.1792120933532715,
+      "step": 4134
+    },
+    {
+      "epoch": 0.4212220123128001,
+      "grad_norm": 0.10828955471515656,
+      "learning_rate": 0.001,
+      "loss": 2.197519302368164,
+      "step": 4136
+    },
+    {
+      "epoch": 0.42142569800540786,
+      "grad_norm": 0.14661340415477753,
+      "learning_rate": 0.001,
+      "loss": 2.3023147583007812,
+      "step": 4138
+    },
+    {
+      "epoch": 0.4216293836980156,
+      "grad_norm": 0.12358168512582779,
+      "learning_rate": 0.001,
+      "loss": 2.2646687030792236,
+      "step": 4140
+    },
+    {
+      "epoch": 0.4218330693906233,
+      "grad_norm": 0.1282944679260254,
+      "learning_rate": 0.001,
+      "loss": 2.177133798599243,
+      "step": 4142
+    },
+    {
+      "epoch": 0.4220367550832311,
+      "grad_norm": 0.11844725906848907,
+      "learning_rate": 0.001,
+      "loss": 2.174407482147217,
+      "step": 4144
+    },
+    {
+      "epoch": 0.4222404407758388,
+      "grad_norm": 0.11349865794181824,
+      "learning_rate": 0.001,
+      "loss": 2.351041793823242,
+      "step": 4146
+    },
+    {
+      "epoch": 0.42244412646844653,
+      "grad_norm": 0.11026953905820847,
+      "learning_rate": 0.001,
+      "loss": 2.1562111377716064,
+      "step": 4148
+    },
+    {
+      "epoch": 0.4226478121610543,
+      "grad_norm": 0.13089168071746826,
+      "learning_rate": 0.001,
+      "loss": 2.1656084060668945,
+      "step": 4150
+    },
+    {
+      "epoch": 0.422851497853662,
+      "grad_norm": 0.13755269348621368,
+      "learning_rate": 0.001,
+      "loss": 2.1976048946380615,
+      "step": 4152
+    },
+    {
+      "epoch": 0.42305518354626975,
+      "grad_norm": 0.12838555872440338,
+      "learning_rate": 0.001,
+      "loss": 2.2098851203918457,
+      "step": 4154
+    },
+    {
+      "epoch": 0.4232588692388775,
+      "grad_norm": 0.13100522756576538,
+      "learning_rate": 0.001,
+      "loss": 2.192282199859619,
+      "step": 4156
+    },
+    {
+      "epoch": 0.4234625549314852,
+      "grad_norm": 0.10611054301261902,
+      "learning_rate": 0.001,
+      "loss": 2.183992862701416,
+      "step": 4158
+    },
+    {
+      "epoch": 0.42366624062409297,
+      "grad_norm": 0.12371915578842163,
+      "learning_rate": 0.001,
+      "loss": 2.2089614868164062,
+      "step": 4160
+    },
+    {
+      "epoch": 0.4238699263167007,
+      "grad_norm": 0.1168675422668457,
+      "learning_rate": 0.001,
+      "loss": 2.21104097366333,
+      "step": 4162
+    },
+    {
+      "epoch": 0.42407361200930843,
+      "grad_norm": 0.12465793639421463,
+      "learning_rate": 0.001,
+      "loss": 2.2150745391845703,
+      "step": 4164
+    },
+    {
+      "epoch": 0.4242772977019162,
+      "grad_norm": 0.11862754076719284,
+      "learning_rate": 0.0009998231966053747,
+      "loss": 2.162632942199707,
+      "step": 4166
+    },
+    {
+      "epoch": 0.4244809833945239,
+      "grad_norm": 0.11651857942342758,
+      "learning_rate": 0.0009994695898161246,
+      "loss": 2.123439311981201,
+      "step": 4168
+    },
+    {
+      "epoch": 0.42468466908713165,
+      "grad_norm": 0.11577537655830383,
+      "learning_rate": 0.0009991159830268742,
+      "loss": 2.2830214500427246,
+      "step": 4170
+    },
+    {
+      "epoch": 0.4248883547797394,
+      "grad_norm": 0.1239021047949791,
+      "learning_rate": 0.0009987623762376238,
+      "loss": 2.246725559234619,
+      "step": 4172
+    },
+    {
+      "epoch": 0.4250920404723471,
+      "grad_norm": 0.13939408957958221,
+      "learning_rate": 0.0009984087694483734,
+      "loss": 2.1188125610351562,
+      "step": 4174
+    },
+    {
+      "epoch": 0.42529572616495487,
+      "grad_norm": 0.10905332118272781,
+      "learning_rate": 0.000998055162659123,
+      "loss": 2.359175443649292,
+      "step": 4176
+    },
+    {
+      "epoch": 0.42549941185756257,
+      "grad_norm": 0.11835073679685593,
+      "learning_rate": 0.0009977015558698728,
+      "loss": 2.1062569618225098,
+      "step": 4178
+    },
+    {
+      "epoch": 0.42570309755017033,
+      "grad_norm": 0.12922857701778412,
+      "learning_rate": 0.0009973479490806222,
+      "loss": 2.207723617553711,
+      "step": 4180
+    },
+    {
+      "epoch": 0.4259067832427781,
+      "grad_norm": 0.13281698524951935,
+      "learning_rate": 0.000996994342291372,
+      "loss": 2.1048521995544434,
+      "step": 4182
+    },
+    {
+      "epoch": 0.4261104689353858,
+      "grad_norm": 0.1781170517206192,
+      "learning_rate": 0.0009966407355021217,
+      "loss": 2.2977778911590576,
+      "step": 4184
+    },
+    {
+      "epoch": 0.42631415462799355,
+      "grad_norm": 0.11545410752296448,
+      "learning_rate": 0.0009962871287128713,
+      "loss": 2.2334914207458496,
+      "step": 4186
+    },
+    {
+      "epoch": 0.4265178403206013,
+      "grad_norm": 0.11393097788095474,
+      "learning_rate": 0.000995933521923621,
+      "loss": 2.222301483154297,
+      "step": 4188
+    },
+    {
+      "epoch": 0.426721526013209,
+      "grad_norm": 0.11272786557674408,
+      "learning_rate": 0.0009955799151343705,
+      "loss": 2.276956558227539,
+      "step": 4190
+    },
+    {
+      "epoch": 0.42692521170581677,
+      "grad_norm": 0.14239895343780518,
+      "learning_rate": 0.0009952263083451204,
+      "loss": 2.2302489280700684,
+      "step": 4192
+    },
+    {
+      "epoch": 0.42712889739842447,
+      "grad_norm": 0.13381977379322052,
+      "learning_rate": 0.0009948727015558698,
+      "loss": 2.2422430515289307,
+      "step": 4194
+    },
+    {
+      "epoch": 0.4273325830910322,
+      "grad_norm": 0.1054474413394928,
+      "learning_rate": 0.0009945190947666196,
+      "loss": 2.10978364944458,
+      "step": 4196
+    },
+    {
+      "epoch": 0.42753626878364,
+      "grad_norm": 0.11890409886837006,
+      "learning_rate": 0.0009941654879773692,
+      "loss": 2.2442421913146973,
+      "step": 4198
+    },
+    {
+      "epoch": 0.4277399544762477,
+      "grad_norm": 0.17273283004760742,
+      "learning_rate": 0.0009938118811881188,
+      "loss": 2.158938407897949,
+      "step": 4200
+    },
+    {
+      "epoch": 0.42794364016885544,
+      "grad_norm": 0.1469118446111679,
+      "learning_rate": 0.0009934582743988684,
+      "loss": 2.2233972549438477,
+      "step": 4202
+    },
+    {
+      "epoch": 0.4281473258614632,
+      "grad_norm": 0.1263427585363388,
+      "learning_rate": 0.000993104667609618,
+      "loss": 2.1840033531188965,
+      "step": 4204
+    },
+    {
+      "epoch": 0.4283510115540709,
+      "grad_norm": 0.13873954117298126,
+      "learning_rate": 0.0009927510608203679,
+      "loss": 2.3403615951538086,
+      "step": 4206
+    },
+    {
+      "epoch": 0.42855469724667866,
+      "grad_norm": 0.1893831342458725,
+      "learning_rate": 0.0009923974540311173,
+      "loss": 2.1420304775238037,
+      "step": 4208
+    },
+    {
+      "epoch": 0.42875838293928636,
+      "grad_norm": 0.11971204727888107,
+      "learning_rate": 0.0009920438472418671,
+      "loss": 2.156568765640259,
+      "step": 4210
+    },
+    {
+      "epoch": 0.4289620686318941,
+      "grad_norm": 0.11398480832576752,
+      "learning_rate": 0.0009916902404526167,
+      "loss": 2.142240524291992,
+      "step": 4212
+    },
+    {
+      "epoch": 0.4291657543245019,
+      "grad_norm": 0.14814355969429016,
+      "learning_rate": 0.0009913366336633663,
+      "loss": 2.2014827728271484,
+      "step": 4214
+    },
+    {
+      "epoch": 0.4293694400171096,
+      "grad_norm": 0.14438605308532715,
+      "learning_rate": 0.000990983026874116,
+      "loss": 2.284620523452759,
+      "step": 4216
+    },
+    {
+      "epoch": 0.42957312570971734,
+      "grad_norm": 0.11435892432928085,
+      "learning_rate": 0.0009906294200848656,
+      "loss": 2.195075511932373,
+      "step": 4218
+    },
+    {
+      "epoch": 0.4297768114023251,
+      "grad_norm": 0.18329235911369324,
+      "learning_rate": 0.0009902758132956154,
+      "loss": 2.2264962196350098,
+      "step": 4220
+    },
+    {
+      "epoch": 0.4299804970949328,
+      "grad_norm": 0.17036262154579163,
+      "learning_rate": 0.000989922206506365,
+      "loss": 2.233908176422119,
+      "step": 4222
+    },
+    {
+      "epoch": 0.43018418278754056,
+      "grad_norm": 0.11597452312707901,
+      "learning_rate": 0.0009895685997171146,
+      "loss": 2.2492361068725586,
+      "step": 4224
+    },
+    {
+      "epoch": 0.43038786848014826,
+      "grad_norm": 0.12845131754875183,
+      "learning_rate": 0.0009892149929278643,
+      "loss": 2.1409029960632324,
+      "step": 4226
+    },
+    {
+      "epoch": 0.430591554172756,
+      "grad_norm": 0.142255038022995,
+      "learning_rate": 0.0009888613861386139,
+      "loss": 2.2832655906677246,
+      "step": 4228
+    },
+    {
+      "epoch": 0.4307952398653638,
+      "grad_norm": 0.12171827256679535,
+      "learning_rate": 0.0009885077793493635,
+      "loss": 2.164909839630127,
+      "step": 4230
+    },
+    {
+      "epoch": 0.4309989255579715,
+      "grad_norm": 0.1148129403591156,
+      "learning_rate": 0.000988154172560113,
+      "loss": 2.2611358165740967,
+      "step": 4232
+    },
+    {
+      "epoch": 0.43120261125057924,
+      "grad_norm": 0.1508050560951233,
+      "learning_rate": 0.000987800565770863,
+      "loss": 2.241694688796997,
+      "step": 4234
+    },
+    {
+      "epoch": 0.431406296943187,
+      "grad_norm": 0.12122002243995667,
+      "learning_rate": 0.0009874469589816125,
+      "loss": 2.1215121746063232,
+      "step": 4236
+    },
+    {
+      "epoch": 0.4316099826357947,
+      "grad_norm": 0.11889788508415222,
+      "learning_rate": 0.0009870933521923622,
+      "loss": 2.3226418495178223,
+      "step": 4238
+    },
+    {
+      "epoch": 0.43181366832840246,
+      "grad_norm": 0.1672208309173584,
+      "learning_rate": 0.0009867397454031118,
+      "loss": 2.28900146484375,
+      "step": 4240
+    },
+    {
+      "epoch": 0.43201735402101016,
+      "grad_norm": 0.1263466626405716,
+      "learning_rate": 0.0009863861386138614,
+      "loss": 2.2135705947875977,
+      "step": 4242
+    },
+    {
+      "epoch": 0.4322210397136179,
+      "grad_norm": 0.1357012540102005,
+      "learning_rate": 0.000986032531824611,
+      "loss": 2.096940755844116,
+      "step": 4244
+    },
+    {
+      "epoch": 0.4324247254062257,
+      "grad_norm": 0.12798616290092468,
+      "learning_rate": 0.0009856789250353606,
+      "loss": 2.2253332138061523,
+      "step": 4246
+    },
+    {
+      "epoch": 0.4326284110988334,
+      "grad_norm": 0.1374821811914444,
+      "learning_rate": 0.0009853253182461105,
+      "loss": 2.1667656898498535,
+      "step": 4248
+    },
+    {
+      "epoch": 0.43283209679144113,
+      "grad_norm": 0.1020515114068985,
+      "learning_rate": 0.00098497171145686,
+      "loss": 2.3298535346984863,
+      "step": 4250
+    },
+    {
+      "epoch": 0.43303578248404884,
+      "grad_norm": 0.13508231937885284,
+      "learning_rate": 0.0009846181046676097,
+      "loss": 2.233224630355835,
+      "step": 4252
+    },
+    {
+      "epoch": 0.4332394681766566,
+      "grad_norm": 0.11288771033287048,
+      "learning_rate": 0.0009842644978783593,
+      "loss": 2.2855472564697266,
+      "step": 4254
+    },
+    {
+      "epoch": 0.43344315386926435,
+      "grad_norm": 0.1412867307662964,
+      "learning_rate": 0.000983910891089109,
+      "loss": 2.2136459350585938,
+      "step": 4256
+    },
+    {
+      "epoch": 0.43364683956187206,
+      "grad_norm": 0.17472001910209656,
+      "learning_rate": 0.0009835572842998585,
+      "loss": 2.2373223304748535,
+      "step": 4258
+    },
+    {
+      "epoch": 0.4338505252544798,
+      "grad_norm": 0.10899540781974792,
+      "learning_rate": 0.0009832036775106081,
+      "loss": 2.114041328430176,
+      "step": 4260
+    },
+    {
+      "epoch": 0.43405421094708757,
+      "grad_norm": 0.14947561919689178,
+      "learning_rate": 0.000982850070721358,
+      "loss": 2.330435276031494,
+      "step": 4262
+    },
+    {
+      "epoch": 0.4342578966396953,
+      "grad_norm": 0.13055101037025452,
+      "learning_rate": 0.0009824964639321076,
+      "loss": 1.9940898418426514,
+      "step": 4264
+    },
+    {
+      "epoch": 0.43446158233230303,
+      "grad_norm": 0.12248721718788147,
+      "learning_rate": 0.0009821428571428572,
+      "loss": 2.2250773906707764,
+      "step": 4266
+    },
+    {
+      "epoch": 0.43466526802491073,
+      "grad_norm": 0.14312873780727386,
+      "learning_rate": 0.0009817892503536068,
+      "loss": 2.152599811553955,
+      "step": 4268
+    },
+    {
+      "epoch": 0.4348689537175185,
+      "grad_norm": 0.1489303857088089,
+      "learning_rate": 0.0009814356435643564,
+      "loss": 2.0902323722839355,
+      "step": 4270
+    },
+    {
+      "epoch": 0.43507263941012625,
+      "grad_norm": 0.1320357620716095,
+      "learning_rate": 0.000981082036775106,
+      "loss": 2.154132604598999,
+      "step": 4272
+    },
+    {
+      "epoch": 0.43527632510273395,
+      "grad_norm": 0.14320127665996552,
+      "learning_rate": 0.0009807284299858557,
+      "loss": 2.138770818710327,
+      "step": 4274
+    },
+    {
+      "epoch": 0.4354800107953417,
+      "grad_norm": 0.12237825989723206,
+      "learning_rate": 0.0009803748231966055,
+      "loss": 2.190579891204834,
+      "step": 4276
+    },
+    {
+      "epoch": 0.43568369648794947,
+      "grad_norm": 0.1189723014831543,
+      "learning_rate": 0.0009800212164073551,
+      "loss": 2.231762409210205,
+      "step": 4278
+    },
+    {
+      "epoch": 0.43588738218055717,
+      "grad_norm": 0.1552569717168808,
+      "learning_rate": 0.0009796676096181047,
+      "loss": 2.219942569732666,
+      "step": 4280
+    },
+    {
+      "epoch": 0.43609106787316493,
+      "grad_norm": 0.14445845782756805,
+      "learning_rate": 0.0009793140028288543,
+      "loss": 2.2041656970977783,
+      "step": 4282
+    },
+    {
+      "epoch": 0.43629475356577263,
+      "grad_norm": 0.11309345066547394,
+      "learning_rate": 0.000978960396039604,
+      "loss": 2.2544217109680176,
+      "step": 4284
+    },
+    {
+      "epoch": 0.4364984392583804,
+      "grad_norm": 0.11555808037519455,
+      "learning_rate": 0.0009786067892503536,
+      "loss": 2.1852335929870605,
+      "step": 4286
+    },
+    {
+      "epoch": 0.43670212495098815,
+      "grad_norm": 0.12599119544029236,
+      "learning_rate": 0.0009782531824611032,
+      "loss": 2.219628095626831,
+      "step": 4288
+    },
+    {
+      "epoch": 0.43690581064359585,
+      "grad_norm": 0.10205013304948807,
+      "learning_rate": 0.000977899575671853,
+      "loss": 2.225231170654297,
+      "step": 4290
+    },
+    {
+      "epoch": 0.4371094963362036,
+      "grad_norm": 0.11422756314277649,
+      "learning_rate": 0.0009775459688826026,
+      "loss": 2.1353530883789062,
+      "step": 4292
+    },
+    {
+      "epoch": 0.43731318202881136,
+      "grad_norm": 0.18837063014507294,
+      "learning_rate": 0.0009771923620933523,
+      "loss": 2.1743812561035156,
+      "step": 4294
+    },
+    {
+      "epoch": 0.43751686772141907,
+      "grad_norm": 0.11036856472492218,
+      "learning_rate": 0.0009768387553041019,
+      "loss": 2.293053388595581,
+      "step": 4296
+    },
+    {
+      "epoch": 0.4377205534140268,
+      "grad_norm": 0.14470547437667847,
+      "learning_rate": 0.0009764851485148515,
+      "loss": 2.2285213470458984,
+      "step": 4298
+    },
+    {
+      "epoch": 0.4379242391066345,
+      "grad_norm": 0.12040532380342484,
+      "learning_rate": 0.0009761315417256011,
+      "loss": 2.1392831802368164,
+      "step": 4300
+    },
+    {
+      "epoch": 0.4381279247992423,
+      "grad_norm": 0.12665696442127228,
+      "learning_rate": 0.0009757779349363508,
+      "loss": 2.043128728866577,
+      "step": 4302
+    },
+    {
+      "epoch": 0.43833161049185004,
+      "grad_norm": 0.11461839824914932,
+      "learning_rate": 0.0009754243281471004,
+      "loss": 2.1160292625427246,
+      "step": 4304
+    },
+    {
+      "epoch": 0.43853529618445775,
+      "grad_norm": 0.1151294857263565,
+      "learning_rate": 0.0009750707213578502,
+      "loss": 2.1871838569641113,
+      "step": 4306
+    },
+    {
+      "epoch": 0.4387389818770655,
+      "grad_norm": 0.13056394457817078,
+      "learning_rate": 0.0009747171145685997,
+      "loss": 2.2960352897644043,
+      "step": 4308
+    },
+    {
+      "epoch": 0.43894266756967326,
+      "grad_norm": 0.12113704532384872,
+      "learning_rate": 0.0009743635077793494,
+      "loss": 2.1452252864837646,
+      "step": 4310
+    },
+    {
+      "epoch": 0.43914635326228096,
+      "grad_norm": 0.113646499812603,
+      "learning_rate": 0.000974009900990099,
+      "loss": 2.261399269104004,
+      "step": 4312
+    },
+    {
+      "epoch": 0.4393500389548887,
+      "grad_norm": 0.11756783723831177,
+      "learning_rate": 0.0009736562942008486,
+      "loss": 2.150895595550537,
+      "step": 4314
+    },
+    {
+      "epoch": 0.4395537246474964,
+      "grad_norm": 0.15133149921894073,
+      "learning_rate": 0.0009733026874115983,
+      "loss": 2.252561330795288,
+      "step": 4316
+    },
+    {
+      "epoch": 0.4397574103401042,
+      "grad_norm": 0.11054699122905731,
+      "learning_rate": 0.000972949080622348,
+      "loss": 2.166551113128662,
+      "step": 4318
+    },
+    {
+      "epoch": 0.43996109603271194,
+      "grad_norm": 0.11198518425226212,
+      "learning_rate": 0.0009725954738330977,
+      "loss": 2.3004539012908936,
+      "step": 4320
+    },
+    {
+      "epoch": 0.44016478172531964,
+      "grad_norm": 0.15852196514606476,
+      "learning_rate": 0.0009722418670438473,
+      "loss": 2.0939407348632812,
+      "step": 4322
+    },
+    {
+      "epoch": 0.4403684674179274,
+      "grad_norm": 0.11138569563627243,
+      "learning_rate": 0.0009718882602545969,
+      "loss": 2.2390646934509277,
+      "step": 4324
+    },
+    {
+      "epoch": 0.44057215311053516,
+      "grad_norm": 0.13162268698215485,
+      "learning_rate": 0.0009715346534653465,
+      "loss": 2.240595817565918,
+      "step": 4326
+    },
+    {
+      "epoch": 0.44077583880314286,
+      "grad_norm": 0.11763791739940643,
+      "learning_rate": 0.0009711810466760961,
+      "loss": 2.1473238468170166,
+      "step": 4328
+    },
+    {
+      "epoch": 0.4409795244957506,
+      "grad_norm": 0.15132999420166016,
+      "learning_rate": 0.0009708274398868459,
+      "loss": 2.350248336791992,
+      "step": 4330
+    },
+    {
+      "epoch": 0.4411832101883583,
+      "grad_norm": 0.11212446540594101,
+      "learning_rate": 0.0009704738330975955,
+      "loss": 2.2141082286834717,
+      "step": 4332
+    },
+    {
+      "epoch": 0.4413868958809661,
+      "grad_norm": 0.11372709274291992,
+      "learning_rate": 0.0009701202263083452,
+      "loss": 2.188967227935791,
+      "step": 4334
+    },
+    {
+      "epoch": 0.44159058157357384,
+      "grad_norm": 0.1144038587808609,
+      "learning_rate": 0.0009697666195190948,
+      "loss": 2.17398738861084,
+      "step": 4336
+    },
+    {
+      "epoch": 0.44179426726618154,
+      "grad_norm": 0.11627436429262161,
+      "learning_rate": 0.0009694130127298444,
+      "loss": 2.0848448276519775,
+      "step": 4338
+    },
+    {
+      "epoch": 0.4419979529587893,
+      "grad_norm": 0.12132667005062103,
+      "learning_rate": 0.000969059405940594,
+      "loss": 2.1416335105895996,
+      "step": 4340
+    },
+    {
+      "epoch": 0.44220163865139706,
+      "grad_norm": 0.12726832926273346,
+      "learning_rate": 0.0009687057991513437,
+      "loss": 2.2278149127960205,
+      "step": 4342
+    },
+    {
+      "epoch": 0.44240532434400476,
+      "grad_norm": 0.12594659626483917,
+      "learning_rate": 0.0009683521923620934,
+      "loss": 2.166984796524048,
+      "step": 4344
+    },
+    {
+      "epoch": 0.4426090100366125,
+      "grad_norm": 0.11619829386472702,
+      "learning_rate": 0.000967998585572843,
+      "loss": 2.1566598415374756,
+      "step": 4346
+    },
+    {
+      "epoch": 0.4428126957292202,
+      "grad_norm": 0.11630271375179291,
+      "learning_rate": 0.0009676449787835927,
+      "loss": 2.310114622116089,
+      "step": 4348
+    },
+    {
+      "epoch": 0.443016381421828,
+      "grad_norm": 0.11392687261104584,
+      "learning_rate": 0.0009672913719943423,
+      "loss": 2.1387219429016113,
+      "step": 4350
+    },
+    {
+      "epoch": 0.44322006711443573,
+      "grad_norm": 0.11599093675613403,
+      "learning_rate": 0.000966937765205092,
+      "loss": 2.1854066848754883,
+      "step": 4352
+    },
+    {
+      "epoch": 0.44342375280704344,
+      "grad_norm": 0.14233894646167755,
+      "learning_rate": 0.0009665841584158416,
+      "loss": 2.2717671394348145,
+      "step": 4354
+    },
+    {
+      "epoch": 0.4436274384996512,
+      "grad_norm": 0.10979294031858444,
+      "learning_rate": 0.0009662305516265912,
+      "loss": 2.1178131103515625,
+      "step": 4356
+    },
+    {
+      "epoch": 0.44383112419225895,
+      "grad_norm": 0.1332825869321823,
+      "learning_rate": 0.0009658769448373409,
+      "loss": 2.3106472492218018,
+      "step": 4358
+    },
+    {
+      "epoch": 0.44403480988486665,
+      "grad_norm": 0.12598037719726562,
+      "learning_rate": 0.0009655233380480905,
+      "loss": 2.2123067378997803,
+      "step": 4360
+    },
+    {
+      "epoch": 0.4442384955774744,
+      "grad_norm": 0.11085667461156845,
+      "learning_rate": 0.0009651697312588403,
+      "loss": 2.27787446975708,
+      "step": 4362
+    },
+    {
+      "epoch": 0.4444421812700821,
+      "grad_norm": 0.10593260824680328,
+      "learning_rate": 0.0009648161244695899,
+      "loss": 2.1992437839508057,
+      "step": 4364
+    },
+    {
+      "epoch": 0.4446458669626899,
+      "grad_norm": 0.11951328814029694,
+      "learning_rate": 0.0009644625176803395,
+      "loss": 2.2590012550354004,
+      "step": 4366
+    },
+    {
+      "epoch": 0.44484955265529763,
+      "grad_norm": 0.11717348545789719,
+      "learning_rate": 0.0009641089108910891,
+      "loss": 2.206866502761841,
+      "step": 4368
+    },
+    {
+      "epoch": 0.44505323834790533,
+      "grad_norm": 0.11078006029129028,
+      "learning_rate": 0.0009637553041018387,
+      "loss": 2.1674516201019287,
+      "step": 4370
+    },
+    {
+      "epoch": 0.4452569240405131,
+      "grad_norm": 0.11901329457759857,
+      "learning_rate": 0.0009634016973125884,
+      "loss": 2.3275575637817383,
+      "step": 4372
+    },
+    {
+      "epoch": 0.4454606097331208,
+      "grad_norm": 0.12545901536941528,
+      "learning_rate": 0.000963048090523338,
+      "loss": 2.1839704513549805,
+      "step": 4374
+    },
+    {
+      "epoch": 0.44566429542572855,
+      "grad_norm": 0.10773919522762299,
+      "learning_rate": 0.0009626944837340878,
+      "loss": 2.0161609649658203,
+      "step": 4376
+    },
+    {
+      "epoch": 0.4458679811183363,
+      "grad_norm": 0.13316288590431213,
+      "learning_rate": 0.0009623408769448374,
+      "loss": 2.3217997550964355,
+      "step": 4378
+    },
+    {
+      "epoch": 0.446071666810944,
+      "grad_norm": 0.1481369286775589,
+      "learning_rate": 0.000961987270155587,
+      "loss": 2.169943332672119,
+      "step": 4380
+    },
+    {
+      "epoch": 0.44627535250355177,
+      "grad_norm": 0.13584120571613312,
+      "learning_rate": 0.0009616336633663366,
+      "loss": 2.246072769165039,
+      "step": 4382
+    },
+    {
+      "epoch": 0.44647903819615953,
+      "grad_norm": 0.15168927609920502,
+      "learning_rate": 0.0009612800565770862,
+      "loss": 2.2057156562805176,
+      "step": 4384
+    },
+    {
+      "epoch": 0.44668272388876723,
+      "grad_norm": 0.11657849699258804,
+      "learning_rate": 0.000960926449787836,
+      "loss": 2.278423309326172,
+      "step": 4386
+    },
+    {
+      "epoch": 0.446886409581375,
+      "grad_norm": 0.11838920414447784,
+      "learning_rate": 0.0009605728429985856,
+      "loss": 1.998152256011963,
+      "step": 4388
+    },
+    {
+      "epoch": 0.4470900952739827,
+      "grad_norm": 0.18011502921581268,
+      "learning_rate": 0.0009602192362093353,
+      "loss": 2.292894124984741,
+      "step": 4390
+    },
+    {
+      "epoch": 0.44729378096659045,
+      "grad_norm": 0.1484185755252838,
+      "learning_rate": 0.0009598656294200849,
+      "loss": 2.2178573608398438,
+      "step": 4392
+    },
+    {
+      "epoch": 0.4474974666591982,
+      "grad_norm": 0.12086524069309235,
+      "learning_rate": 0.0009595120226308345,
+      "loss": 2.2623579502105713,
+      "step": 4394
+    },
+    {
+      "epoch": 0.4477011523518059,
+      "grad_norm": 0.1690281629562378,
+      "learning_rate": 0.0009591584158415841,
+      "loss": 2.320298194885254,
+      "step": 4396
+    },
+    {
+      "epoch": 0.44790483804441367,
+      "grad_norm": 0.15073975920677185,
+      "learning_rate": 0.0009588048090523338,
+      "loss": 2.1139626502990723,
+      "step": 4398
+    },
+    {
+      "epoch": 0.4481085237370214,
+      "grad_norm": 0.13161656260490417,
+      "learning_rate": 0.0009584512022630835,
+      "loss": 2.252819538116455,
+      "step": 4400
+    },
+    {
+      "epoch": 0.4483122094296291,
+      "grad_norm": 0.14897215366363525,
+      "learning_rate": 0.0009580975954738331,
+      "loss": 2.2076938152313232,
+      "step": 4402
+    },
+    {
+      "epoch": 0.4485158951222369,
+      "grad_norm": 0.20672737061977386,
+      "learning_rate": 0.0009577439886845828,
+      "loss": 2.2520596981048584,
+      "step": 4404
+    },
+    {
+      "epoch": 0.4487195808148446,
+      "grad_norm": 0.12532411515712738,
+      "learning_rate": 0.0009573903818953324,
+      "loss": 2.208359956741333,
+      "step": 4406
+    },
+    {
+      "epoch": 0.44892326650745235,
+      "grad_norm": 0.11771109700202942,
+      "learning_rate": 0.000957036775106082,
+      "loss": 2.162736415863037,
+      "step": 4408
+    },
+    {
+      "epoch": 0.4491269522000601,
+      "grad_norm": 0.17962761223316193,
+      "learning_rate": 0.0009566831683168317,
+      "loss": 2.2087392807006836,
+      "step": 4410
+    },
+    {
+      "epoch": 0.4493306378926678,
+      "grad_norm": 0.11363206058740616,
+      "learning_rate": 0.0009563295615275813,
+      "loss": 2.1970815658569336,
+      "step": 4412
+    },
+    {
+      "epoch": 0.44953432358527556,
+      "grad_norm": 0.1161012277007103,
+      "learning_rate": 0.000955975954738331,
+      "loss": 2.201359272003174,
+      "step": 4414
+    },
+    {
+      "epoch": 0.4497380092778833,
+      "grad_norm": 0.1562829613685608,
+      "learning_rate": 0.0009556223479490806,
+      "loss": 2.1833038330078125,
+      "step": 4416
+    },
+    {
+      "epoch": 0.449941694970491,
+      "grad_norm": 0.12501488626003265,
+      "learning_rate": 0.0009552687411598303,
+      "loss": 2.2777647972106934,
+      "step": 4418
+    },
+    {
+      "epoch": 0.4501453806630988,
+      "grad_norm": 0.1460924744606018,
+      "learning_rate": 0.00095491513437058,
+      "loss": 2.3600897789001465,
+      "step": 4420
+    },
+    {
+      "epoch": 0.4503490663557065,
+      "grad_norm": 0.1303459256887436,
+      "learning_rate": 0.0009545615275813297,
+      "loss": 2.1813454627990723,
+      "step": 4422
+    },
+    {
+      "epoch": 0.45055275204831424,
+      "grad_norm": 0.12850989401340485,
+      "learning_rate": 0.0009542079207920792,
+      "loss": 2.2656354904174805,
+      "step": 4424
+    },
+    {
+      "epoch": 0.450756437740922,
+      "grad_norm": 0.13071349263191223,
+      "learning_rate": 0.0009538543140028288,
+      "loss": 2.276623249053955,
+      "step": 4426
+    },
+    {
+      "epoch": 0.4509601234335297,
+      "grad_norm": 0.11832624673843384,
+      "learning_rate": 0.0009535007072135785,
+      "loss": 2.14739990234375,
+      "step": 4428
+    },
+    {
+      "epoch": 0.45116380912613746,
+      "grad_norm": 0.11867155879735947,
+      "learning_rate": 0.0009531471004243281,
+      "loss": 2.1903433799743652,
+      "step": 4430
+    },
+    {
+      "epoch": 0.4513674948187452,
+      "grad_norm": 0.11709476262331009,
+      "learning_rate": 0.0009527934936350779,
+      "loss": 2.203371047973633,
+      "step": 4432
+    },
+    {
+      "epoch": 0.4515711805113529,
+      "grad_norm": 0.1188405305147171,
+      "learning_rate": 0.0009524398868458275,
+      "loss": 2.1705784797668457,
+      "step": 4434
+    },
+    {
+      "epoch": 0.4517748662039607,
+      "grad_norm": 0.11651252955198288,
+      "learning_rate": 0.0009520862800565772,
+      "loss": 2.209197521209717,
+      "step": 4436
+    },
+    {
+      "epoch": 0.4519785518965684,
+      "grad_norm": 0.1263955682516098,
+      "learning_rate": 0.0009517326732673267,
+      "loss": 2.2215709686279297,
+      "step": 4438
+    },
+    {
+      "epoch": 0.45218223758917614,
+      "grad_norm": 0.11652659624814987,
+      "learning_rate": 0.0009513790664780763,
+      "loss": 2.1509881019592285,
+      "step": 4440
+    },
+    {
+      "epoch": 0.4523859232817839,
+      "grad_norm": 0.11473740637302399,
+      "learning_rate": 0.000951025459688826,
+      "loss": 2.2191076278686523,
+      "step": 4442
+    },
+    {
+      "epoch": 0.4525896089743916,
+      "grad_norm": 0.1386810541152954,
+      "learning_rate": 0.0009506718528995757,
+      "loss": 2.2249457836151123,
+      "step": 4444
+    },
+    {
+      "epoch": 0.45279329466699936,
+      "grad_norm": 0.12152288854122162,
+      "learning_rate": 0.0009503182461103254,
+      "loss": 2.0329904556274414,
+      "step": 4446
+    },
+    {
+      "epoch": 0.4529969803596071,
+      "grad_norm": 0.2049301564693451,
+      "learning_rate": 0.000949964639321075,
+      "loss": 2.2301712036132812,
+      "step": 4448
+    },
+    {
+      "epoch": 0.4532006660522148,
+      "grad_norm": 0.11563771963119507,
+      "learning_rate": 0.0009496110325318247,
+      "loss": 2.157712459564209,
+      "step": 4450
+    },
+    {
+      "epoch": 0.4534043517448226,
+      "grad_norm": 0.13029226660728455,
+      "learning_rate": 0.0009492574257425742,
+      "loss": 2.167325019836426,
+      "step": 4452
+    },
+    {
+      "epoch": 0.4536080374374303,
+      "grad_norm": 0.16012638807296753,
+      "learning_rate": 0.0009489038189533238,
+      "loss": 2.1951656341552734,
+      "step": 4454
+    },
+    {
+      "epoch": 0.45381172313003804,
+      "grad_norm": 0.14239563047885895,
+      "learning_rate": 0.0009485502121640736,
+      "loss": 2.196239948272705,
+      "step": 4456
+    },
+    {
+      "epoch": 0.4540154088226458,
+      "grad_norm": 0.1368158608675003,
+      "learning_rate": 0.0009481966053748232,
+      "loss": 2.3310964107513428,
+      "step": 4458
+    },
+    {
+      "epoch": 0.4542190945152535,
+      "grad_norm": 0.11941059678792953,
+      "learning_rate": 0.0009478429985855729,
+      "loss": 2.2303993701934814,
+      "step": 4460
+    },
+    {
+      "epoch": 0.45442278020786125,
+      "grad_norm": 0.11550739407539368,
+      "learning_rate": 0.0009474893917963225,
+      "loss": 2.232518196105957,
+      "step": 4462
+    },
+    {
+      "epoch": 0.454626465900469,
+      "grad_norm": 0.1612805873155594,
+      "learning_rate": 0.0009471357850070722,
+      "loss": 2.2875092029571533,
+      "step": 4464
+    },
+    {
+      "epoch": 0.4548301515930767,
+      "grad_norm": 0.139884352684021,
+      "learning_rate": 0.0009467821782178218,
+      "loss": 2.242872714996338,
+      "step": 4466
+    },
+    {
+      "epoch": 0.4550338372856845,
+      "grad_norm": 0.12729321420192719,
+      "learning_rate": 0.0009464285714285714,
+      "loss": 2.256532907485962,
+      "step": 4468
+    },
+    {
+      "epoch": 0.4552375229782922,
+      "grad_norm": 0.1498723030090332,
+      "learning_rate": 0.0009460749646393211,
+      "loss": 2.1910200119018555,
+      "step": 4470
+    },
+    {
+      "epoch": 0.45544120867089993,
+      "grad_norm": 0.11553414165973663,
+      "learning_rate": 0.0009457213578500707,
+      "loss": 2.1738436222076416,
+      "step": 4472
+    },
+    {
+      "epoch": 0.4556448943635077,
+      "grad_norm": 0.11114083975553513,
+      "learning_rate": 0.0009453677510608204,
+      "loss": 2.148408889770508,
+      "step": 4474
+    },
+    {
+      "epoch": 0.4558485800561154,
+      "grad_norm": 0.11569429188966751,
+      "learning_rate": 0.00094501414427157,
+      "loss": 2.125425100326538,
+      "step": 4476
+    },
+    {
+      "epoch": 0.45605226574872315,
+      "grad_norm": 0.13036134839057922,
+      "learning_rate": 0.0009446605374823198,
+      "loss": 2.1818900108337402,
+      "step": 4478
+    },
+    {
+      "epoch": 0.4562559514413309,
+      "grad_norm": 0.16318650543689728,
+      "learning_rate": 0.0009443069306930693,
+      "loss": 2.113551378250122,
+      "step": 4480
+    },
+    {
+      "epoch": 0.4564596371339386,
+      "grad_norm": 0.1188010573387146,
+      "learning_rate": 0.0009439533239038189,
+      "loss": 2.118746757507324,
+      "step": 4482
+    },
+    {
+      "epoch": 0.45666332282654637,
+      "grad_norm": 0.1337585300207138,
+      "learning_rate": 0.0009435997171145686,
+      "loss": 2.317044258117676,
+      "step": 4484
+    },
+    {
+      "epoch": 0.45686700851915407,
+      "grad_norm": 0.10741809755563736,
+      "learning_rate": 0.0009432461103253182,
+      "loss": 2.2648725509643555,
+      "step": 4486
+    },
+    {
+      "epoch": 0.45707069421176183,
+      "grad_norm": 0.1303325891494751,
+      "learning_rate": 0.000942892503536068,
+      "loss": 2.079585075378418,
+      "step": 4488
+    },
+    {
+      "epoch": 0.4572743799043696,
+      "grad_norm": 0.13622227311134338,
+      "learning_rate": 0.0009425388967468176,
+      "loss": 2.187987804412842,
+      "step": 4490
+    },
+    {
+      "epoch": 0.4574780655969773,
+      "grad_norm": 0.13003699481487274,
+      "learning_rate": 0.0009421852899575673,
+      "loss": 2.205852508544922,
+      "step": 4492
+    },
+    {
+      "epoch": 0.45768175128958505,
+      "grad_norm": 0.15368115901947021,
+      "learning_rate": 0.0009418316831683168,
+      "loss": 2.2658562660217285,
+      "step": 4494
+    },
+    {
+      "epoch": 0.45788543698219275,
+      "grad_norm": 0.1095602959394455,
+      "learning_rate": 0.0009414780763790664,
+      "loss": 2.312866687774658,
+      "step": 4496
+    },
+    {
+      "epoch": 0.4580891226748005,
+      "grad_norm": 0.10826533287763596,
+      "learning_rate": 0.0009411244695898161,
+      "loss": 2.1380162239074707,
+      "step": 4498
+    },
+    {
+      "epoch": 0.45829280836740827,
+      "grad_norm": 0.132846400141716,
+      "learning_rate": 0.0009407708628005658,
+      "loss": 2.0847649574279785,
+      "step": 4500
+    },
+    {
+      "epoch": 0.45849649406001597,
+      "grad_norm": 0.149509996175766,
+      "learning_rate": 0.0009404172560113155,
+      "loss": 2.360865592956543,
+      "step": 4502
+    },
+    {
+      "epoch": 0.4587001797526237,
+      "grad_norm": 0.109349824488163,
+      "learning_rate": 0.0009400636492220651,
+      "loss": 2.1312148571014404,
+      "step": 4504
+    },
+    {
+      "epoch": 0.4589038654452315,
+      "grad_norm": 0.1262819916009903,
+      "learning_rate": 0.0009397100424328148,
+      "loss": 2.2583096027374268,
+      "step": 4506
+    },
+    {
+      "epoch": 0.4591075511378392,
+      "grad_norm": 0.13042666018009186,
+      "learning_rate": 0.0009393564356435643,
+      "loss": 2.2028722763061523,
+      "step": 4508
+    },
+    {
+      "epoch": 0.45931123683044695,
+      "grad_norm": 0.12320356070995331,
+      "learning_rate": 0.0009390028288543139,
+      "loss": 2.2139830589294434,
+      "step": 4510
+    },
+    {
+      "epoch": 0.45951492252305465,
+      "grad_norm": 0.11396641284227371,
+      "learning_rate": 0.0009386492220650637,
+      "loss": 2.1654281616210938,
+      "step": 4512
+    },
+    {
+      "epoch": 0.4597186082156624,
+      "grad_norm": 0.1073664203286171,
+      "learning_rate": 0.0009382956152758133,
+      "loss": 2.2789506912231445,
+      "step": 4514
+    },
+    {
+      "epoch": 0.45992229390827016,
+      "grad_norm": 0.12553894519805908,
+      "learning_rate": 0.000937942008486563,
+      "loss": 2.2695937156677246,
+      "step": 4516
+    },
+    {
+      "epoch": 0.46012597960087787,
+      "grad_norm": 0.12091857194900513,
+      "learning_rate": 0.0009375884016973126,
+      "loss": 2.2762794494628906,
+      "step": 4518
+    },
+    {
+      "epoch": 0.4603296652934856,
+      "grad_norm": 0.10877951979637146,
+      "learning_rate": 0.0009372347949080623,
+      "loss": 2.1890954971313477,
+      "step": 4520
+    },
+    {
+      "epoch": 0.4605333509860934,
+      "grad_norm": 0.1450071632862091,
+      "learning_rate": 0.000936881188118812,
+      "loss": 2.3168787956237793,
+      "step": 4522
+    },
+    {
+      "epoch": 0.4607370366787011,
+      "grad_norm": 0.11485420912504196,
+      "learning_rate": 0.0009365275813295615,
+      "loss": 2.184518337249756,
+      "step": 4524
+    },
+    {
+      "epoch": 0.46094072237130884,
+      "grad_norm": 0.1416822373867035,
+      "learning_rate": 0.0009361739745403112,
+      "loss": 2.2299935817718506,
+      "step": 4526
+    },
+    {
+      "epoch": 0.46114440806391654,
+      "grad_norm": 0.1386720985174179,
+      "learning_rate": 0.0009358203677510608,
+      "loss": 2.1448328495025635,
+      "step": 4528
+    },
+    {
+      "epoch": 0.4613480937565243,
+      "grad_norm": 0.11416134238243103,
+      "learning_rate": 0.0009354667609618105,
+      "loss": 2.1203298568725586,
+      "step": 4530
+    },
+    {
+      "epoch": 0.46155177944913206,
+      "grad_norm": 0.13752859830856323,
+      "learning_rate": 0.0009351131541725601,
+      "loss": 2.148169755935669,
+      "step": 4532
+    },
+    {
+      "epoch": 0.46175546514173976,
+      "grad_norm": 0.16493940353393555,
+      "learning_rate": 0.0009347595473833099,
+      "loss": 2.2548978328704834,
+      "step": 4534
+    },
+    {
+      "epoch": 0.4619591508343475,
+      "grad_norm": 0.14364545047283173,
+      "learning_rate": 0.0009344059405940595,
+      "loss": 2.228271484375,
+      "step": 4536
+    },
+    {
+      "epoch": 0.4621628365269553,
+      "grad_norm": 0.11524691432714462,
+      "learning_rate": 0.000934052333804809,
+      "loss": 2.2563278675079346,
+      "step": 4538
+    },
+    {
+      "epoch": 0.462366522219563,
+      "grad_norm": 0.1620008945465088,
+      "learning_rate": 0.0009336987270155587,
+      "loss": 2.2421956062316895,
+      "step": 4540
+    },
+    {
+      "epoch": 0.46257020791217074,
+      "grad_norm": 0.12945519387722015,
+      "learning_rate": 0.0009333451202263083,
+      "loss": 2.222079277038574,
+      "step": 4542
+    },
+    {
+      "epoch": 0.46277389360477844,
+      "grad_norm": 0.13271155953407288,
+      "learning_rate": 0.000932991513437058,
+      "loss": 2.117818593978882,
+      "step": 4544
+    },
+    {
+      "epoch": 0.4629775792973862,
+      "grad_norm": 0.1185743510723114,
+      "learning_rate": 0.0009326379066478077,
+      "loss": 2.2372612953186035,
+      "step": 4546
+    },
+    {
+      "epoch": 0.46318126498999396,
+      "grad_norm": 0.13853168487548828,
+      "learning_rate": 0.0009322842998585574,
+      "loss": 2.168034076690674,
+      "step": 4548
+    },
+    {
+      "epoch": 0.46338495068260166,
+      "grad_norm": 0.12362273782491684,
+      "learning_rate": 0.000931930693069307,
+      "loss": 2.260960102081299,
+      "step": 4550
+    },
+    {
+      "epoch": 0.4635886363752094,
+      "grad_norm": 0.13363778591156006,
+      "learning_rate": 0.0009315770862800565,
+      "loss": 2.0831398963928223,
+      "step": 4552
+    },
+    {
+      "epoch": 0.4637923220678172,
+      "grad_norm": 0.11144138127565384,
+      "learning_rate": 0.0009312234794908062,
+      "loss": 2.1213834285736084,
+      "step": 4554
+    },
+    {
+      "epoch": 0.4639960077604249,
+      "grad_norm": 0.1225903332233429,
+      "learning_rate": 0.0009308698727015558,
+      "loss": 2.2389798164367676,
+      "step": 4556
+    },
+    {
+      "epoch": 0.46419969345303264,
+      "grad_norm": 0.12594036757946014,
+      "learning_rate": 0.0009305162659123056,
+      "loss": 2.2691171169281006,
+      "step": 4558
+    },
+    {
+      "epoch": 0.46440337914564034,
+      "grad_norm": 0.14439961314201355,
+      "learning_rate": 0.0009301626591230552,
+      "loss": 2.322134256362915,
+      "step": 4560
+    },
+    {
+      "epoch": 0.4646070648382481,
+      "grad_norm": 0.12116039544343948,
+      "learning_rate": 0.0009298090523338049,
+      "loss": 2.2643065452575684,
+      "step": 4562
+    },
+    {
+      "epoch": 0.46481075053085585,
+      "grad_norm": 0.10569262504577637,
+      "learning_rate": 0.0009294554455445545,
+      "loss": 2.1556806564331055,
+      "step": 4564
+    },
+    {
+      "epoch": 0.46501443622346356,
+      "grad_norm": 0.13451336324214935,
+      "learning_rate": 0.000929101838755304,
+      "loss": 2.277400493621826,
+      "step": 4566
+    },
+    {
+      "epoch": 0.4652181219160713,
+      "grad_norm": 0.1093977838754654,
+      "learning_rate": 0.0009287482319660538,
+      "loss": 2.2632699012756348,
+      "step": 4568
+    },
+    {
+      "epoch": 0.4654218076086791,
+      "grad_norm": 0.1303178071975708,
+      "learning_rate": 0.0009283946251768034,
+      "loss": 2.1009511947631836,
+      "step": 4570
+    },
+    {
+      "epoch": 0.4656254933012868,
+      "grad_norm": 0.12130264937877655,
+      "learning_rate": 0.0009280410183875531,
+      "loss": 2.2588562965393066,
+      "step": 4572
+    },
+    {
+      "epoch": 0.46582917899389453,
+      "grad_norm": 0.138875812292099,
+      "learning_rate": 0.0009276874115983027,
+      "loss": 2.2083396911621094,
+      "step": 4574
+    },
+    {
+      "epoch": 0.46603286468650224,
+      "grad_norm": 0.11544418334960938,
+      "learning_rate": 0.0009273338048090524,
+      "loss": 2.1770074367523193,
+      "step": 4576
+    },
+    {
+      "epoch": 0.46623655037911,
+      "grad_norm": 0.11872239410877228,
+      "learning_rate": 0.000926980198019802,
+      "loss": 2.114316940307617,
+      "step": 4578
+    },
+    {
+      "epoch": 0.46644023607171775,
+      "grad_norm": 0.10941053926944733,
+      "learning_rate": 0.0009266265912305516,
+      "loss": 2.191135883331299,
+      "step": 4580
+    },
+    {
+      "epoch": 0.46664392176432545,
+      "grad_norm": 0.18240197002887726,
+      "learning_rate": 0.0009262729844413013,
+      "loss": 2.240601062774658,
+      "step": 4582
+    },
+    {
+      "epoch": 0.4668476074569332,
+      "grad_norm": 0.16108688712120056,
+      "learning_rate": 0.0009259193776520509,
+      "loss": 2.089489698410034,
+      "step": 4584
+    },
+    {
+      "epoch": 0.46705129314954097,
+      "grad_norm": 0.12510554492473602,
+      "learning_rate": 0.0009255657708628006,
+      "loss": 2.145279884338379,
+      "step": 4586
+    },
+    {
+      "epoch": 0.46725497884214867,
+      "grad_norm": 0.17591729760169983,
+      "learning_rate": 0.0009252121640735502,
+      "loss": 2.2700133323669434,
+      "step": 4588
+    },
+    {
+      "epoch": 0.46745866453475643,
+      "grad_norm": 0.13197676837444305,
+      "learning_rate": 0.0009248585572843,
+      "loss": 2.037383794784546,
+      "step": 4590
+    },
+    {
+      "epoch": 0.46766235022736413,
+      "grad_norm": 0.15310487151145935,
+      "learning_rate": 0.0009245049504950496,
+      "loss": 2.2234151363372803,
+      "step": 4592
+    },
+    {
+      "epoch": 0.4678660359199719,
+      "grad_norm": 0.21063393354415894,
+      "learning_rate": 0.0009241513437057991,
+      "loss": 2.2171249389648438,
+      "step": 4594
+    },
+    {
+      "epoch": 0.46806972161257965,
+      "grad_norm": 0.1246064305305481,
+      "learning_rate": 0.0009237977369165488,
+      "loss": 2.133751153945923,
+      "step": 4596
+    },
+    {
+      "epoch": 0.46827340730518735,
+      "grad_norm": 0.1334415078163147,
+      "learning_rate": 0.0009234441301272984,
+      "loss": 2.283499240875244,
+      "step": 4598
+    },
+    {
+      "epoch": 0.4684770929977951,
+      "grad_norm": 0.19017153978347778,
+      "learning_rate": 0.0009230905233380481,
+      "loss": 2.266451358795166,
+      "step": 4600
+    },
+    {
+      "epoch": 0.46868077869040287,
+      "grad_norm": 0.17900705337524414,
+      "learning_rate": 0.0009227369165487978,
+      "loss": 2.2166764736175537,
+      "step": 4602
+    },
+    {
+      "epoch": 0.46888446438301057,
+      "grad_norm": 0.11171939969062805,
+      "learning_rate": 0.0009223833097595475,
+      "loss": 2.082451343536377,
+      "step": 4604
+    },
+    {
+      "epoch": 0.4690881500756183,
+      "grad_norm": 0.1073419600725174,
+      "learning_rate": 0.0009220297029702971,
+      "loss": 2.0717008113861084,
+      "step": 4606
+    },
+    {
+      "epoch": 0.46929183576822603,
+      "grad_norm": 0.12128672748804092,
+      "learning_rate": 0.0009216760961810466,
+      "loss": 2.160865306854248,
+      "step": 4608
+    },
+    {
+      "epoch": 0.4694955214608338,
+      "grad_norm": 0.14447112381458282,
+      "learning_rate": 0.0009213224893917963,
+      "loss": 2.195280075073242,
+      "step": 4610
+    },
+    {
+      "epoch": 0.46969920715344154,
+      "grad_norm": 0.147905632853508,
+      "learning_rate": 0.0009209688826025459,
+      "loss": 2.271442413330078,
+      "step": 4612
+    },
+    {
+      "epoch": 0.46990289284604925,
+      "grad_norm": 0.11903314292430878,
+      "learning_rate": 0.0009206152758132957,
+      "loss": 2.2420997619628906,
+      "step": 4614
+    },
+    {
+      "epoch": 0.470106578538657,
+      "grad_norm": 0.12563195824623108,
+      "learning_rate": 0.0009202616690240453,
+      "loss": 2.2566580772399902,
+      "step": 4616
+    },
+    {
+      "epoch": 0.47031026423126476,
+      "grad_norm": 0.11203783005475998,
+      "learning_rate": 0.000919908062234795,
+      "loss": 2.1690585613250732,
+      "step": 4618
+    },
+    {
+      "epoch": 0.47051394992387247,
+      "grad_norm": 0.13423675298690796,
+      "learning_rate": 0.0009195544554455446,
+      "loss": 2.0896193981170654,
+      "step": 4620
+    },
+    {
+      "epoch": 0.4707176356164802,
+      "grad_norm": 0.16609716415405273,
+      "learning_rate": 0.0009192008486562942,
+      "loss": 2.2711873054504395,
+      "step": 4622
+    },
+    {
+      "epoch": 0.4709213213090879,
+      "grad_norm": 0.11692589521408081,
+      "learning_rate": 0.0009188472418670438,
+      "loss": 2.147109031677246,
+      "step": 4624
+    },
+    {
+      "epoch": 0.4711250070016957,
+      "grad_norm": 0.12507468461990356,
+      "learning_rate": 0.0009184936350777935,
+      "loss": 2.15876841545105,
+      "step": 4626
+    },
+    {
+      "epoch": 0.47132869269430344,
+      "grad_norm": 0.14635099470615387,
+      "learning_rate": 0.0009181400282885432,
+      "loss": 2.2498507499694824,
+      "step": 4628
+    },
+    {
+      "epoch": 0.47153237838691114,
+      "grad_norm": 0.1396169513463974,
+      "learning_rate": 0.0009177864214992928,
+      "loss": 2.241702079772949,
+      "step": 4630
+    },
+    {
+      "epoch": 0.4717360640795189,
+      "grad_norm": 0.1083715558052063,
+      "learning_rate": 0.0009174328147100425,
+      "loss": 2.183156967163086,
+      "step": 4632
+    },
+    {
+      "epoch": 0.4719397497721266,
+      "grad_norm": 0.10736341029405594,
+      "learning_rate": 0.0009170792079207921,
+      "loss": 2.1250216960906982,
+      "step": 4634
+    },
+    {
+      "epoch": 0.47214343546473436,
+      "grad_norm": 0.11951309442520142,
+      "learning_rate": 0.0009167256011315417,
+      "loss": 2.271010398864746,
+      "step": 4636
+    },
+    {
+      "epoch": 0.4723471211573421,
+      "grad_norm": 0.12312625348567963,
+      "learning_rate": 0.0009163719943422914,
+      "loss": 2.041473388671875,
+      "step": 4638
+    },
+    {
+      "epoch": 0.4725508068499498,
+      "grad_norm": 0.12153839319944382,
+      "learning_rate": 0.000916018387553041,
+      "loss": 2.297431230545044,
+      "step": 4640
+    },
+    {
+      "epoch": 0.4727544925425576,
+      "grad_norm": 0.1217447966337204,
+      "learning_rate": 0.0009156647807637907,
+      "loss": 2.1084144115448,
+      "step": 4642
+    },
+    {
+      "epoch": 0.47295817823516534,
+      "grad_norm": 0.11667545139789581,
+      "learning_rate": 0.0009153111739745403,
+      "loss": 2.2127456665039062,
+      "step": 4644
+    },
+    {
+      "epoch": 0.47316186392777304,
+      "grad_norm": 0.12402203679084778,
+      "learning_rate": 0.00091495756718529,
+      "loss": 2.1745026111602783,
+      "step": 4646
+    },
+    {
+      "epoch": 0.4733655496203808,
+      "grad_norm": 0.12920121848583221,
+      "learning_rate": 0.0009146039603960397,
+      "loss": 2.2726244926452637,
+      "step": 4648
+    },
+    {
+      "epoch": 0.4735692353129885,
+      "grad_norm": 0.14174428582191467,
+      "learning_rate": 0.0009142503536067893,
+      "loss": 2.243546962738037,
+      "step": 4650
+    },
+    {
+      "epoch": 0.47377292100559626,
+      "grad_norm": 0.12071418762207031,
+      "learning_rate": 0.0009138967468175389,
+      "loss": 2.2711455821990967,
+      "step": 4652
+    },
+    {
+      "epoch": 0.473976606698204,
+      "grad_norm": 0.12173871695995331,
+      "learning_rate": 0.0009135431400282885,
+      "loss": 2.041210651397705,
+      "step": 4654
+    },
+    {
+      "epoch": 0.4741802923908117,
+      "grad_norm": 0.1421707570552826,
+      "learning_rate": 0.0009131895332390382,
+      "loss": 2.224005699157715,
+      "step": 4656
+    },
+    {
+      "epoch": 0.4743839780834195,
+      "grad_norm": 0.11143438518047333,
+      "learning_rate": 0.0009128359264497878,
+      "loss": 2.262967824935913,
+      "step": 4658
+    },
+    {
+      "epoch": 0.47458766377602724,
+      "grad_norm": 0.1207672730088234,
+      "learning_rate": 0.0009124823196605376,
+      "loss": 2.249025821685791,
+      "step": 4660
+    },
+    {
+      "epoch": 0.47479134946863494,
+      "grad_norm": 0.11522098630666733,
+      "learning_rate": 0.0009121287128712872,
+      "loss": 2.2892255783081055,
+      "step": 4662
+    },
+    {
+      "epoch": 0.4749950351612427,
+      "grad_norm": 0.10159576684236526,
+      "learning_rate": 0.0009117751060820368,
+      "loss": 2.223292112350464,
+      "step": 4664
+    },
+    {
+      "epoch": 0.4751987208538504,
+      "grad_norm": 0.15874212980270386,
+      "learning_rate": 0.0009114214992927864,
+      "loss": 2.3133318424224854,
+      "step": 4666
+    },
+    {
+      "epoch": 0.47540240654645816,
+      "grad_norm": 0.11143343895673752,
+      "learning_rate": 0.000911067892503536,
+      "loss": 2.2569117546081543,
+      "step": 4668
+    },
+    {
+      "epoch": 0.4756060922390659,
+      "grad_norm": 0.1572769582271576,
+      "learning_rate": 0.0009107142857142857,
+      "loss": 2.356480121612549,
+      "step": 4670
+    },
+    {
+      "epoch": 0.4758097779316736,
+      "grad_norm": 0.11646872758865356,
+      "learning_rate": 0.0009103606789250354,
+      "loss": 2.153618574142456,
+      "step": 4672
+    },
+    {
+      "epoch": 0.4760134636242814,
+      "grad_norm": 0.1093180775642395,
+      "learning_rate": 0.0009100070721357851,
+      "loss": 2.2985758781433105,
+      "step": 4674
+    },
+    {
+      "epoch": 0.47621714931688913,
+      "grad_norm": 0.12663964927196503,
+      "learning_rate": 0.0009096534653465347,
+      "loss": 2.120077133178711,
+      "step": 4676
+    },
+    {
+      "epoch": 0.47642083500949683,
+      "grad_norm": 0.10697914659976959,
+      "learning_rate": 0.0009092998585572843,
+      "loss": 2.1009583473205566,
+      "step": 4678
+    },
+    {
+      "epoch": 0.4766245207021046,
+      "grad_norm": 0.1185147687792778,
+      "learning_rate": 0.0009089462517680339,
+      "loss": 2.1521239280700684,
+      "step": 4680
+    },
+    {
+      "epoch": 0.4768282063947123,
+      "grad_norm": 0.12941861152648926,
+      "learning_rate": 0.0009085926449787835,
+      "loss": 2.1537222862243652,
+      "step": 4682
+    },
+    {
+      "epoch": 0.47703189208732005,
+      "grad_norm": 0.12805911898612976,
+      "learning_rate": 0.0009082390381895333,
+      "loss": 2.3298213481903076,
+      "step": 4684
+    },
+    {
+      "epoch": 0.4772355777799278,
+      "grad_norm": 0.12878035008907318,
+      "learning_rate": 0.0009078854314002829,
+      "loss": 2.2663540840148926,
+      "step": 4686
+    },
+    {
+      "epoch": 0.4774392634725355,
+      "grad_norm": 0.14399686455726624,
+      "learning_rate": 0.0009075318246110326,
+      "loss": 2.165882110595703,
+      "step": 4688
+    },
+    {
+      "epoch": 0.47764294916514327,
+      "grad_norm": 0.18520574271678925,
+      "learning_rate": 0.0009071782178217822,
+      "loss": 2.157289981842041,
+      "step": 4690
+    },
+    {
+      "epoch": 0.47784663485775103,
+      "grad_norm": 0.1315893679857254,
+      "learning_rate": 0.0009068246110325318,
+      "loss": 2.270570755004883,
+      "step": 4692
+    },
+    {
+      "epoch": 0.47805032055035873,
+      "grad_norm": 0.1279367357492447,
+      "learning_rate": 0.0009064710042432815,
+      "loss": 2.1966519355773926,
+      "step": 4694
+    },
+    {
+      "epoch": 0.4782540062429665,
+      "grad_norm": 0.14573341608047485,
+      "learning_rate": 0.0009061173974540311,
+      "loss": 2.252955913543701,
+      "step": 4696
+    },
+    {
+      "epoch": 0.4784576919355742,
+      "grad_norm": 0.11123565584421158,
+      "learning_rate": 0.0009057637906647808,
+      "loss": 2.3083770275115967,
+      "step": 4698
+    },
+    {
+      "epoch": 0.47866137762818195,
+      "grad_norm": 0.12670496106147766,
+      "learning_rate": 0.0009054101838755304,
+      "loss": 2.1851253509521484,
+      "step": 4700
+    },
+    {
+      "epoch": 0.4788650633207897,
+      "grad_norm": 0.13188141584396362,
+      "learning_rate": 0.0009050565770862801,
+      "loss": 2.2099075317382812,
+      "step": 4702
+    },
+    {
+      "epoch": 0.4790687490133974,
+      "grad_norm": 0.09896662831306458,
+      "learning_rate": 0.0009047029702970297,
+      "loss": 2.035644054412842,
+      "step": 4704
+    },
+    {
+      "epoch": 0.47927243470600517,
+      "grad_norm": 0.17076706886291504,
+      "learning_rate": 0.0009043493635077794,
+      "loss": 2.3483753204345703,
+      "step": 4706
+    },
+    {
+      "epoch": 0.4794761203986129,
+      "grad_norm": 0.11471794545650482,
+      "learning_rate": 0.000903995756718529,
+      "loss": 2.1785130500793457,
+      "step": 4708
+    },
+    {
+      "epoch": 0.47967980609122063,
+      "grad_norm": 0.1200699731707573,
+      "learning_rate": 0.0009036421499292786,
+      "loss": 2.1699445247650146,
+      "step": 4710
+    },
+    {
+      "epoch": 0.4798834917838284,
+      "grad_norm": 0.11952436715364456,
+      "learning_rate": 0.0009032885431400283,
+      "loss": 2.0885415077209473,
+      "step": 4712
+    },
+    {
+      "epoch": 0.4800871774764361,
+      "grad_norm": 0.18040212988853455,
+      "learning_rate": 0.0009029349363507779,
+      "loss": 2.222677230834961,
+      "step": 4714
+    },
+    {
+      "epoch": 0.48029086316904385,
+      "grad_norm": 0.15156938135623932,
+      "learning_rate": 0.0009025813295615277,
+      "loss": 2.183131217956543,
+      "step": 4716
+    },
+    {
+      "epoch": 0.4804945488616516,
+      "grad_norm": 0.13971301913261414,
+      "learning_rate": 0.0009022277227722773,
+      "loss": 2.1118955612182617,
+      "step": 4718
+    },
+    {
+      "epoch": 0.4806982345542593,
+      "grad_norm": 0.16743847727775574,
+      "learning_rate": 0.0009018741159830269,
+      "loss": 2.149616241455078,
+      "step": 4720
+    },
+    {
+      "epoch": 0.48090192024686707,
+      "grad_norm": 0.12019199132919312,
+      "learning_rate": 0.0009015205091937766,
+      "loss": 2.1799120903015137,
+      "step": 4722
+    },
+    {
+      "epoch": 0.4811056059394748,
+      "grad_norm": 0.09825186431407928,
+      "learning_rate": 0.0009011669024045261,
+      "loss": 1.9995477199554443,
+      "step": 4724
+    },
+    {
+      "epoch": 0.4813092916320825,
+      "grad_norm": 0.15617519617080688,
+      "learning_rate": 0.0009008132956152758,
+      "loss": 2.1660380363464355,
+      "step": 4726
+    },
+    {
+      "epoch": 0.4815129773246903,
+      "grad_norm": 0.1688273698091507,
+      "learning_rate": 0.0009004596888260255,
+      "loss": 2.29870343208313,
+      "step": 4728
+    },
+    {
+      "epoch": 0.481716663017298,
+      "grad_norm": 0.1358804553747177,
+      "learning_rate": 0.0009001060820367752,
+      "loss": 2.2847900390625,
+      "step": 4730
+    },
+    {
+      "epoch": 0.48192034870990574,
+      "grad_norm": 0.1450638473033905,
+      "learning_rate": 0.0008997524752475248,
+      "loss": 2.121986150741577,
+      "step": 4732
+    },
+    {
+      "epoch": 0.4821240344025135,
+      "grad_norm": 0.13609428703784943,
+      "learning_rate": 0.0008993988684582744,
+      "loss": 2.3344759941101074,
+      "step": 4734
+    },
+    {
+      "epoch": 0.4823277200951212,
+      "grad_norm": 0.11523746699094772,
+      "learning_rate": 0.0008990452616690241,
+      "loss": 2.2089929580688477,
+      "step": 4736
+    },
+    {
+      "epoch": 0.48253140578772896,
+      "grad_norm": 0.12423786520957947,
+      "learning_rate": 0.0008986916548797736,
+      "loss": 2.365591049194336,
+      "step": 4738
+    },
+    {
+      "epoch": 0.4827350914803367,
+      "grad_norm": 0.13455383479595184,
+      "learning_rate": 0.0008983380480905234,
+      "loss": 2.1496877670288086,
+      "step": 4740
+    },
+    {
+      "epoch": 0.4829387771729444,
+      "grad_norm": 0.1088300421833992,
+      "learning_rate": 0.000897984441301273,
+      "loss": 2.2358715534210205,
+      "step": 4742
+    },
+    {
+      "epoch": 0.4831424628655522,
+      "grad_norm": 0.10394295305013657,
+      "learning_rate": 0.0008976308345120227,
+      "loss": 2.2629523277282715,
+      "step": 4744
+    },
+    {
+      "epoch": 0.4833461485581599,
+      "grad_norm": 0.11890991032123566,
+      "learning_rate": 0.0008972772277227723,
+      "loss": 2.300826072692871,
+      "step": 4746
+    },
+    {
+      "epoch": 0.48354983425076764,
+      "grad_norm": 0.10701869428157806,
+      "learning_rate": 0.0008969236209335219,
+      "loss": 2.0974245071411133,
+      "step": 4748
+    },
+    {
+      "epoch": 0.4837535199433754,
+      "grad_norm": 0.12070364505052567,
+      "learning_rate": 0.0008965700141442717,
+      "loss": 2.255922555923462,
+      "step": 4750
+    },
+    {
+      "epoch": 0.4839572056359831,
+      "grad_norm": 0.1318737268447876,
+      "learning_rate": 0.0008962164073550212,
+      "loss": 2.2108776569366455,
+      "step": 4752
+    },
+    {
+      "epoch": 0.48416089132859086,
+      "grad_norm": 0.12049932777881622,
+      "learning_rate": 0.0008958628005657709,
+      "loss": 2.270115852355957,
+      "step": 4754
+    },
+    {
+      "epoch": 0.48436457702119856,
+      "grad_norm": 0.11139902472496033,
+      "learning_rate": 0.0008955091937765205,
+      "loss": 2.2738208770751953,
+      "step": 4756
+    },
+    {
+      "epoch": 0.4845682627138063,
+      "grad_norm": 0.12910480797290802,
+      "learning_rate": 0.0008951555869872702,
+      "loss": 2.0943491458892822,
+      "step": 4758
+    },
+    {
+      "epoch": 0.4847719484064141,
+      "grad_norm": 0.12974999845027924,
+      "learning_rate": 0.0008948019801980198,
+      "loss": 2.145627021789551,
+      "step": 4760
+    },
+    {
+      "epoch": 0.4849756340990218,
+      "grad_norm": 0.11960352212190628,
+      "learning_rate": 0.0008944483734087695,
+      "loss": 2.1442575454711914,
+      "step": 4762
+    },
+    {
+      "epoch": 0.48517931979162954,
+      "grad_norm": 0.15973255038261414,
+      "learning_rate": 0.0008940947666195192,
+      "loss": 2.1510133743286133,
+      "step": 4764
+    },
+    {
+      "epoch": 0.4853830054842373,
+      "grad_norm": 0.12670853734016418,
+      "learning_rate": 0.0008937411598302687,
+      "loss": 2.19404935836792,
+      "step": 4766
+    },
+    {
+      "epoch": 0.485586691176845,
+      "grad_norm": 0.11740083247423172,
+      "learning_rate": 0.0008933875530410184,
+      "loss": 2.1237831115722656,
+      "step": 4768
+    },
+    {
+      "epoch": 0.48579037686945276,
+      "grad_norm": 0.16061419248580933,
+      "learning_rate": 0.000893033946251768,
+      "loss": 2.2734453678131104,
+      "step": 4770
+    },
+    {
+      "epoch": 0.48599406256206046,
+      "grad_norm": 0.1250544935464859,
+      "learning_rate": 0.0008926803394625177,
+      "loss": 2.1865363121032715,
+      "step": 4772
+    },
+    {
+      "epoch": 0.4861977482546682,
+      "grad_norm": 0.11283563822507858,
+      "learning_rate": 0.0008923267326732674,
+      "loss": 2.1140050888061523,
+      "step": 4774
+    },
+    {
+      "epoch": 0.486401433947276,
+      "grad_norm": 0.11459698528051376,
+      "learning_rate": 0.000891973125884017,
+      "loss": 2.11739444732666,
+      "step": 4776
+    },
+    {
+      "epoch": 0.4866051196398837,
+      "grad_norm": 0.12477733194828033,
+      "learning_rate": 0.0008916195190947667,
+      "loss": 2.2733802795410156,
+      "step": 4778
+    },
+    {
+      "epoch": 0.48680880533249143,
+      "grad_norm": 0.12662135064601898,
+      "learning_rate": 0.0008912659123055162,
+      "loss": 2.222902774810791,
+      "step": 4780
+    },
+    {
+      "epoch": 0.4870124910250992,
+      "grad_norm": 0.1317615807056427,
+      "learning_rate": 0.0008909123055162659,
+      "loss": 2.181884288787842,
+      "step": 4782
+    },
+    {
+      "epoch": 0.4872161767177069,
+      "grad_norm": 0.14360646903514862,
+      "learning_rate": 0.0008905586987270155,
+      "loss": 2.2596731185913086,
+      "step": 4784
+    },
+    {
+      "epoch": 0.48741986241031465,
+      "grad_norm": 0.10456658899784088,
+      "learning_rate": 0.0008902050919377653,
+      "loss": 2.2897191047668457,
+      "step": 4786
+    },
+    {
+      "epoch": 0.48762354810292236,
+      "grad_norm": 0.12053926289081573,
+      "learning_rate": 0.0008898514851485149,
+      "loss": 2.251544237136841,
+      "step": 4788
+    },
+    {
+      "epoch": 0.4878272337955301,
+      "grad_norm": 0.12110774964094162,
+      "learning_rate": 0.0008894978783592645,
+      "loss": 2.016080856323242,
+      "step": 4790
+    },
+    {
+      "epoch": 0.48803091948813787,
+      "grad_norm": 0.15052752196788788,
+      "learning_rate": 0.0008891442715700142,
+      "loss": 2.1524555683135986,
+      "step": 4792
+    },
+    {
+      "epoch": 0.4882346051807456,
+      "grad_norm": 0.12412973493337631,
+      "learning_rate": 0.0008887906647807637,
+      "loss": 1.9916410446166992,
+      "step": 4794
+    },
+    {
+      "epoch": 0.48843829087335333,
+      "grad_norm": 0.17985013127326965,
+      "learning_rate": 0.0008884370579915135,
+      "loss": 2.2016749382019043,
+      "step": 4796
+    },
+    {
+      "epoch": 0.4886419765659611,
+      "grad_norm": 0.10758385062217712,
+      "learning_rate": 0.0008880834512022631,
+      "loss": 2.0514492988586426,
+      "step": 4798
+    },
+    {
+      "epoch": 0.4888456622585688,
+      "grad_norm": 0.13457533717155457,
+      "learning_rate": 0.0008877298444130128,
+      "loss": 2.2183165550231934,
+      "step": 4800
+    },
+    {
+      "epoch": 0.48904934795117655,
+      "grad_norm": 0.13166062533855438,
+      "learning_rate": 0.0008873762376237624,
+      "loss": 2.349989175796509,
+      "step": 4802
+    },
+    {
+      "epoch": 0.48925303364378425,
+      "grad_norm": 0.14633508026599884,
+      "learning_rate": 0.000887022630834512,
+      "loss": 2.3118934631347656,
+      "step": 4804
+    },
+    {
+      "epoch": 0.489456719336392,
+      "grad_norm": 0.1214480921626091,
+      "learning_rate": 0.0008866690240452617,
+      "loss": 2.195310354232788,
+      "step": 4806
+    },
+    {
+      "epoch": 0.48966040502899977,
+      "grad_norm": 0.10843229293823242,
+      "learning_rate": 0.0008863154172560113,
+      "loss": 2.0960402488708496,
+      "step": 4808
+    },
+    {
+      "epoch": 0.48986409072160747,
+      "grad_norm": 0.10975231230258942,
+      "learning_rate": 0.000885961810466761,
+      "loss": 2.150974988937378,
+      "step": 4810
+    },
+    {
+      "epoch": 0.49006777641421523,
+      "grad_norm": 0.12202282249927521,
+      "learning_rate": 0.0008856082036775106,
+      "loss": 2.2255401611328125,
+      "step": 4812
+    },
+    {
+      "epoch": 0.490271462106823,
+      "grad_norm": 0.13704191148281097,
+      "learning_rate": 0.0008852545968882603,
+      "loss": 2.2655138969421387,
+      "step": 4814
+    },
+    {
+      "epoch": 0.4904751477994307,
+      "grad_norm": 0.12641043961048126,
+      "learning_rate": 0.0008849009900990099,
+      "loss": 2.1809802055358887,
+      "step": 4816
+    },
+    {
+      "epoch": 0.49067883349203845,
+      "grad_norm": 0.13467901945114136,
+      "learning_rate": 0.0008845473833097595,
+      "loss": 2.265690803527832,
+      "step": 4818
+    },
+    {
+      "epoch": 0.49088251918464615,
+      "grad_norm": 0.14265763759613037,
+      "learning_rate": 0.0008841937765205093,
+      "loss": 2.1960744857788086,
+      "step": 4820
+    },
+    {
+      "epoch": 0.4910862048772539,
+      "grad_norm": 0.132966086268425,
+      "learning_rate": 0.0008838401697312589,
+      "loss": 2.211818218231201,
+      "step": 4822
+    },
+    {
+      "epoch": 0.49128989056986166,
+      "grad_norm": 0.11924374848604202,
+      "learning_rate": 0.0008834865629420085,
+      "loss": 2.141209125518799,
+      "step": 4824
+    },
+    {
+      "epoch": 0.49149357626246937,
+      "grad_norm": 0.14768558740615845,
+      "learning_rate": 0.0008831329561527581,
+      "loss": 2.255418300628662,
+      "step": 4826
+    },
+    {
+      "epoch": 0.4916972619550771,
+      "grad_norm": 0.11781346797943115,
+      "learning_rate": 0.0008827793493635078,
+      "loss": 2.16153621673584,
+      "step": 4828
+    },
+    {
+      "epoch": 0.4919009476476849,
+      "grad_norm": 0.13667428493499756,
+      "learning_rate": 0.0008824257425742575,
+      "loss": 2.231654644012451,
+      "step": 4830
+    },
+    {
+      "epoch": 0.4921046333402926,
+      "grad_norm": 0.18724015355110168,
+      "learning_rate": 0.0008820721357850071,
+      "loss": 2.1190671920776367,
+      "step": 4832
+    },
+    {
+      "epoch": 0.49230831903290034,
+      "grad_norm": 0.10730881243944168,
+      "learning_rate": 0.0008817185289957568,
+      "loss": 2.1985535621643066,
+      "step": 4834
+    },
+    {
+      "epoch": 0.49251200472550805,
+      "grad_norm": 0.11654356122016907,
+      "learning_rate": 0.0008813649222065064,
+      "loss": 2.1441972255706787,
+      "step": 4836
+    },
+    {
+      "epoch": 0.4927156904181158,
+      "grad_norm": 0.17904196679592133,
+      "learning_rate": 0.000881011315417256,
+      "loss": 2.273231267929077,
+      "step": 4838
+    },
+    {
+      "epoch": 0.49291937611072356,
+      "grad_norm": 0.13681036233901978,
+      "learning_rate": 0.0008806577086280056,
+      "loss": 2.1762032508850098,
+      "step": 4840
+    },
+    {
+      "epoch": 0.49312306180333126,
+      "grad_norm": 0.1366841346025467,
+      "learning_rate": 0.0008803041018387554,
+      "loss": 2.1838529109954834,
+      "step": 4842
+    },
+    {
+      "epoch": 0.493326747495939,
+      "grad_norm": 0.11755393445491791,
+      "learning_rate": 0.000879950495049505,
+      "loss": 2.2875030040740967,
+      "step": 4844
+    },
+    {
+      "epoch": 0.4935304331885468,
+      "grad_norm": 0.21625502407550812,
+      "learning_rate": 0.0008795968882602546,
+      "loss": 2.234309196472168,
+      "step": 4846
+    },
+    {
+      "epoch": 0.4937341188811545,
+      "grad_norm": 0.12441275268793106,
+      "learning_rate": 0.0008792432814710043,
+      "loss": 2.179758071899414,
+      "step": 4848
+    },
+    {
+      "epoch": 0.49393780457376224,
+      "grad_norm": 0.18150700628757477,
+      "learning_rate": 0.0008788896746817539,
+      "loss": 2.3365015983581543,
+      "step": 4850
+    },
+    {
+      "epoch": 0.49414149026636994,
+      "grad_norm": 0.12365284562110901,
+      "learning_rate": 0.0008785360678925035,
+      "loss": 2.1963324546813965,
+      "step": 4852
+    },
+    {
+      "epoch": 0.4943451759589777,
+      "grad_norm": 0.11641213297843933,
+      "learning_rate": 0.0008781824611032532,
+      "loss": 2.0649592876434326,
+      "step": 4854
+    },
+    {
+      "epoch": 0.49454886165158546,
+      "grad_norm": 0.16246701776981354,
+      "learning_rate": 0.0008778288543140029,
+      "loss": 2.3083534240722656,
+      "step": 4856
+    },
+    {
+      "epoch": 0.49475254734419316,
+      "grad_norm": 0.11017903685569763,
+      "learning_rate": 0.0008774752475247525,
+      "loss": 2.2137868404388428,
+      "step": 4858
+    },
+    {
+      "epoch": 0.4949562330368009,
+      "grad_norm": 0.16723176836967468,
+      "learning_rate": 0.0008771216407355021,
+      "loss": 2.2528328895568848,
+      "step": 4860
+    },
+    {
+      "epoch": 0.4951599187294087,
+      "grad_norm": 0.15064918994903564,
+      "learning_rate": 0.0008767680339462518,
+      "loss": 2.2863686084747314,
+      "step": 4862
+    },
+    {
+      "epoch": 0.4953636044220164,
+      "grad_norm": 0.14373983442783356,
+      "learning_rate": 0.0008764144271570014,
+      "loss": 2.2215733528137207,
+      "step": 4864
+    },
+    {
+      "epoch": 0.49556729011462414,
+      "grad_norm": 0.12899497151374817,
+      "learning_rate": 0.0008760608203677511,
+      "loss": 2.208493709564209,
+      "step": 4866
+    },
+    {
+      "epoch": 0.49577097580723184,
+      "grad_norm": 0.11388517916202545,
+      "learning_rate": 0.0008757072135785007,
+      "loss": 2.022732734680176,
+      "step": 4868
+    },
+    {
+      "epoch": 0.4959746614998396,
+      "grad_norm": 0.12080218642950058,
+      "learning_rate": 0.0008753536067892504,
+      "loss": 2.283005475997925,
+      "step": 4870
+    },
+    {
+      "epoch": 0.49617834719244736,
+      "grad_norm": 0.11749328672885895,
+      "learning_rate": 0.000875,
+      "loss": 2.219442367553711,
+      "step": 4872
+    },
+    {
+      "epoch": 0.49638203288505506,
+      "grad_norm": 0.11574358493089676,
+      "learning_rate": 0.0008746463932107496,
+      "loss": 2.1388936042785645,
+      "step": 4874
+    },
+    {
+      "epoch": 0.4965857185776628,
+      "grad_norm": 0.16069753468036652,
+      "learning_rate": 0.0008742927864214994,
+      "loss": 2.187541961669922,
+      "step": 4876
+    },
+    {
+      "epoch": 0.4967894042702705,
+      "grad_norm": 0.1355918049812317,
+      "learning_rate": 0.000873939179632249,
+      "loss": 2.1153206825256348,
+      "step": 4878
+    },
+    {
+      "epoch": 0.4969930899628783,
+      "grad_norm": 0.13793177902698517,
+      "learning_rate": 0.0008735855728429986,
+      "loss": 2.145829200744629,
+      "step": 4880
+    },
+    {
+      "epoch": 0.49719677565548603,
+      "grad_norm": 0.19802600145339966,
+      "learning_rate": 0.0008732319660537482,
+      "loss": 2.2374377250671387,
+      "step": 4882
+    },
+    {
+      "epoch": 0.49740046134809374,
+      "grad_norm": 0.13155874609947205,
+      "learning_rate": 0.0008728783592644979,
+      "loss": 2.2675070762634277,
+      "step": 4884
+    },
+    {
+      "epoch": 0.4976041470407015,
+      "grad_norm": 0.13055989146232605,
+      "learning_rate": 0.0008725247524752475,
+      "loss": 2.155550956726074,
+      "step": 4886
+    },
+    {
+      "epoch": 0.49780783273330925,
+      "grad_norm": 0.16608566045761108,
+      "learning_rate": 0.0008721711456859972,
+      "loss": 2.138080596923828,
+      "step": 4888
+    },
+    {
+      "epoch": 0.49801151842591695,
+      "grad_norm": 0.13015252351760864,
+      "learning_rate": 0.0008718175388967469,
+      "loss": 2.043149948120117,
+      "step": 4890
+    },
+    {
+      "epoch": 0.4982152041185247,
+      "grad_norm": 0.12644249200820923,
+      "learning_rate": 0.0008714639321074965,
+      "loss": 2.1886301040649414,
+      "step": 4892
+    },
+    {
+      "epoch": 0.4984188898111324,
+      "grad_norm": 0.11616521328687668,
+      "learning_rate": 0.0008711103253182461,
+      "loss": 2.172900438308716,
+      "step": 4894
+    },
+    {
+      "epoch": 0.4986225755037402,
+      "grad_norm": 0.131742462515831,
+      "learning_rate": 0.0008707567185289957,
+      "loss": 2.175194501876831,
+      "step": 4896
+    },
+    {
+      "epoch": 0.49882626119634793,
+      "grad_norm": 0.15420173108577728,
+      "learning_rate": 0.0008704031117397454,
+      "loss": 2.171351432800293,
+      "step": 4898
+    },
+    {
+      "epoch": 0.49902994688895563,
+      "grad_norm": 0.11715130507946014,
+      "learning_rate": 0.0008700495049504951,
+      "loss": 2.103121280670166,
+      "step": 4900
+    },
+    {
+      "epoch": 0.4992336325815634,
+      "grad_norm": 0.1276465356349945,
+      "learning_rate": 0.0008696958981612447,
+      "loss": 2.1572413444519043,
+      "step": 4902
+    },
+    {
+      "epoch": 0.49943731827417115,
+      "grad_norm": 0.14851492643356323,
+      "learning_rate": 0.0008693422913719944,
+      "loss": 2.2446117401123047,
+      "step": 4904
+    },
+    {
+      "epoch": 0.49964100396677885,
+      "grad_norm": 0.16749806702136993,
+      "learning_rate": 0.000868988684582744,
+      "loss": 2.2180745601654053,
+      "step": 4906
+    },
+    {
+      "epoch": 0.4998446896593866,
+      "grad_norm": 0.13275262713432312,
+      "learning_rate": 0.0008686350777934936,
+      "loss": 2.238365650177002,
+      "step": 4908
+    },
+    {
+      "epoch": 0.5000483753519943,
+      "grad_norm": 0.12023303657770157,
+      "learning_rate": 0.0008682814710042432,
+      "loss": 2.0071849822998047,
+      "step": 4910
     }
   ],
   "logging_steps": 2,
       "attributes": {}
     }
   },
+  "total_flos": 1.5133462673573806e+18,
   "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null