Instructions to use craa/exceptions with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use craa/exceptions with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="craa/exceptions")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("craa/exceptions", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use craa/exceptions with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "craa/exceptions" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "craa/exceptions", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/craa/exceptions
- SGLang
How to use craa/exceptions with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "craa/exceptions" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "craa/exceptions", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "craa/exceptions" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "craa/exceptions", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use craa/exceptions with Docker Model Runner:
docker model run hf.co/craa/exceptions
Download exp3/high_2000_1208/trainer_state.json from craa/exceptions: direct link, hf CLI and curl.
- Browser
- Download file 468 kB
-
https://huggingface.co/craa/exceptions/resolve/main/exp3/high_2000_1208/trainer_state.json
- Command line
-
hf download hf://craa/exceptions/exp3/high_2000_1208/trainer_state.json
-
curl -L -o trainer_state.json https://huggingface.co/craa/exceptions/resolve/main/exp3/high_2000_1208/trainer_state.json
468 kB
Invalid JSON:Unexpected token 'I', ..."ad_norm": Infinity,
"... is not valid JSON
{
"best_global_step": 104000,
"best_metric": 3.2727417945861816,
"best_model_checkpoint": "/scratch/cl5625/exceptions/models/high_2000_1208/checkpoint-20000",
"epoch": 33.36929371231697,
"eval_steps": 1000,
"global_step": 124000,
"is_hyper_param_search": false,
"is_local_process_zero": true,
"is_world_process_zero": true,
"log_history": [
{
"epoch": 0.013458225667527994,
"grad_norm": 0.8966911435127258,
"learning_rate": 0.000294,
"loss": 8.6065,
"step": 50
},
{
"epoch": 0.026916451335055987,
"grad_norm": 1.2048732042312622,
"learning_rate": 0.0005939999999999999,
"loss": 6.8585,
"step": 100
},
{
"epoch": 0.04037467700258398,
"grad_norm": 0.9153667688369751,
"learning_rate": 0.0005998020735155513,
"loss": 6.3263,
"step": 150
},
{
"epoch": 0.053832902670111975,
"grad_norm": 1.4322563409805298,
"learning_rate": 0.0005996001077150935,
"loss": 6.0257,
"step": 200
},
{
"epoch": 0.06729112833763996,
"grad_norm": 1.2138304710388184,
"learning_rate": 0.0005993981419146358,
"loss": 5.865,
"step": 250
},
{
"epoch": 0.08074935400516796,
"grad_norm": 1.4077575206756592,
"learning_rate": 0.0005991961761141779,
"loss": 5.7384,
"step": 300
},
{
"epoch": 0.09420757967269595,
"grad_norm": 1.1934620141983032,
"learning_rate": 0.0005989942103137202,
"loss": 5.6093,
"step": 350
},
{
"epoch": 0.10766580534022395,
"grad_norm": 1.1857770681381226,
"learning_rate": 0.0005987922445132624,
"loss": 5.4578,
"step": 400
},
{
"epoch": 0.12112403100775193,
"grad_norm": 1.8576644659042358,
"learning_rate": 0.0005985902787128047,
"loss": 5.3535,
"step": 450
},
{
"epoch": 0.13458225667527993,
"grad_norm": 1.6441043615341187,
"learning_rate": 0.0005983883129123468,
"loss": 5.2506,
"step": 500
},
{
"epoch": 0.14804048234280792,
"grad_norm": 1.3366986513137817,
"learning_rate": 0.000598186347111889,
"loss": 5.14,
"step": 550
},
{
"epoch": 0.16149870801033592,
"grad_norm": 1.1094809770584106,
"learning_rate": 0.0005979843813114312,
"loss": 5.0689,
"step": 600
},
{
"epoch": 0.1749569336778639,
"grad_norm": 1.1344627141952515,
"learning_rate": 0.0005977824155109734,
"loss": 4.9848,
"step": 650
},
{
"epoch": 0.1884151593453919,
"grad_norm": 1.487619400024414,
"learning_rate": 0.0005975804497105157,
"loss": 4.9104,
"step": 700
},
{
"epoch": 0.2018733850129199,
"grad_norm": 1.3195019960403442,
"learning_rate": 0.0005973784839100578,
"loss": 4.8522,
"step": 750
},
{
"epoch": 0.2153316106804479,
"grad_norm": 1.1873126029968262,
"learning_rate": 0.0005971765181096001,
"loss": 4.7907,
"step": 800
},
{
"epoch": 0.2287898363479759,
"grad_norm": 0.8753162026405334,
"learning_rate": 0.0005969745523091422,
"loss": 4.7346,
"step": 850
},
{
"epoch": 0.24224806201550386,
"grad_norm": 1.0871766805648804,
"learning_rate": 0.0005967725865086845,
"loss": 4.6963,
"step": 900
},
{
"epoch": 0.2557062876830319,
"grad_norm": 1.1119087934494019,
"learning_rate": 0.0005965706207082267,
"loss": 4.6261,
"step": 950
},
{
"epoch": 0.26916451335055985,
"grad_norm": 1.053986668586731,
"learning_rate": 0.0005963686549077689,
"loss": 4.5929,
"step": 1000
},
{
"epoch": 0.26916451335055985,
"eval_accuracy": 0.2670213769207134,
"eval_loss": 4.52205228805542,
"eval_runtime": 90.1745,
"eval_samples_per_second": 199.746,
"eval_steps_per_second": 12.487,
"step": 1000
},
{
"epoch": 0.2826227390180879,
"grad_norm": 0.7153794169425964,
"learning_rate": 0.0005961666891073111,
"loss": 4.5503,
"step": 1050
},
{
"epoch": 0.29608096468561584,
"grad_norm": 0.7922793030738831,
"learning_rate": 0.0005959647233068533,
"loss": 4.508,
"step": 1100
},
{
"epoch": 0.30953919035314387,
"grad_norm": 0.9259533286094666,
"learning_rate": 0.0005957627575063955,
"loss": 4.4437,
"step": 1150
},
{
"epoch": 0.32299741602067183,
"grad_norm": 0.8578700423240662,
"learning_rate": 0.0005955607917059378,
"loss": 4.4307,
"step": 1200
},
{
"epoch": 0.3364556416881998,
"grad_norm": 0.851991593837738,
"learning_rate": 0.00059535882590548,
"loss": 4.397,
"step": 1250
},
{
"epoch": 0.3499138673557278,
"grad_norm": 0.8914237022399902,
"learning_rate": 0.0005951568601050221,
"loss": 4.3653,
"step": 1300
},
{
"epoch": 0.3633720930232558,
"grad_norm": 0.7544556856155396,
"learning_rate": 0.0005949548943045644,
"loss": 4.3258,
"step": 1350
},
{
"epoch": 0.3768303186907838,
"grad_norm": 0.9407439231872559,
"learning_rate": 0.0005947529285041066,
"loss": 4.3163,
"step": 1400
},
{
"epoch": 0.3902885443583118,
"grad_norm": 0.8338985443115234,
"learning_rate": 0.0005945509627036488,
"loss": 4.2708,
"step": 1450
},
{
"epoch": 0.4037467700258398,
"grad_norm": 0.6592851877212524,
"learning_rate": 0.000594348996903191,
"loss": 4.2533,
"step": 1500
},
{
"epoch": 0.4172049956933678,
"grad_norm": 0.9020254611968994,
"learning_rate": 0.0005941470311027332,
"loss": 4.2288,
"step": 1550
},
{
"epoch": 0.4306632213608958,
"grad_norm": 0.6539929509162903,
"learning_rate": 0.0005939450653022754,
"loss": 4.2256,
"step": 1600
},
{
"epoch": 0.44412144702842377,
"grad_norm": 0.6174359917640686,
"learning_rate": 0.0005937430995018177,
"loss": 4.1885,
"step": 1650
},
{
"epoch": 0.4575796726959518,
"grad_norm": 0.8427085876464844,
"learning_rate": 0.0005935411337013598,
"loss": 4.173,
"step": 1700
},
{
"epoch": 0.47103789836347976,
"grad_norm": 0.7832385301589966,
"learning_rate": 0.0005933391679009021,
"loss": 4.1612,
"step": 1750
},
{
"epoch": 0.4844961240310077,
"grad_norm": 0.7887812852859497,
"learning_rate": 0.0005931372021004442,
"loss": 4.1244,
"step": 1800
},
{
"epoch": 0.49795434969853575,
"grad_norm": 0.8242790102958679,
"learning_rate": 0.0005929352362999865,
"loss": 4.1335,
"step": 1850
},
{
"epoch": 0.5114125753660638,
"grad_norm": 0.6264904737472534,
"learning_rate": 0.0005927332704995287,
"loss": 4.1285,
"step": 1900
},
{
"epoch": 0.5248708010335917,
"grad_norm": 0.5883820652961731,
"learning_rate": 0.0005925313046990709,
"loss": 4.0959,
"step": 1950
},
{
"epoch": 0.5383290267011197,
"grad_norm": 0.6146206855773926,
"learning_rate": 0.0005923293388986131,
"loss": 4.0854,
"step": 2000
},
{
"epoch": 0.5383290267011197,
"eval_accuracy": 0.31779552922535226,
"eval_loss": 4.020822048187256,
"eval_runtime": 90.5526,
"eval_samples_per_second": 198.912,
"eval_steps_per_second": 12.435,
"step": 2000
},
{
"epoch": 0.5517872523686477,
"grad_norm": 0.6017832159996033,
"learning_rate": 0.0005921273730981553,
"loss": 4.0682,
"step": 2050
},
{
"epoch": 0.5652454780361758,
"grad_norm": 0.6516919732093811,
"learning_rate": 0.0005919254072976975,
"loss": 4.0639,
"step": 2100
},
{
"epoch": 0.5787037037037037,
"grad_norm": 0.7468438744544983,
"learning_rate": 0.0005917234414972398,
"loss": 4.0659,
"step": 2150
},
{
"epoch": 0.5921619293712317,
"grad_norm": 0.6093896627426147,
"learning_rate": 0.000591521475696782,
"loss": 4.0391,
"step": 2200
},
{
"epoch": 0.6056201550387597,
"grad_norm": 0.5305492877960205,
"learning_rate": 0.0005913195098963241,
"loss": 4.0327,
"step": 2250
},
{
"epoch": 0.6190783807062877,
"grad_norm": 0.5827754735946655,
"learning_rate": 0.0005911175440958664,
"loss": 4.018,
"step": 2300
},
{
"epoch": 0.6325366063738157,
"grad_norm": 0.6132373213768005,
"learning_rate": 0.0005909155782954086,
"loss": 4.0198,
"step": 2350
},
{
"epoch": 0.6459948320413437,
"grad_norm": 0.5304743051528931,
"learning_rate": 0.0005907136124949508,
"loss": 3.9967,
"step": 2400
},
{
"epoch": 0.6594530577088716,
"grad_norm": 0.6900047659873962,
"learning_rate": 0.000590511646694493,
"loss": 3.9902,
"step": 2450
},
{
"epoch": 0.6729112833763996,
"grad_norm": 0.612856388092041,
"learning_rate": 0.0005903096808940352,
"loss": 3.9779,
"step": 2500
},
{
"epoch": 0.6863695090439277,
"grad_norm": 0.6652615666389465,
"learning_rate": 0.0005901077150935774,
"loss": 3.9835,
"step": 2550
},
{
"epoch": 0.6998277347114557,
"grad_norm": 0.5629779100418091,
"learning_rate": 0.0005899057492931197,
"loss": 3.9563,
"step": 2600
},
{
"epoch": 0.7132859603789836,
"grad_norm": 0.5258368253707886,
"learning_rate": 0.0005897037834926618,
"loss": 3.9506,
"step": 2650
},
{
"epoch": 0.7267441860465116,
"grad_norm": 0.5157285928726196,
"learning_rate": 0.0005895018176922041,
"loss": 3.9386,
"step": 2700
},
{
"epoch": 0.7402024117140397,
"grad_norm": 0.5549437403678894,
"learning_rate": 0.0005892998518917462,
"loss": 3.937,
"step": 2750
},
{
"epoch": 0.7536606373815676,
"grad_norm": 0.6192649602890015,
"learning_rate": 0.0005890978860912885,
"loss": 3.9281,
"step": 2800
},
{
"epoch": 0.7671188630490956,
"grad_norm": 0.5465920567512512,
"learning_rate": 0.0005888959202908307,
"loss": 3.9218,
"step": 2850
},
{
"epoch": 0.7805770887166236,
"grad_norm": 0.4951372742652893,
"learning_rate": 0.0005886939544903729,
"loss": 3.9108,
"step": 2900
},
{
"epoch": 0.7940353143841516,
"grad_norm": 0.6514776945114136,
"learning_rate": 0.0005884919886899151,
"loss": 3.9119,
"step": 2950
},
{
"epoch": 0.8074935400516796,
"grad_norm": 0.5681062936782837,
"learning_rate": 0.0005882900228894573,
"loss": 3.8989,
"step": 3000
},
{
"epoch": 0.8074935400516796,
"eval_accuracy": 0.3335286803796382,
"eval_loss": 3.8467557430267334,
"eval_runtime": 89.2556,
"eval_samples_per_second": 201.803,
"eval_steps_per_second": 12.615,
"step": 3000
},
{
"epoch": 0.8209517657192076,
"grad_norm": 0.4961033761501312,
"learning_rate": 0.0005880880570889996,
"loss": 3.895,
"step": 3050
},
{
"epoch": 0.8344099913867356,
"grad_norm": 0.5452906489372253,
"learning_rate": 0.0005878860912885418,
"loss": 3.8861,
"step": 3100
},
{
"epoch": 0.8478682170542635,
"grad_norm": 0.5429787039756775,
"learning_rate": 0.000587684125488084,
"loss": 3.881,
"step": 3150
},
{
"epoch": 0.8613264427217916,
"grad_norm": 0.46722593903541565,
"learning_rate": 0.0005874821596876261,
"loss": 3.8706,
"step": 3200
},
{
"epoch": 0.8747846683893196,
"grad_norm": 0.5518487095832825,
"learning_rate": 0.0005872801938871684,
"loss": 3.8696,
"step": 3250
},
{
"epoch": 0.8882428940568475,
"grad_norm": 0.4713578224182129,
"learning_rate": 0.0005870782280867106,
"loss": 3.8552,
"step": 3300
},
{
"epoch": 0.9017011197243755,
"grad_norm": 0.4993533194065094,
"learning_rate": 0.0005868762622862528,
"loss": 3.8544,
"step": 3350
},
{
"epoch": 0.9151593453919036,
"grad_norm": 0.498419851064682,
"learning_rate": 0.000586674296485795,
"loss": 3.8421,
"step": 3400
},
{
"epoch": 0.9286175710594315,
"grad_norm": 0.5733362436294556,
"learning_rate": 0.0005864723306853373,
"loss": 3.8391,
"step": 3450
},
{
"epoch": 0.9420757967269595,
"grad_norm": 0.4247209429740906,
"learning_rate": 0.0005862703648848794,
"loss": 3.8337,
"step": 3500
},
{
"epoch": 0.9555340223944875,
"grad_norm": 0.43807166814804077,
"learning_rate": 0.0005860683990844217,
"loss": 3.8269,
"step": 3550
},
{
"epoch": 0.9689922480620154,
"grad_norm": 0.5553157329559326,
"learning_rate": 0.0005858664332839638,
"loss": 3.8157,
"step": 3600
},
{
"epoch": 0.9824504737295435,
"grad_norm": 0.463137149810791,
"learning_rate": 0.0005856644674835061,
"loss": 3.8169,
"step": 3650
},
{
"epoch": 0.9959086993970715,
"grad_norm": 0.455089271068573,
"learning_rate": 0.0005854625016830483,
"loss": 3.8122,
"step": 3700
},
{
"epoch": 1.009151593453919,
"grad_norm": 0.4528520703315735,
"learning_rate": 0.0005852605358825905,
"loss": 3.7817,
"step": 3750
},
{
"epoch": 1.022609819121447,
"grad_norm": 0.4980250895023346,
"learning_rate": 0.0005850585700821327,
"loss": 3.7517,
"step": 3800
},
{
"epoch": 1.036068044788975,
"grad_norm": 0.46913275122642517,
"learning_rate": 0.0005848566042816749,
"loss": 3.7477,
"step": 3850
},
{
"epoch": 1.049526270456503,
"grad_norm": 0.4094890058040619,
"learning_rate": 0.0005846546384812171,
"loss": 3.7329,
"step": 3900
},
{
"epoch": 1.062984496124031,
"grad_norm": 0.4407455325126648,
"learning_rate": 0.0005844526726807594,
"loss": 3.7414,
"step": 3950
},
{
"epoch": 1.076442721791559,
"grad_norm": 0.44919925928115845,
"learning_rate": 0.0005842507068803016,
"loss": 3.7397,
"step": 4000
},
{
"epoch": 1.076442721791559,
"eval_accuracy": 0.3441485845704951,
"eval_loss": 3.736539363861084,
"eval_runtime": 89.4001,
"eval_samples_per_second": 201.476,
"eval_steps_per_second": 12.595,
"step": 4000
},
{
"epoch": 1.089900947459087,
"grad_norm": 0.4611460566520691,
"learning_rate": 0.0005840487410798437,
"loss": 3.7266,
"step": 4050
},
{
"epoch": 1.103359173126615,
"grad_norm": 0.40715456008911133,
"learning_rate": 0.000583846775279386,
"loss": 3.7411,
"step": 4100
},
{
"epoch": 1.116817398794143,
"grad_norm": 0.45220470428466797,
"learning_rate": 0.0005836448094789282,
"loss": 3.7195,
"step": 4150
},
{
"epoch": 1.130275624461671,
"grad_norm": 0.46470025181770325,
"learning_rate": 0.0005834428436784704,
"loss": 3.7253,
"step": 4200
},
{
"epoch": 1.143733850129199,
"grad_norm": 0.42880234122276306,
"learning_rate": 0.0005832408778780126,
"loss": 3.7232,
"step": 4250
},
{
"epoch": 1.157192075796727,
"grad_norm": 0.45918765664100647,
"learning_rate": 0.0005830389120775548,
"loss": 3.7164,
"step": 4300
},
{
"epoch": 1.1706503014642549,
"grad_norm": 0.4585005044937134,
"learning_rate": 0.000582836946277097,
"loss": 3.7264,
"step": 4350
},
{
"epoch": 1.1841085271317828,
"grad_norm": 0.4340853989124298,
"learning_rate": 0.0005826349804766393,
"loss": 3.7137,
"step": 4400
},
{
"epoch": 1.197566752799311,
"grad_norm": 0.45874127745628357,
"learning_rate": 0.0005824330146761814,
"loss": 3.7157,
"step": 4450
},
{
"epoch": 1.211024978466839,
"grad_norm": 0.4334906339645386,
"learning_rate": 0.0005822310488757237,
"loss": 3.6989,
"step": 4500
},
{
"epoch": 1.224483204134367,
"grad_norm": 0.45151418447494507,
"learning_rate": 0.0005820290830752658,
"loss": 3.7126,
"step": 4550
},
{
"epoch": 1.237941429801895,
"grad_norm": 0.36777910590171814,
"learning_rate": 0.0005818271172748081,
"loss": 3.717,
"step": 4600
},
{
"epoch": 1.251399655469423,
"grad_norm": 0.4326784014701843,
"learning_rate": 0.0005816251514743503,
"loss": 3.7005,
"step": 4650
},
{
"epoch": 1.2648578811369509,
"grad_norm": 0.41935795545578003,
"learning_rate": 0.0005814231856738925,
"loss": 3.7017,
"step": 4700
},
{
"epoch": 1.2783161068044788,
"grad_norm": 0.44362640380859375,
"learning_rate": 0.0005812212198734347,
"loss": 3.6986,
"step": 4750
},
{
"epoch": 1.291774332472007,
"grad_norm": 0.4299990236759186,
"learning_rate": 0.0005810192540729769,
"loss": 3.6967,
"step": 4800
},
{
"epoch": 1.3052325581395348,
"grad_norm": 0.442768394947052,
"learning_rate": 0.0005808172882725192,
"loss": 3.6887,
"step": 4850
},
{
"epoch": 1.318690783807063,
"grad_norm": 0.42501920461654663,
"learning_rate": 0.0005806153224720614,
"loss": 3.6987,
"step": 4900
},
{
"epoch": 1.332149009474591,
"grad_norm": 0.5787553191184998,
"learning_rate": 0.0005804133566716036,
"loss": 3.6963,
"step": 4950
},
{
"epoch": 1.3456072351421189,
"grad_norm": 0.43269771337509155,
"learning_rate": 0.0005802113908711457,
"loss": 3.6972,
"step": 5000
},
{
"epoch": 1.3456072351421189,
"eval_accuracy": 0.3507485551054679,
"eval_loss": 3.6660983562469482,
"eval_runtime": 89.5086,
"eval_samples_per_second": 201.232,
"eval_steps_per_second": 12.58,
"step": 5000
},
{
"epoch": 1.3590654608096469,
"grad_norm": 0.40254127979278564,
"learning_rate": 0.000580009425070688,
"loss": 3.6895,
"step": 5050
},
{
"epoch": 1.3725236864771748,
"grad_norm": 0.41719353199005127,
"learning_rate": 0.0005798074592702302,
"loss": 3.6812,
"step": 5100
},
{
"epoch": 1.3859819121447028,
"grad_norm": 0.455530047416687,
"learning_rate": 0.0005796054934697724,
"loss": 3.679,
"step": 5150
},
{
"epoch": 1.3994401378122308,
"grad_norm": 0.38591596484184265,
"learning_rate": 0.0005794035276693146,
"loss": 3.6641,
"step": 5200
},
{
"epoch": 1.412898363479759,
"grad_norm": 0.46197277307510376,
"learning_rate": 0.0005792015618688568,
"loss": 3.6713,
"step": 5250
},
{
"epoch": 1.4263565891472867,
"grad_norm": 0.45006898045539856,
"learning_rate": 0.000578999596068399,
"loss": 3.6638,
"step": 5300
},
{
"epoch": 1.4398148148148149,
"grad_norm": 0.42323538661003113,
"learning_rate": 0.0005787976302679413,
"loss": 3.6613,
"step": 5350
},
{
"epoch": 1.4532730404823428,
"grad_norm": 0.4862610697746277,
"learning_rate": 0.0005785956644674834,
"loss": 3.6683,
"step": 5400
},
{
"epoch": 1.4667312661498708,
"grad_norm": 0.4114309847354889,
"learning_rate": 0.0005783936986670257,
"loss": 3.6687,
"step": 5450
},
{
"epoch": 1.4801894918173988,
"grad_norm": 0.4172651171684265,
"learning_rate": 0.0005781917328665678,
"loss": 3.668,
"step": 5500
},
{
"epoch": 1.4936477174849268,
"grad_norm": 0.3980918228626251,
"learning_rate": 0.0005779897670661101,
"loss": 3.659,
"step": 5550
},
{
"epoch": 1.507105943152455,
"grad_norm": 0.43365082144737244,
"learning_rate": 0.0005777878012656523,
"loss": 3.6637,
"step": 5600
},
{
"epoch": 1.5205641688199827,
"grad_norm": 0.39800316095352173,
"learning_rate": 0.0005775858354651945,
"loss": 3.6608,
"step": 5650
},
{
"epoch": 1.5340223944875109,
"grad_norm": 0.4080260396003723,
"learning_rate": 0.0005773838696647367,
"loss": 3.6434,
"step": 5700
},
{
"epoch": 1.5474806201550386,
"grad_norm": 0.3940686881542206,
"learning_rate": 0.000577181903864279,
"loss": 3.6455,
"step": 5750
},
{
"epoch": 1.5609388458225668,
"grad_norm": 0.3766889274120331,
"learning_rate": 0.0005769799380638212,
"loss": 3.6487,
"step": 5800
},
{
"epoch": 1.5743970714900948,
"grad_norm": 0.39470019936561584,
"learning_rate": 0.0005767779722633633,
"loss": 3.6451,
"step": 5850
},
{
"epoch": 1.5878552971576227,
"grad_norm": 0.41684457659721375,
"learning_rate": 0.0005765760064629056,
"loss": 3.6579,
"step": 5900
},
{
"epoch": 1.6013135228251507,
"grad_norm": 0.38935387134552,
"learning_rate": 0.0005763740406624477,
"loss": 3.6381,
"step": 5950
},
{
"epoch": 1.6147717484926787,
"grad_norm": 0.38764357566833496,
"learning_rate": 0.00057617207486199,
"loss": 3.6419,
"step": 6000
},
{
"epoch": 1.6147717484926787,
"eval_accuracy": 0.35599695875721904,
"eval_loss": 3.6079609394073486,
"eval_runtime": 89.4352,
"eval_samples_per_second": 201.397,
"eval_steps_per_second": 12.59,
"step": 6000
},
{
"epoch": 1.6282299741602069,
"grad_norm": 0.44735968112945557,
"learning_rate": 0.0005759701090615322,
"loss": 3.6374,
"step": 6050
},
{
"epoch": 1.6416881998277346,
"grad_norm": 0.4507206678390503,
"learning_rate": 0.0005757681432610744,
"loss": 3.6237,
"step": 6100
},
{
"epoch": 1.6551464254952628,
"grad_norm": 0.3999354839324951,
"learning_rate": 0.0005755661774606166,
"loss": 3.6312,
"step": 6150
},
{
"epoch": 1.6686046511627906,
"grad_norm": 0.4594976603984833,
"learning_rate": 0.0005753642116601588,
"loss": 3.6364,
"step": 6200
},
{
"epoch": 1.6820628768303187,
"grad_norm": 0.40472447872161865,
"learning_rate": 0.000575162245859701,
"loss": 3.6215,
"step": 6250
},
{
"epoch": 1.6955211024978467,
"grad_norm": 0.367737352848053,
"learning_rate": 0.0005749602800592433,
"loss": 3.6221,
"step": 6300
},
{
"epoch": 1.7089793281653747,
"grad_norm": 0.36534759402275085,
"learning_rate": 0.0005747583142587854,
"loss": 3.6185,
"step": 6350
},
{
"epoch": 1.7224375538329026,
"grad_norm": 0.3866477608680725,
"learning_rate": 0.0005745563484583277,
"loss": 3.6153,
"step": 6400
},
{
"epoch": 1.7358957795004306,
"grad_norm": 0.4261724352836609,
"learning_rate": 0.0005743543826578698,
"loss": 3.6152,
"step": 6450
},
{
"epoch": 1.7493540051679588,
"grad_norm": 0.3843180239200592,
"learning_rate": 0.0005741524168574121,
"loss": 3.6064,
"step": 6500
},
{
"epoch": 1.7628122308354865,
"grad_norm": 0.4083123803138733,
"learning_rate": 0.0005739504510569543,
"loss": 3.6175,
"step": 6550
},
{
"epoch": 1.7762704565030147,
"grad_norm": 0.38491323590278625,
"learning_rate": 0.0005737484852564966,
"loss": 3.6114,
"step": 6600
},
{
"epoch": 1.7897286821705425,
"grad_norm": 0.41620877385139465,
"learning_rate": 0.0005735465194560387,
"loss": 3.6221,
"step": 6650
},
{
"epoch": 1.8031869078380707,
"grad_norm": 0.334351509809494,
"learning_rate": 0.000573344553655581,
"loss": 3.5979,
"step": 6700
},
{
"epoch": 1.8166451335055986,
"grad_norm": 0.45867523550987244,
"learning_rate": 0.0005731425878551232,
"loss": 3.6177,
"step": 6750
},
{
"epoch": 1.8301033591731266,
"grad_norm": 0.3858473598957062,
"learning_rate": 0.0005729406220546653,
"loss": 3.6184,
"step": 6800
},
{
"epoch": 1.8435615848406546,
"grad_norm": 0.34941157698631287,
"learning_rate": 0.0005727386562542076,
"loss": 3.6057,
"step": 6850
},
{
"epoch": 1.8570198105081825,
"grad_norm": 0.364150732755661,
"learning_rate": 0.0005725366904537497,
"loss": 3.6044,
"step": 6900
},
{
"epoch": 1.8704780361757107,
"grad_norm": 0.41074541211128235,
"learning_rate": 0.000572334724653292,
"loss": 3.6055,
"step": 6950
},
{
"epoch": 1.8839362618432385,
"grad_norm": 0.3702612817287445,
"learning_rate": 0.0005721327588528342,
"loss": 3.5926,
"step": 7000
},
{
"epoch": 1.8839362618432385,
"eval_accuracy": 0.36019105332257295,
"eval_loss": 3.5668296813964844,
"eval_runtime": 89.053,
"eval_samples_per_second": 202.262,
"eval_steps_per_second": 12.644,
"step": 7000
},
{
"epoch": 1.8973944875107667,
"grad_norm": 0.36478501558303833,
"learning_rate": 0.0005719307930523764,
"loss": 3.5984,
"step": 7050
},
{
"epoch": 1.9108527131782944,
"grad_norm": 0.40379798412323,
"learning_rate": 0.0005717288272519186,
"loss": 3.5937,
"step": 7100
},
{
"epoch": 1.9243109388458226,
"grad_norm": 0.4081585109233856,
"learning_rate": 0.0005715268614514608,
"loss": 3.5918,
"step": 7150
},
{
"epoch": 1.9377691645133506,
"grad_norm": 0.3517591655254364,
"learning_rate": 0.000571324895651003,
"loss": 3.5924,
"step": 7200
},
{
"epoch": 1.9512273901808785,
"grad_norm": 0.37822774052619934,
"learning_rate": 0.0005711229298505453,
"loss": 3.5838,
"step": 7250
},
{
"epoch": 1.9646856158484065,
"grad_norm": 0.3480222821235657,
"learning_rate": 0.0005709209640500874,
"loss": 3.5794,
"step": 7300
},
{
"epoch": 1.9781438415159345,
"grad_norm": 0.3855062425136566,
"learning_rate": 0.0005707189982496297,
"loss": 3.5761,
"step": 7350
},
{
"epoch": 1.9916020671834627,
"grad_norm": 0.36663126945495605,
"learning_rate": 0.0005705170324491719,
"loss": 3.5871,
"step": 7400
},
{
"epoch": 2.00484496124031,
"grad_norm": 0.3746161460876465,
"learning_rate": 0.0005703150666487141,
"loss": 3.5418,
"step": 7450
},
{
"epoch": 2.018303186907838,
"grad_norm": 0.3908737897872925,
"learning_rate": 0.0005701131008482563,
"loss": 3.493,
"step": 7500
},
{
"epoch": 2.0317614125753662,
"grad_norm": 0.36380451917648315,
"learning_rate": 0.0005699111350477986,
"loss": 3.4989,
"step": 7550
},
{
"epoch": 2.045219638242894,
"grad_norm": 0.39646074175834656,
"learning_rate": 0.0005697091692473408,
"loss": 3.4959,
"step": 7600
},
{
"epoch": 2.058677863910422,
"grad_norm": 0.36035415530204773,
"learning_rate": 0.000569507203446883,
"loss": 3.4878,
"step": 7650
},
{
"epoch": 2.07213608957795,
"grad_norm": 0.3832433819770813,
"learning_rate": 0.0005693052376464252,
"loss": 3.5133,
"step": 7700
},
{
"epoch": 2.085594315245478,
"grad_norm": 0.3533892035484314,
"learning_rate": 0.0005691032718459673,
"loss": 3.5027,
"step": 7750
},
{
"epoch": 2.099052540913006,
"grad_norm": 0.3850933313369751,
"learning_rate": 0.0005689013060455096,
"loss": 3.5023,
"step": 7800
},
{
"epoch": 2.112510766580534,
"grad_norm": 0.3618728816509247,
"learning_rate": 0.0005686993402450518,
"loss": 3.5023,
"step": 7850
},
{
"epoch": 2.125968992248062,
"grad_norm": 0.36867016553878784,
"learning_rate": 0.000568497374444594,
"loss": 3.5004,
"step": 7900
},
{
"epoch": 2.13942721791559,
"grad_norm": 0.3630622625350952,
"learning_rate": 0.0005682954086441362,
"loss": 3.4875,
"step": 7950
},
{
"epoch": 2.152885443583118,
"grad_norm": 0.4012206196784973,
"learning_rate": 0.0005680934428436784,
"loss": 3.5152,
"step": 8000
},
{
"epoch": 2.152885443583118,
"eval_accuracy": 0.3640480166951104,
"eval_loss": 3.5348846912384033,
"eval_runtime": 89.1099,
"eval_samples_per_second": 202.132,
"eval_steps_per_second": 12.636,
"step": 8000
},
{
"epoch": 2.166343669250646,
"grad_norm": 0.4097427427768707,
"learning_rate": 0.0005678914770432206,
"loss": 3.5051,
"step": 8050
},
{
"epoch": 2.179801894918174,
"grad_norm": 0.37478891015052795,
"learning_rate": 0.0005676895112427629,
"loss": 3.5036,
"step": 8100
},
{
"epoch": 2.193260120585702,
"grad_norm": 0.385329931974411,
"learning_rate": 0.000567487545442305,
"loss": 3.4956,
"step": 8150
},
{
"epoch": 2.20671834625323,
"grad_norm": 0.40505632758140564,
"learning_rate": 0.0005672855796418473,
"loss": 3.5012,
"step": 8200
},
{
"epoch": 2.220176571920758,
"grad_norm": 0.4362744390964508,
"learning_rate": 0.0005670836138413894,
"loss": 3.5054,
"step": 8250
},
{
"epoch": 2.233634797588286,
"grad_norm": 0.337893545627594,
"learning_rate": 0.0005668816480409317,
"loss": 3.4965,
"step": 8300
},
{
"epoch": 2.2470930232558137,
"grad_norm": 0.3597669005393982,
"learning_rate": 0.0005666796822404739,
"loss": 3.5006,
"step": 8350
},
{
"epoch": 2.260551248923342,
"grad_norm": 0.3700796961784363,
"learning_rate": 0.0005664777164400162,
"loss": 3.4999,
"step": 8400
},
{
"epoch": 2.27400947459087,
"grad_norm": 0.37950870394706726,
"learning_rate": 0.0005662757506395583,
"loss": 3.4976,
"step": 8450
},
{
"epoch": 2.287467700258398,
"grad_norm": 0.37483659386634827,
"learning_rate": 0.0005660737848391006,
"loss": 3.5031,
"step": 8500
},
{
"epoch": 2.300925925925926,
"grad_norm": 0.3778204023838043,
"learning_rate": 0.0005658718190386428,
"loss": 3.5025,
"step": 8550
},
{
"epoch": 2.314384151593454,
"grad_norm": 0.39371633529663086,
"learning_rate": 0.000565669853238185,
"loss": 3.5017,
"step": 8600
},
{
"epoch": 2.327842377260982,
"grad_norm": 0.38425666093826294,
"learning_rate": 0.0005654678874377272,
"loss": 3.5019,
"step": 8650
},
{
"epoch": 2.3413006029285097,
"grad_norm": 0.34810662269592285,
"learning_rate": 0.0005652659216372693,
"loss": 3.509,
"step": 8700
},
{
"epoch": 2.354758828596038,
"grad_norm": 0.3727887272834778,
"learning_rate": 0.0005650639558368116,
"loss": 3.4938,
"step": 8750
},
{
"epoch": 2.3682170542635657,
"grad_norm": 0.34845060110092163,
"learning_rate": 0.0005648619900363538,
"loss": 3.5106,
"step": 8800
},
{
"epoch": 2.381675279931094,
"grad_norm": 0.3758419454097748,
"learning_rate": 0.000564660024235896,
"loss": 3.4994,
"step": 8850
},
{
"epoch": 2.395133505598622,
"grad_norm": 0.34191444516181946,
"learning_rate": 0.0005644580584354382,
"loss": 3.4965,
"step": 8900
},
{
"epoch": 2.40859173126615,
"grad_norm": 0.36028486490249634,
"learning_rate": 0.0005642560926349804,
"loss": 3.4893,
"step": 8950
},
{
"epoch": 2.422049956933678,
"grad_norm": 0.37876632809638977,
"learning_rate": 0.0005640541268345226,
"loss": 3.4969,
"step": 9000
},
{
"epoch": 2.422049956933678,
"eval_accuracy": 0.36681166676010296,
"eval_loss": 3.508786916732788,
"eval_runtime": 89.0481,
"eval_samples_per_second": 202.273,
"eval_steps_per_second": 12.645,
"step": 9000
},
{
"epoch": 2.4355081826012057,
"grad_norm": 0.35556432604789734,
"learning_rate": 0.0005638521610340649,
"loss": 3.5017,
"step": 9050
},
{
"epoch": 2.448966408268734,
"grad_norm": 0.35798487067222595,
"learning_rate": 0.000563650195233607,
"loss": 3.4861,
"step": 9100
},
{
"epoch": 2.4624246339362617,
"grad_norm": 0.3988047242164612,
"learning_rate": 0.0005634482294331493,
"loss": 3.4987,
"step": 9150
},
{
"epoch": 2.47588285960379,
"grad_norm": 0.35068902373313904,
"learning_rate": 0.0005632462636326914,
"loss": 3.4886,
"step": 9200
},
{
"epoch": 2.4893410852713176,
"grad_norm": 0.3887159526348114,
"learning_rate": 0.0005630442978322337,
"loss": 3.4944,
"step": 9250
},
{
"epoch": 2.502799310938846,
"grad_norm": 0.3577377200126648,
"learning_rate": 0.0005628423320317759,
"loss": 3.4902,
"step": 9300
},
{
"epoch": 2.516257536606374,
"grad_norm": 0.3493818938732147,
"learning_rate": 0.0005626403662313182,
"loss": 3.474,
"step": 9350
},
{
"epoch": 2.5297157622739017,
"grad_norm": 0.35755589604377747,
"learning_rate": 0.0005624384004308603,
"loss": 3.4943,
"step": 9400
},
{
"epoch": 2.54317398794143,
"grad_norm": 0.34538453817367554,
"learning_rate": 0.0005622364346304026,
"loss": 3.4928,
"step": 9450
},
{
"epoch": 2.5566322136089576,
"grad_norm": 0.37912717461586,
"learning_rate": 0.0005620344688299448,
"loss": 3.4964,
"step": 9500
},
{
"epoch": 2.570090439276486,
"grad_norm": 0.4231334328651428,
"learning_rate": 0.0005618325030294869,
"loss": 3.5014,
"step": 9550
},
{
"epoch": 2.583548664944014,
"grad_norm": 0.35249027609825134,
"learning_rate": 0.0005616305372290292,
"loss": 3.4989,
"step": 9600
},
{
"epoch": 2.5970068906115418,
"grad_norm": 0.3673330247402191,
"learning_rate": 0.0005614285714285713,
"loss": 3.4933,
"step": 9650
},
{
"epoch": 2.6104651162790695,
"grad_norm": 0.36924853920936584,
"learning_rate": 0.0005612266056281136,
"loss": 3.4852,
"step": 9700
},
{
"epoch": 2.6239233419465977,
"grad_norm": 0.37280353903770447,
"learning_rate": 0.0005610246398276558,
"loss": 3.4825,
"step": 9750
},
{
"epoch": 2.637381567614126,
"grad_norm": 0.3669414818286896,
"learning_rate": 0.000560822674027198,
"loss": 3.4856,
"step": 9800
},
{
"epoch": 2.6508397932816536,
"grad_norm": 0.33487293124198914,
"learning_rate": 0.0005606207082267402,
"loss": 3.4863,
"step": 9850
},
{
"epoch": 2.664298018949182,
"grad_norm": 0.42584335803985596,
"learning_rate": 0.0005604187424262824,
"loss": 3.485,
"step": 9900
},
{
"epoch": 2.6777562446167096,
"grad_norm": 0.3721439242362976,
"learning_rate": 0.0005602167766258246,
"loss": 3.473,
"step": 9950
},
{
"epoch": 2.6912144702842378,
"grad_norm": 0.3362332880496979,
"learning_rate": 0.0005600148108253669,
"loss": 3.4725,
"step": 10000
},
{
"epoch": 2.6912144702842378,
"eval_accuracy": 0.3691482260358717,
"eval_loss": 3.4851438999176025,
"eval_runtime": 88.9071,
"eval_samples_per_second": 202.593,
"eval_steps_per_second": 12.665,
"step": 10000
},
{
"epoch": 2.704672695951766,
"grad_norm": 0.4206249713897705,
"learning_rate": 0.000559812845024909,
"loss": 3.4844,
"step": 10050
},
{
"epoch": 2.7181309216192937,
"grad_norm": 0.3479953706264496,
"learning_rate": 0.0005596108792244513,
"loss": 3.4837,
"step": 10100
},
{
"epoch": 2.7315891472868215,
"grad_norm": 0.379986047744751,
"learning_rate": 0.0005594089134239935,
"loss": 3.4661,
"step": 10150
},
{
"epoch": 2.7450473729543496,
"grad_norm": 0.35398250818252563,
"learning_rate": 0.0005592069476235358,
"loss": 3.4803,
"step": 10200
},
{
"epoch": 2.758505598621878,
"grad_norm": 0.3484811782836914,
"learning_rate": 0.0005590049818230779,
"loss": 3.4786,
"step": 10250
},
{
"epoch": 2.7719638242894056,
"grad_norm": 0.3662926256656647,
"learning_rate": 0.0005588030160226202,
"loss": 3.4711,
"step": 10300
},
{
"epoch": 2.7854220499569338,
"grad_norm": 0.3605268597602844,
"learning_rate": 0.0005586010502221623,
"loss": 3.4897,
"step": 10350
},
{
"epoch": 2.7988802756244615,
"grad_norm": 0.34791073203086853,
"learning_rate": 0.0005583990844217045,
"loss": 3.4742,
"step": 10400
},
{
"epoch": 2.8123385012919897,
"grad_norm": 0.34151312708854675,
"learning_rate": 0.0005581971186212468,
"loss": 3.4723,
"step": 10450
},
{
"epoch": 2.825796726959518,
"grad_norm": 0.3514784574508667,
"learning_rate": 0.0005579951528207889,
"loss": 3.4734,
"step": 10500
},
{
"epoch": 2.8392549526270456,
"grad_norm": 0.35521063208580017,
"learning_rate": 0.0005577931870203312,
"loss": 3.4809,
"step": 10550
},
{
"epoch": 2.8527131782945734,
"grad_norm": 0.35965123772621155,
"learning_rate": 0.0005575912212198733,
"loss": 3.4683,
"step": 10600
},
{
"epoch": 2.8661714039621016,
"grad_norm": 0.37720105051994324,
"learning_rate": 0.0005573892554194156,
"loss": 3.4777,
"step": 10650
},
{
"epoch": 2.8796296296296298,
"grad_norm": 0.35280463099479675,
"learning_rate": 0.0005571872896189578,
"loss": 3.4709,
"step": 10700
},
{
"epoch": 2.8930878552971575,
"grad_norm": 0.34837934374809265,
"learning_rate": 0.0005569853238185,
"loss": 3.4713,
"step": 10750
},
{
"epoch": 2.9065460809646857,
"grad_norm": 0.3864405155181885,
"learning_rate": 0.0005567833580180422,
"loss": 3.4581,
"step": 10800
},
{
"epoch": 2.9200043066322134,
"grad_norm": 0.3737727701663971,
"learning_rate": 0.0005565813922175844,
"loss": 3.4633,
"step": 10850
},
{
"epoch": 2.9334625322997416,
"grad_norm": 0.3452976942062378,
"learning_rate": 0.0005563794264171266,
"loss": 3.4727,
"step": 10900
},
{
"epoch": 2.94692075796727,
"grad_norm": 0.3639717400074005,
"learning_rate": 0.0005561774606166689,
"loss": 3.464,
"step": 10950
},
{
"epoch": 2.9603789836347976,
"grad_norm": 0.42750686407089233,
"learning_rate": 0.000555975494816211,
"loss": 3.4718,
"step": 11000
},
{
"epoch": 2.9603789836347976,
"eval_accuracy": 0.3713497372701739,
"eval_loss": 3.46138334274292,
"eval_runtime": 89.357,
"eval_samples_per_second": 201.574,
"eval_steps_per_second": 12.601,
"step": 11000
},
{
"epoch": 2.9738372093023253,
"grad_norm": 0.34111878275871277,
"learning_rate": 0.0005557735290157534,
"loss": 3.4719,
"step": 11050
},
{
"epoch": 2.9872954349698535,
"grad_norm": 0.3508542478084564,
"learning_rate": 0.0005555715632152955,
"loss": 3.4659,
"step": 11100
},
{
"epoch": 3.000538329026701,
"grad_norm": 0.4099224805831909,
"learning_rate": 0.0005553695974148378,
"loss": 3.4607,
"step": 11150
},
{
"epoch": 3.013996554694229,
"grad_norm": 0.3317889869213104,
"learning_rate": 0.0005551676316143799,
"loss": 3.3802,
"step": 11200
},
{
"epoch": 3.027454780361757,
"grad_norm": 0.349567174911499,
"learning_rate": 0.0005549656658139222,
"loss": 3.3764,
"step": 11250
},
{
"epoch": 3.0409130060292853,
"grad_norm": 0.38516244292259216,
"learning_rate": 0.0005547637000134644,
"loss": 3.3685,
"step": 11300
},
{
"epoch": 3.054371231696813,
"grad_norm": 0.33860960602760315,
"learning_rate": 0.0005545617342130065,
"loss": 3.3929,
"step": 11350
},
{
"epoch": 3.067829457364341,
"grad_norm": 0.37568598985671997,
"learning_rate": 0.0005543597684125488,
"loss": 3.3781,
"step": 11400
},
{
"epoch": 3.081287683031869,
"grad_norm": 0.36252379417419434,
"learning_rate": 0.0005541578026120909,
"loss": 3.3823,
"step": 11450
},
{
"epoch": 3.094745908699397,
"grad_norm": 0.3808792233467102,
"learning_rate": 0.0005539558368116332,
"loss": 3.3856,
"step": 11500
},
{
"epoch": 3.108204134366925,
"grad_norm": 0.3400123715400696,
"learning_rate": 0.0005537538710111754,
"loss": 3.3834,
"step": 11550
},
{
"epoch": 3.121662360034453,
"grad_norm": 0.365142822265625,
"learning_rate": 0.0005535519052107176,
"loss": 3.392,
"step": 11600
},
{
"epoch": 3.135120585701981,
"grad_norm": 0.3488098382949829,
"learning_rate": 0.0005533499394102598,
"loss": 3.3778,
"step": 11650
},
{
"epoch": 3.148578811369509,
"grad_norm": 0.31865015625953674,
"learning_rate": 0.000553147973609802,
"loss": 3.391,
"step": 11700
},
{
"epoch": 3.162037037037037,
"grad_norm": 0.36687901616096497,
"learning_rate": 0.0005529460078093442,
"loss": 3.3963,
"step": 11750
},
{
"epoch": 3.175495262704565,
"grad_norm": 0.3641960024833679,
"learning_rate": 0.0005527440420088865,
"loss": 3.3926,
"step": 11800
},
{
"epoch": 3.188953488372093,
"grad_norm": 0.3725584149360657,
"learning_rate": 0.0005525420762084286,
"loss": 3.3953,
"step": 11850
},
{
"epoch": 3.202411714039621,
"grad_norm": 0.40352463722229004,
"learning_rate": 0.0005523401104079709,
"loss": 3.396,
"step": 11900
},
{
"epoch": 3.215869939707149,
"grad_norm": 0.37560948729515076,
"learning_rate": 0.0005521381446075131,
"loss": 3.3956,
"step": 11950
},
{
"epoch": 3.229328165374677,
"grad_norm": 0.3621182143688202,
"learning_rate": 0.0005519361788070554,
"loss": 3.3896,
"step": 12000
},
{
"epoch": 3.229328165374677,
"eval_accuracy": 0.3730891734277605,
"eval_loss": 3.452282190322876,
"eval_runtime": 89.6688,
"eval_samples_per_second": 200.872,
"eval_steps_per_second": 12.557,
"step": 12000
},
{
"epoch": 3.242786391042205,
"grad_norm": 0.35463830828666687,
"learning_rate": 0.0005517342130065975,
"loss": 3.3955,
"step": 12050
},
{
"epoch": 3.2562446167097328,
"grad_norm": 0.3728417158126831,
"learning_rate": 0.0005515322472061398,
"loss": 3.4032,
"step": 12100
},
{
"epoch": 3.269702842377261,
"grad_norm": 0.3544614613056183,
"learning_rate": 0.0005513302814056819,
"loss": 3.3998,
"step": 12150
},
{
"epoch": 3.283161068044789,
"grad_norm": 0.34991875290870667,
"learning_rate": 0.0005511283156052242,
"loss": 3.3919,
"step": 12200
},
{
"epoch": 3.296619293712317,
"grad_norm": 0.3334318995475769,
"learning_rate": 0.0005509263498047664,
"loss": 3.3917,
"step": 12250
},
{
"epoch": 3.310077519379845,
"grad_norm": 0.3585456609725952,
"learning_rate": 0.0005507243840043085,
"loss": 3.4012,
"step": 12300
},
{
"epoch": 3.323535745047373,
"grad_norm": 0.36692893505096436,
"learning_rate": 0.0005505224182038508,
"loss": 3.3976,
"step": 12350
},
{
"epoch": 3.336993970714901,
"grad_norm": 0.3426796793937683,
"learning_rate": 0.0005503204524033929,
"loss": 3.3981,
"step": 12400
},
{
"epoch": 3.3504521963824287,
"grad_norm": 0.3477403521537781,
"learning_rate": 0.0005501184866029352,
"loss": 3.3912,
"step": 12450
},
{
"epoch": 3.363910422049957,
"grad_norm": 0.35531580448150635,
"learning_rate": 0.0005499165208024774,
"loss": 3.3949,
"step": 12500
},
{
"epoch": 3.3773686477174847,
"grad_norm": 0.334770143032074,
"learning_rate": 0.0005497145550020196,
"loss": 3.3899,
"step": 12550
},
{
"epoch": 3.390826873385013,
"grad_norm": 0.35484474897384644,
"learning_rate": 0.0005495125892015618,
"loss": 3.3913,
"step": 12600
},
{
"epoch": 3.404285099052541,
"grad_norm": 0.3469596207141876,
"learning_rate": 0.000549310623401104,
"loss": 3.3932,
"step": 12650
},
{
"epoch": 3.417743324720069,
"grad_norm": 0.37239179015159607,
"learning_rate": 0.0005491086576006462,
"loss": 3.4103,
"step": 12700
},
{
"epoch": 3.431201550387597,
"grad_norm": 0.3561572730541229,
"learning_rate": 0.0005489066918001885,
"loss": 3.3926,
"step": 12750
},
{
"epoch": 3.4446597760551247,
"grad_norm": 0.3374239206314087,
"learning_rate": 0.0005487047259997306,
"loss": 3.4029,
"step": 12800
},
{
"epoch": 3.458118001722653,
"grad_norm": 0.3496391773223877,
"learning_rate": 0.0005485027601992729,
"loss": 3.3964,
"step": 12850
},
{
"epoch": 3.471576227390181,
"grad_norm": 0.3446263372898102,
"learning_rate": 0.0005483007943988151,
"loss": 3.3918,
"step": 12900
},
{
"epoch": 3.485034453057709,
"grad_norm": 0.32869866490364075,
"learning_rate": 0.0005480988285983574,
"loss": 3.4067,
"step": 12950
},
{
"epoch": 3.4984926787252366,
"grad_norm": 0.3769306242465973,
"learning_rate": 0.0005478968627978995,
"loss": 3.3969,
"step": 13000
},
{
"epoch": 3.4984926787252366,
"eval_accuracy": 0.37452874426398924,
"eval_loss": 3.4374747276306152,
"eval_runtime": 88.8675,
"eval_samples_per_second": 202.684,
"eval_steps_per_second": 12.671,
"step": 13000
},
{
"epoch": 3.511950904392765,
"grad_norm": 0.3424180746078491,
"learning_rate": 0.0005476948969974418,
"loss": 3.4045,
"step": 13050
},
{
"epoch": 3.525409130060293,
"grad_norm": 0.3807520270347595,
"learning_rate": 0.0005474929311969839,
"loss": 3.3979,
"step": 13100
},
{
"epoch": 3.5388673557278207,
"grad_norm": 0.4000932574272156,
"learning_rate": 0.0005472909653965261,
"loss": 3.4005,
"step": 13150
},
{
"epoch": 3.552325581395349,
"grad_norm": 0.342479944229126,
"learning_rate": 0.0005470889995960684,
"loss": 3.3852,
"step": 13200
},
{
"epoch": 3.5657838070628767,
"grad_norm": 0.3163456320762634,
"learning_rate": 0.0005468870337956105,
"loss": 3.3971,
"step": 13250
},
{
"epoch": 3.579242032730405,
"grad_norm": 0.36020761728286743,
"learning_rate": 0.0005466850679951528,
"loss": 3.3911,
"step": 13300
},
{
"epoch": 3.592700258397933,
"grad_norm": 0.34629690647125244,
"learning_rate": 0.0005464831021946949,
"loss": 3.3972,
"step": 13350
},
{
"epoch": 3.606158484065461,
"grad_norm": 0.3748731315135956,
"learning_rate": 0.0005462811363942372,
"loss": 3.3925,
"step": 13400
},
{
"epoch": 3.6196167097329885,
"grad_norm": 0.34598422050476074,
"learning_rate": 0.0005460791705937794,
"loss": 3.3977,
"step": 13450
},
{
"epoch": 3.6330749354005167,
"grad_norm": 0.33285003900527954,
"learning_rate": 0.0005458772047933216,
"loss": 3.3862,
"step": 13500
},
{
"epoch": 3.646533161068045,
"grad_norm": 0.36419209837913513,
"learning_rate": 0.0005456752389928638,
"loss": 3.3893,
"step": 13550
},
{
"epoch": 3.6599913867355727,
"grad_norm": 0.3598211407661438,
"learning_rate": 0.000545473273192406,
"loss": 3.4047,
"step": 13600
},
{
"epoch": 3.673449612403101,
"grad_norm": 0.3460333049297333,
"learning_rate": 0.0005452713073919482,
"loss": 3.395,
"step": 13650
},
{
"epoch": 3.6869078380706286,
"grad_norm": 0.3223993182182312,
"learning_rate": 0.0005450693415914905,
"loss": 3.3951,
"step": 13700
},
{
"epoch": 3.700366063738157,
"grad_norm": 0.367910236120224,
"learning_rate": 0.0005448673757910327,
"loss": 3.3963,
"step": 13750
},
{
"epoch": 3.713824289405685,
"grad_norm": 0.31978780031204224,
"learning_rate": 0.0005446654099905749,
"loss": 3.3843,
"step": 13800
},
{
"epoch": 3.7272825150732127,
"grad_norm": 0.37149783968925476,
"learning_rate": 0.0005444634441901171,
"loss": 3.3958,
"step": 13850
},
{
"epoch": 3.7407407407407405,
"grad_norm": 0.35949620604515076,
"learning_rate": 0.0005442614783896594,
"loss": 3.3863,
"step": 13900
},
{
"epoch": 3.7541989664082687,
"grad_norm": 0.355377197265625,
"learning_rate": 0.0005440595125892015,
"loss": 3.3976,
"step": 13950
},
{
"epoch": 3.767657192075797,
"grad_norm": 0.32358214259147644,
"learning_rate": 0.0005438575467887438,
"loss": 3.3913,
"step": 14000
},
{
"epoch": 3.767657192075797,
"eval_accuracy": 0.37618463098964683,
"eval_loss": 3.420905590057373,
"eval_runtime": 89.8357,
"eval_samples_per_second": 200.499,
"eval_steps_per_second": 12.534,
"step": 14000
},
{
"epoch": 3.7811154177433246,
"grad_norm": 0.34888753294944763,
"learning_rate": 0.0005436555809882859,
"loss": 3.3917,
"step": 14050
},
{
"epoch": 3.794573643410853,
"grad_norm": 0.3531152307987213,
"learning_rate": 0.0005434536151878281,
"loss": 3.3893,
"step": 14100
},
{
"epoch": 3.8080318690783805,
"grad_norm": 0.3416253924369812,
"learning_rate": 0.0005432516493873704,
"loss": 3.3961,
"step": 14150
},
{
"epoch": 3.8214900947459087,
"grad_norm": 0.3324134945869446,
"learning_rate": 0.0005430496835869125,
"loss": 3.387,
"step": 14200
},
{
"epoch": 3.834948320413437,
"grad_norm": 0.3525480628013611,
"learning_rate": 0.0005428477177864548,
"loss": 3.3958,
"step": 14250
},
{
"epoch": 3.8484065460809647,
"grad_norm": 0.3697124123573303,
"learning_rate": 0.0005426457519859969,
"loss": 3.4013,
"step": 14300
},
{
"epoch": 3.8618647717484924,
"grad_norm": 0.3347870409488678,
"learning_rate": 0.0005424437861855392,
"loss": 3.3922,
"step": 14350
},
{
"epoch": 3.8753229974160206,
"grad_norm": 0.34625548124313354,
"learning_rate": 0.0005422418203850814,
"loss": 3.3933,
"step": 14400
},
{
"epoch": 3.888781223083549,
"grad_norm": 0.3729144334793091,
"learning_rate": 0.0005420398545846236,
"loss": 3.3915,
"step": 14450
},
{
"epoch": 3.9022394487510765,
"grad_norm": 0.3475784361362457,
"learning_rate": 0.0005418378887841658,
"loss": 3.3944,
"step": 14500
},
{
"epoch": 3.9156976744186047,
"grad_norm": 0.3474798798561096,
"learning_rate": 0.000541635922983708,
"loss": 3.4017,
"step": 14550
},
{
"epoch": 3.9291559000861325,
"grad_norm": 0.3410467505455017,
"learning_rate": 0.0005414339571832503,
"loss": 3.386,
"step": 14600
},
{
"epoch": 3.9426141257536607,
"grad_norm": 0.34732016921043396,
"learning_rate": 0.0005412319913827925,
"loss": 3.394,
"step": 14650
},
{
"epoch": 3.956072351421189,
"grad_norm": 0.32139548659324646,
"learning_rate": 0.0005410300255823347,
"loss": 3.385,
"step": 14700
},
{
"epoch": 3.9695305770887166,
"grad_norm": 0.328720360994339,
"learning_rate": 0.0005408280597818769,
"loss": 3.3803,
"step": 14750
},
{
"epoch": 3.9829888027562443,
"grad_norm": 0.35579606890678406,
"learning_rate": 0.0005406260939814191,
"loss": 3.4056,
"step": 14800
},
{
"epoch": 3.9964470284237725,
"grad_norm": 0.3351224660873413,
"learning_rate": 0.0005404241281809614,
"loss": 3.3863,
"step": 14850
},
{
"epoch": 4.00968992248062,
"grad_norm": 0.3696632981300354,
"learning_rate": 0.0005402221623805035,
"loss": 3.3029,
"step": 14900
},
{
"epoch": 4.023148148148148,
"grad_norm": 0.33017241954803467,
"learning_rate": 0.0005400201965800457,
"loss": 3.2878,
"step": 14950
},
{
"epoch": 4.036606373815676,
"grad_norm": 0.34758812189102173,
"learning_rate": 0.000539818230779588,
"loss": 3.2989,
"step": 15000
},
{
"epoch": 4.036606373815676,
"eval_accuracy": 0.3775983438742513,
"eval_loss": 3.4153592586517334,
"eval_runtime": 89.1199,
"eval_samples_per_second": 202.11,
"eval_steps_per_second": 12.635,
"step": 15000
},
{
"epoch": 4.050064599483204,
"grad_norm": 0.33058854937553406,
"learning_rate": 0.0005396162649791301,
"loss": 3.3013,
"step": 15050
},
{
"epoch": 4.0635228251507325,
"grad_norm": 0.35816290974617004,
"learning_rate": 0.0005394142991786724,
"loss": 3.2947,
"step": 15100
},
{
"epoch": 4.07698105081826,
"grad_norm": 0.3572397530078888,
"learning_rate": 0.0005392123333782145,
"loss": 3.3099,
"step": 15150
},
{
"epoch": 4.090439276485788,
"grad_norm": 0.36381787061691284,
"learning_rate": 0.0005390103675777568,
"loss": 3.3076,
"step": 15200
},
{
"epoch": 4.103897502153316,
"grad_norm": 0.39142516255378723,
"learning_rate": 0.000538808401777299,
"loss": 3.3173,
"step": 15250
},
{
"epoch": 4.117355727820844,
"grad_norm": 0.3246189057826996,
"learning_rate": 0.0005386064359768412,
"loss": 3.3056,
"step": 15300
},
{
"epoch": 4.1308139534883725,
"grad_norm": 0.3524934649467468,
"learning_rate": 0.0005384044701763834,
"loss": 3.3093,
"step": 15350
},
{
"epoch": 4.1442721791559,
"grad_norm": 0.3528584837913513,
"learning_rate": 0.0005382025043759256,
"loss": 3.2963,
"step": 15400
},
{
"epoch": 4.157730404823428,
"grad_norm": 0.3665054142475128,
"learning_rate": 0.0005380005385754678,
"loss": 3.313,
"step": 15450
},
{
"epoch": 4.171188630490956,
"grad_norm": 0.36652475595474243,
"learning_rate": 0.0005377985727750101,
"loss": 3.3135,
"step": 15500
},
{
"epoch": 4.184646856158484,
"grad_norm": 0.3758777976036072,
"learning_rate": 0.0005375966069745523,
"loss": 3.3163,
"step": 15550
},
{
"epoch": 4.198105081826012,
"grad_norm": 0.33912044763565063,
"learning_rate": 0.0005373946411740945,
"loss": 3.3075,
"step": 15600
},
{
"epoch": 4.21156330749354,
"grad_norm": 0.332670122385025,
"learning_rate": 0.0005371926753736367,
"loss": 3.3225,
"step": 15650
},
{
"epoch": 4.225021533161068,
"grad_norm": 0.3336251974105835,
"learning_rate": 0.000536990709573179,
"loss": 3.3212,
"step": 15700
},
{
"epoch": 4.238479758828596,
"grad_norm": 0.35916876792907715,
"learning_rate": 0.0005367887437727211,
"loss": 3.3289,
"step": 15750
},
{
"epoch": 4.251937984496124,
"grad_norm": 0.32652392983436584,
"learning_rate": 0.0005365867779722634,
"loss": 3.329,
"step": 15800
},
{
"epoch": 4.265396210163652,
"grad_norm": 0.34005942940711975,
"learning_rate": 0.0005363848121718055,
"loss": 3.3379,
"step": 15850
},
{
"epoch": 4.27885443583118,
"grad_norm": 0.3562110662460327,
"learning_rate": 0.0005361828463713477,
"loss": 3.3135,
"step": 15900
},
{
"epoch": 4.292312661498708,
"grad_norm": 0.36522018909454346,
"learning_rate": 0.00053598088057089,
"loss": 3.3163,
"step": 15950
},
{
"epoch": 4.305770887166236,
"grad_norm": 0.35446590185165405,
"learning_rate": 0.0005357789147704321,
"loss": 3.3334,
"step": 16000
},
{
"epoch": 4.305770887166236,
"eval_accuracy": 0.37851575792263736,
"eval_loss": 3.4047274589538574,
"eval_runtime": 88.4624,
"eval_samples_per_second": 203.612,
"eval_steps_per_second": 12.729,
"step": 16000
},
{
"epoch": 4.319229112833764,
"grad_norm": 0.3302324712276459,
"learning_rate": 0.0005355769489699744,
"loss": 3.3239,
"step": 16050
},
{
"epoch": 4.332687338501292,
"grad_norm": 0.3508687913417816,
"learning_rate": 0.0005353749831695165,
"loss": 3.3308,
"step": 16100
},
{
"epoch": 4.34614556416882,
"grad_norm": 0.34046798944473267,
"learning_rate": 0.0005351730173690588,
"loss": 3.3358,
"step": 16150
},
{
"epoch": 4.359603789836348,
"grad_norm": 0.3654434382915497,
"learning_rate": 0.000534971051568601,
"loss": 3.3117,
"step": 16200
},
{
"epoch": 4.373062015503876,
"grad_norm": 0.35325589776039124,
"learning_rate": 0.0005347690857681432,
"loss": 3.3336,
"step": 16250
},
{
"epoch": 4.386520241171404,
"grad_norm": 0.3389158546924591,
"learning_rate": 0.0005345671199676854,
"loss": 3.325,
"step": 16300
},
{
"epoch": 4.399978466838932,
"grad_norm": 0.3626989722251892,
"learning_rate": 0.0005343651541672276,
"loss": 3.3272,
"step": 16350
},
{
"epoch": 4.41343669250646,
"grad_norm": 0.34801793098449707,
"learning_rate": 0.0005341631883667699,
"loss": 3.3257,
"step": 16400
},
{
"epoch": 4.426894918173988,
"grad_norm": 0.321744441986084,
"learning_rate": 0.0005339612225663121,
"loss": 3.335,
"step": 16450
},
{
"epoch": 4.440353143841516,
"grad_norm": 0.34780287742614746,
"learning_rate": 0.0005337592567658543,
"loss": 3.3402,
"step": 16500
},
{
"epoch": 4.453811369509044,
"grad_norm": 0.36421075463294983,
"learning_rate": 0.0005335572909653965,
"loss": 3.3383,
"step": 16550
},
{
"epoch": 4.467269595176572,
"grad_norm": 0.3883045017719269,
"learning_rate": 0.0005333553251649387,
"loss": 3.3352,
"step": 16600
},
{
"epoch": 4.4807278208441,
"grad_norm": 0.3691977262496948,
"learning_rate": 0.000533153359364481,
"loss": 3.3244,
"step": 16650
},
{
"epoch": 4.4941860465116275,
"grad_norm": 0.35588181018829346,
"learning_rate": 0.0005329513935640231,
"loss": 3.3301,
"step": 16700
},
{
"epoch": 4.507644272179156,
"grad_norm": 0.33899396657943726,
"learning_rate": 0.0005327494277635654,
"loss": 3.3302,
"step": 16750
},
{
"epoch": 4.521102497846684,
"grad_norm": 0.351391464471817,
"learning_rate": 0.0005325474619631075,
"loss": 3.3355,
"step": 16800
},
{
"epoch": 4.534560723514212,
"grad_norm": 0.33637428283691406,
"learning_rate": 0.0005323454961626497,
"loss": 3.3384,
"step": 16850
},
{
"epoch": 4.54801894918174,
"grad_norm": 0.34806662797927856,
"learning_rate": 0.000532143530362192,
"loss": 3.3453,
"step": 16900
},
{
"epoch": 4.5614771748492675,
"grad_norm": 0.32941877841949463,
"learning_rate": 0.0005319415645617341,
"loss": 3.3331,
"step": 16950
},
{
"epoch": 4.574935400516796,
"grad_norm": 0.3837328851222992,
"learning_rate": 0.0005317395987612764,
"loss": 3.3276,
"step": 17000
},
{
"epoch": 4.574935400516796,
"eval_accuracy": 0.3797251060719251,
"eval_loss": 3.393202543258667,
"eval_runtime": 88.7513,
"eval_samples_per_second": 202.949,
"eval_steps_per_second": 12.687,
"step": 17000
},
{
"epoch": 4.588393626184324,
"grad_norm": 0.34744006395339966,
"learning_rate": 0.0005315376329608185,
"loss": 3.3315,
"step": 17050
},
{
"epoch": 4.601851851851852,
"grad_norm": 0.3779725432395935,
"learning_rate": 0.0005313356671603608,
"loss": 3.3307,
"step": 17100
},
{
"epoch": 4.61531007751938,
"grad_norm": 0.3435402512550354,
"learning_rate": 0.000531133701359903,
"loss": 3.3437,
"step": 17150
},
{
"epoch": 4.628768303186908,
"grad_norm": 0.35903772711753845,
"learning_rate": 0.0005309317355594452,
"loss": 3.3371,
"step": 17200
},
{
"epoch": 4.642226528854436,
"grad_norm": 0.3584959805011749,
"learning_rate": 0.0005307297697589874,
"loss": 3.3397,
"step": 17250
},
{
"epoch": 4.655684754521964,
"grad_norm": 0.3358038663864136,
"learning_rate": 0.0005305278039585297,
"loss": 3.3294,
"step": 17300
},
{
"epoch": 4.669142980189492,
"grad_norm": 0.3338400721549988,
"learning_rate": 0.0005303258381580719,
"loss": 3.3274,
"step": 17350
},
{
"epoch": 4.682601205857019,
"grad_norm": 0.3644915819168091,
"learning_rate": 0.0005301238723576141,
"loss": 3.3364,
"step": 17400
},
{
"epoch": 4.696059431524548,
"grad_norm": 0.3266715407371521,
"learning_rate": 0.0005299219065571563,
"loss": 3.3418,
"step": 17450
},
{
"epoch": 4.709517657192076,
"grad_norm": 0.3627817630767822,
"learning_rate": 0.0005297199407566985,
"loss": 3.3476,
"step": 17500
},
{
"epoch": 4.722975882859604,
"grad_norm": 0.3106552064418793,
"learning_rate": 0.0005295179749562407,
"loss": 3.3472,
"step": 17550
},
{
"epoch": 4.736434108527131,
"grad_norm": 0.36165159940719604,
"learning_rate": 0.000529316009155783,
"loss": 3.3336,
"step": 17600
},
{
"epoch": 4.7498923341946595,
"grad_norm": 0.3304913341999054,
"learning_rate": 0.0005291140433553251,
"loss": 3.3463,
"step": 17650
},
{
"epoch": 4.763350559862188,
"grad_norm": 0.3533960282802582,
"learning_rate": 0.0005289120775548673,
"loss": 3.3416,
"step": 17700
},
{
"epoch": 4.776808785529716,
"grad_norm": 0.3458789587020874,
"learning_rate": 0.0005287101117544095,
"loss": 3.3445,
"step": 17750
},
{
"epoch": 4.790267011197244,
"grad_norm": 0.32624074816703796,
"learning_rate": 0.0005285081459539517,
"loss": 3.3296,
"step": 17800
},
{
"epoch": 4.803725236864771,
"grad_norm": 0.3615491986274719,
"learning_rate": 0.000528306180153494,
"loss": 3.3306,
"step": 17850
},
{
"epoch": 4.8171834625323,
"grad_norm": 0.3486765921115875,
"learning_rate": 0.0005281042143530361,
"loss": 3.3317,
"step": 17900
},
{
"epoch": 4.830641688199828,
"grad_norm": 0.3309303820133209,
"learning_rate": 0.0005279022485525784,
"loss": 3.3415,
"step": 17950
},
{
"epoch": 4.844099913867356,
"grad_norm": 0.34980034828186035,
"learning_rate": 0.0005277002827521205,
"loss": 3.3278,
"step": 18000
},
{
"epoch": 4.844099913867356,
"eval_accuracy": 0.3803980646952912,
"eval_loss": 3.3822619915008545,
"eval_runtime": 84.7527,
"eval_samples_per_second": 212.524,
"eval_steps_per_second": 13.286,
"step": 18000
},
{
"epoch": 4.857558139534884,
"grad_norm": 0.31751900911331177,
"learning_rate": 0.0005274983169516628,
"loss": 3.3359,
"step": 18050
},
{
"epoch": 4.871016365202411,
"grad_norm": 0.4001691937446594,
"learning_rate": 0.000527296351151205,
"loss": 3.3373,
"step": 18100
},
{
"epoch": 4.88447459086994,
"grad_norm": 0.32267269492149353,
"learning_rate": 0.0005270943853507472,
"loss": 3.3312,
"step": 18150
},
{
"epoch": 4.897932816537468,
"grad_norm": 0.3582165837287903,
"learning_rate": 0.0005268924195502894,
"loss": 3.3349,
"step": 18200
},
{
"epoch": 4.911391042204996,
"grad_norm": 0.3568687438964844,
"learning_rate": 0.0005266904537498317,
"loss": 3.3309,
"step": 18250
},
{
"epoch": 4.924849267872523,
"grad_norm": 0.3639770448207855,
"learning_rate": 0.0005264884879493739,
"loss": 3.3262,
"step": 18300
},
{
"epoch": 4.9383074935400515,
"grad_norm": 0.321569561958313,
"learning_rate": 0.0005262865221489161,
"loss": 3.3273,
"step": 18350
},
{
"epoch": 4.95176571920758,
"grad_norm": 0.3282260000705719,
"learning_rate": 0.0005260845563484583,
"loss": 3.3389,
"step": 18400
},
{
"epoch": 4.965223944875108,
"grad_norm": 0.34112313389778137,
"learning_rate": 0.0005258825905480005,
"loss": 3.3316,
"step": 18450
},
{
"epoch": 4.978682170542635,
"grad_norm": 0.3521008789539337,
"learning_rate": 0.0005256806247475427,
"loss": 3.3318,
"step": 18500
},
{
"epoch": 4.992140396210163,
"grad_norm": 0.3335284888744354,
"learning_rate": 0.000525478658947085,
"loss": 3.3378,
"step": 18550
},
{
"epoch": 5.0053832902670115,
"grad_norm": 0.34815871715545654,
"learning_rate": 0.0005252766931466271,
"loss": 3.3033,
"step": 18600
},
{
"epoch": 5.018841515934539,
"grad_norm": 0.3716835677623749,
"learning_rate": 0.0005250747273461693,
"loss": 3.2465,
"step": 18650
},
{
"epoch": 5.032299741602067,
"grad_norm": 0.34348708391189575,
"learning_rate": 0.0005248727615457115,
"loss": 3.22,
"step": 18700
},
{
"epoch": 5.045757967269595,
"grad_norm": 0.34355980157852173,
"learning_rate": 0.0005246707957452537,
"loss": 3.2468,
"step": 18750
},
{
"epoch": 5.059216192937123,
"grad_norm": 0.3393841087818146,
"learning_rate": 0.000524468829944796,
"loss": 3.2384,
"step": 18800
},
{
"epoch": 5.0726744186046515,
"grad_norm": 0.37807780504226685,
"learning_rate": 0.0005242668641443381,
"loss": 3.2484,
"step": 18850
},
{
"epoch": 5.086132644272179,
"grad_norm": 0.3223433494567871,
"learning_rate": 0.0005240648983438804,
"loss": 3.2499,
"step": 18900
},
{
"epoch": 5.099590869939707,
"grad_norm": 0.3463059961795807,
"learning_rate": 0.0005238629325434225,
"loss": 3.2496,
"step": 18950
},
{
"epoch": 5.113049095607235,
"grad_norm": 0.3511742055416107,
"learning_rate": 0.0005236609667429648,
"loss": 3.2588,
"step": 19000
},
{
"epoch": 5.113049095607235,
"eval_accuracy": 0.38150713179689294,
"eval_loss": 3.3810367584228516,
"eval_runtime": 84.1776,
"eval_samples_per_second": 213.976,
"eval_steps_per_second": 13.376,
"step": 19000
},
{
"epoch": 5.126507321274763,
"grad_norm": 0.34395796060562134,
"learning_rate": 0.000523459000942507,
"loss": 3.2508,
"step": 19050
},
{
"epoch": 5.139965546942291,
"grad_norm": 0.3511422872543335,
"learning_rate": 0.0005232570351420493,
"loss": 3.2636,
"step": 19100
},
{
"epoch": 5.153423772609819,
"grad_norm": 0.3584088087081909,
"learning_rate": 0.0005230550693415915,
"loss": 3.2743,
"step": 19150
},
{
"epoch": 5.166881998277347,
"grad_norm": 0.36160993576049805,
"learning_rate": 0.0005228531035411337,
"loss": 3.2551,
"step": 19200
},
{
"epoch": 5.180340223944875,
"grad_norm": 0.33585062623023987,
"learning_rate": 0.0005226511377406759,
"loss": 3.2666,
"step": 19250
},
{
"epoch": 5.1937984496124034,
"grad_norm": 0.37425586581230164,
"learning_rate": 0.0005224491719402181,
"loss": 3.2686,
"step": 19300
},
{
"epoch": 5.207256675279931,
"grad_norm": 0.3439202606678009,
"learning_rate": 0.0005222472061397603,
"loss": 3.2681,
"step": 19350
},
{
"epoch": 5.220714900947459,
"grad_norm": 0.3334115743637085,
"learning_rate": 0.0005220452403393026,
"loss": 3.2665,
"step": 19400
},
{
"epoch": 5.234173126614987,
"grad_norm": 0.3346470892429352,
"learning_rate": 0.0005218432745388447,
"loss": 3.2712,
"step": 19450
},
{
"epoch": 5.247631352282515,
"grad_norm": 0.3484283983707428,
"learning_rate": 0.000521641308738387,
"loss": 3.268,
"step": 19500
},
{
"epoch": 5.2610895779500435,
"grad_norm": 0.3476269841194153,
"learning_rate": 0.0005214393429379291,
"loss": 3.2694,
"step": 19550
},
{
"epoch": 5.274547803617571,
"grad_norm": 0.33217039704322815,
"learning_rate": 0.0005212373771374713,
"loss": 3.279,
"step": 19600
},
{
"epoch": 5.288006029285099,
"grad_norm": 0.3590564429759979,
"learning_rate": 0.0005210354113370136,
"loss": 3.2609,
"step": 19650
},
{
"epoch": 5.301464254952627,
"grad_norm": 0.3503280580043793,
"learning_rate": 0.0005208334455365557,
"loss": 3.2769,
"step": 19700
},
{
"epoch": 5.314922480620155,
"grad_norm": 0.3696460723876953,
"learning_rate": 0.000520631479736098,
"loss": 3.2856,
"step": 19750
},
{
"epoch": 5.328380706287683,
"grad_norm": 0.35827770829200745,
"learning_rate": 0.0005204295139356401,
"loss": 3.2806,
"step": 19800
},
{
"epoch": 5.341838931955211,
"grad_norm": 0.38634854555130005,
"learning_rate": 0.0005202275481351824,
"loss": 3.2765,
"step": 19850
},
{
"epoch": 5.355297157622739,
"grad_norm": 0.33159148693084717,
"learning_rate": 0.0005200255823347246,
"loss": 3.2766,
"step": 19900
},
{
"epoch": 5.368755383290267,
"grad_norm": 0.3398463726043701,
"learning_rate": 0.0005198236165342669,
"loss": 3.2782,
"step": 19950
},
{
"epoch": 5.3822136089577945,
"grad_norm": 0.32563892006874084,
"learning_rate": 0.000519621650733809,
"loss": 3.2688,
"step": 20000
},
{
"epoch": 5.3822136089577945,
"eval_accuracy": 0.38212424593649896,
"eval_loss": 3.374769687652588,
"eval_runtime": 84.557,
"eval_samples_per_second": 213.016,
"eval_steps_per_second": 13.316,
"step": 20000
},
{
"epoch": 5.395671834625323,
"grad_norm": 0.3491804599761963,
"learning_rate": 0.0005194196849333513,
"loss": 3.2753,
"step": 20050
},
{
"epoch": 5.409130060292851,
"grad_norm": 0.32255983352661133,
"learning_rate": 0.0005192177191328935,
"loss": 3.3006,
"step": 20100
},
{
"epoch": 5.422588285960379,
"grad_norm": 0.3570587933063507,
"learning_rate": 0.0005190157533324357,
"loss": 3.2828,
"step": 20150
},
{
"epoch": 5.436046511627907,
"grad_norm": 0.3464536964893341,
"learning_rate": 0.0005188137875319779,
"loss": 3.2947,
"step": 20200
},
{
"epoch": 5.449504737295435,
"grad_norm": 0.33288997411727905,
"learning_rate": 0.00051861182173152,
"loss": 3.2925,
"step": 20250
},
{
"epoch": 5.462962962962963,
"grad_norm": 0.3999335765838623,
"learning_rate": 0.0005184098559310623,
"loss": 3.2864,
"step": 20300
},
{
"epoch": 5.476421188630491,
"grad_norm": 0.3677201569080353,
"learning_rate": 0.0005182078901306046,
"loss": 3.2725,
"step": 20350
},
{
"epoch": 5.489879414298019,
"grad_norm": 0.3399883806705475,
"learning_rate": 0.0005180059243301467,
"loss": 3.2955,
"step": 20400
},
{
"epoch": 5.503337639965547,
"grad_norm": 0.3565647304058075,
"learning_rate": 0.000517803958529689,
"loss": 3.2879,
"step": 20450
},
{
"epoch": 5.516795865633075,
"grad_norm": 0.3631874918937683,
"learning_rate": 0.0005176019927292311,
"loss": 3.2881,
"step": 20500
},
{
"epoch": 5.530254091300603,
"grad_norm": 0.35753047466278076,
"learning_rate": 0.0005174000269287733,
"loss": 3.2765,
"step": 20550
},
{
"epoch": 5.543712316968131,
"grad_norm": 0.3494873344898224,
"learning_rate": 0.0005171980611283156,
"loss": 3.2798,
"step": 20600
},
{
"epoch": 5.557170542635659,
"grad_norm": 0.3338465392589569,
"learning_rate": 0.0005169960953278577,
"loss": 3.276,
"step": 20650
},
{
"epoch": 5.5706287683031865,
"grad_norm": 0.3469972610473633,
"learning_rate": 0.0005167941295274,
"loss": 3.2867,
"step": 20700
},
{
"epoch": 5.584086993970715,
"grad_norm": 0.3427984118461609,
"learning_rate": 0.0005165921637269421,
"loss": 3.2872,
"step": 20750
},
{
"epoch": 5.597545219638243,
"grad_norm": 0.34492751955986023,
"learning_rate": 0.0005163901979264844,
"loss": 3.2895,
"step": 20800
},
{
"epoch": 5.611003445305771,
"grad_norm": 0.32378867268562317,
"learning_rate": 0.0005161882321260266,
"loss": 3.2828,
"step": 20850
},
{
"epoch": 5.624461670973298,
"grad_norm": 0.33641505241394043,
"learning_rate": 0.0005159862663255689,
"loss": 3.2687,
"step": 20900
},
{
"epoch": 5.637919896640827,
"grad_norm": 0.38284191489219666,
"learning_rate": 0.000515784300525111,
"loss": 3.2925,
"step": 20950
},
{
"epoch": 5.651378122308355,
"grad_norm": 0.3551783263683319,
"learning_rate": 0.0005155823347246533,
"loss": 3.2819,
"step": 21000
},
{
"epoch": 5.651378122308355,
"eval_accuracy": 0.3831818144285632,
"eval_loss": 3.362936496734619,
"eval_runtime": 84.2532,
"eval_samples_per_second": 213.784,
"eval_steps_per_second": 13.364,
"step": 21000
},
{
"epoch": 5.664836347975883,
"grad_norm": 0.3598018288612366,
"learning_rate": 0.0005153803689241955,
"loss": 3.2921,
"step": 21050
},
{
"epoch": 5.678294573643411,
"grad_norm": 0.3473359942436218,
"learning_rate": 0.0005151784031237377,
"loss": 3.2824,
"step": 21100
},
{
"epoch": 5.6917527993109385,
"grad_norm": 0.33983397483825684,
"learning_rate": 0.0005149764373232799,
"loss": 3.2783,
"step": 21150
},
{
"epoch": 5.705211024978467,
"grad_norm": 0.34451305866241455,
"learning_rate": 0.000514774471522822,
"loss": 3.2899,
"step": 21200
},
{
"epoch": 5.718669250645995,
"grad_norm": 0.34196949005126953,
"learning_rate": 0.0005145725057223643,
"loss": 3.2842,
"step": 21250
},
{
"epoch": 5.732127476313523,
"grad_norm": 0.3753472864627838,
"learning_rate": 0.0005143705399219066,
"loss": 3.2919,
"step": 21300
},
{
"epoch": 5.745585701981051,
"grad_norm": 0.3294236660003662,
"learning_rate": 0.0005141685741214487,
"loss": 3.288,
"step": 21350
},
{
"epoch": 5.7590439276485785,
"grad_norm": 0.32671308517456055,
"learning_rate": 0.0005139666083209909,
"loss": 3.2783,
"step": 21400
},
{
"epoch": 5.772502153316107,
"grad_norm": 0.31655827164649963,
"learning_rate": 0.0005137646425205331,
"loss": 3.2926,
"step": 21450
},
{
"epoch": 5.785960378983635,
"grad_norm": 0.34189605712890625,
"learning_rate": 0.0005135626767200753,
"loss": 3.2937,
"step": 21500
},
{
"epoch": 5.799418604651163,
"grad_norm": 0.3352980315685272,
"learning_rate": 0.0005133607109196176,
"loss": 3.286,
"step": 21550
},
{
"epoch": 5.81287683031869,
"grad_norm": 0.35656702518463135,
"learning_rate": 0.0005131587451191597,
"loss": 3.2827,
"step": 21600
},
{
"epoch": 5.826335055986219,
"grad_norm": 0.32037410140037537,
"learning_rate": 0.000512956779318702,
"loss": 3.2897,
"step": 21650
},
{
"epoch": 5.839793281653747,
"grad_norm": 0.33292341232299805,
"learning_rate": 0.0005127548135182441,
"loss": 3.2969,
"step": 21700
},
{
"epoch": 5.853251507321275,
"grad_norm": 0.3731493353843689,
"learning_rate": 0.0005125528477177865,
"loss": 3.2891,
"step": 21750
},
{
"epoch": 5.866709732988802,
"grad_norm": 0.3637171685695648,
"learning_rate": 0.0005123508819173286,
"loss": 3.2919,
"step": 21800
},
{
"epoch": 5.8801679586563305,
"grad_norm": 0.3596208691596985,
"learning_rate": 0.0005121489161168709,
"loss": 3.2926,
"step": 21850
},
{
"epoch": 5.893626184323859,
"grad_norm": 0.3490025997161865,
"learning_rate": 0.000511946950316413,
"loss": 3.2855,
"step": 21900
},
{
"epoch": 5.907084409991387,
"grad_norm": 0.33612897992134094,
"learning_rate": 0.0005117449845159553,
"loss": 3.2957,
"step": 21950
},
{
"epoch": 5.920542635658915,
"grad_norm": 0.331669420003891,
"learning_rate": 0.0005115430187154975,
"loss": 3.2922,
"step": 22000
},
{
"epoch": 5.920542635658915,
"eval_accuracy": 0.38385922757300744,
"eval_loss": 3.354418992996216,
"eval_runtime": 84.493,
"eval_samples_per_second": 213.178,
"eval_steps_per_second": 13.327,
"step": 22000
},
{
"epoch": 5.934000861326442,
"grad_norm": 0.3502340316772461,
"learning_rate": 0.0005113410529150397,
"loss": 3.2938,
"step": 22050
},
{
"epoch": 5.9474590869939705,
"grad_norm": 0.3122784197330475,
"learning_rate": 0.0005111390871145819,
"loss": 3.2833,
"step": 22100
},
{
"epoch": 5.960917312661499,
"grad_norm": 0.34915393590927124,
"learning_rate": 0.000510937121314124,
"loss": 3.2983,
"step": 22150
},
{
"epoch": 5.974375538329027,
"grad_norm": 0.35112547874450684,
"learning_rate": 0.0005107351555136663,
"loss": 3.3069,
"step": 22200
},
{
"epoch": 5.987833763996555,
"grad_norm": 0.3398374915122986,
"learning_rate": 0.0005105331897132085,
"loss": 3.2882,
"step": 22250
},
{
"epoch": 6.001076658053402,
"grad_norm": 0.352754682302475,
"learning_rate": 0.0005103312239127507,
"loss": 3.2717,
"step": 22300
},
{
"epoch": 6.0145348837209305,
"grad_norm": 0.38981321454048157,
"learning_rate": 0.0005101292581122929,
"loss": 3.1929,
"step": 22350
},
{
"epoch": 6.027993109388458,
"grad_norm": 0.3339367210865021,
"learning_rate": 0.0005099272923118351,
"loss": 3.1948,
"step": 22400
},
{
"epoch": 6.041451335055986,
"grad_norm": 0.37438926100730896,
"learning_rate": 0.0005097253265113773,
"loss": 3.1856,
"step": 22450
},
{
"epoch": 6.054909560723514,
"grad_norm": 0.3462595045566559,
"learning_rate": 0.0005095233607109196,
"loss": 3.2029,
"step": 22500
},
{
"epoch": 6.068367786391042,
"grad_norm": 0.3641872704029083,
"learning_rate": 0.0005093213949104617,
"loss": 3.197,
"step": 22550
},
{
"epoch": 6.0818260120585705,
"grad_norm": 0.3788128197193146,
"learning_rate": 0.000509119429110004,
"loss": 3.2085,
"step": 22600
},
{
"epoch": 6.095284237726098,
"grad_norm": 0.3462986648082733,
"learning_rate": 0.0005089174633095462,
"loss": 3.2136,
"step": 22650
},
{
"epoch": 6.108742463393626,
"grad_norm": 0.34483978152275085,
"learning_rate": 0.0005087154975090885,
"loss": 3.2224,
"step": 22700
},
{
"epoch": 6.122200689061154,
"grad_norm": 0.3590666353702545,
"learning_rate": 0.0005085135317086306,
"loss": 3.2083,
"step": 22750
},
{
"epoch": 6.135658914728682,
"grad_norm": 0.3543896973133087,
"learning_rate": 0.0005083115659081729,
"loss": 3.224,
"step": 22800
},
{
"epoch": 6.149117140396211,
"grad_norm": 0.32914939522743225,
"learning_rate": 0.0005081096001077151,
"loss": 3.212,
"step": 22850
},
{
"epoch": 6.162575366063738,
"grad_norm": 0.3688567876815796,
"learning_rate": 0.0005079076343072573,
"loss": 3.2234,
"step": 22900
},
{
"epoch": 6.176033591731266,
"grad_norm": 0.3502994179725647,
"learning_rate": 0.0005077056685067995,
"loss": 3.2192,
"step": 22950
},
{
"epoch": 6.189491817398794,
"grad_norm": 0.3282775580883026,
"learning_rate": 0.0005075037027063417,
"loss": 3.2204,
"step": 23000
},
{
"epoch": 6.189491817398794,
"eval_accuracy": 0.38417778015352233,
"eval_loss": 3.3597404956817627,
"eval_runtime": 87.7605,
"eval_samples_per_second": 205.24,
"eval_steps_per_second": 12.83,
"step": 23000
},
{
"epoch": 6.2029500430663225,
"grad_norm": 0.3568970263004303,
"learning_rate": 0.0005073017369058839,
"loss": 3.2305,
"step": 23050
},
{
"epoch": 6.21640826873385,
"grad_norm": 0.3501264452934265,
"learning_rate": 0.0005070997711054262,
"loss": 3.2408,
"step": 23100
},
{
"epoch": 6.229866494401378,
"grad_norm": 0.38300803303718567,
"learning_rate": 0.0005068978053049683,
"loss": 3.2223,
"step": 23150
},
{
"epoch": 6.243324720068906,
"grad_norm": 0.34357237815856934,
"learning_rate": 0.0005066958395045105,
"loss": 3.2282,
"step": 23200
},
{
"epoch": 6.256782945736434,
"grad_norm": 0.35204562544822693,
"learning_rate": 0.0005064938737040527,
"loss": 3.2218,
"step": 23250
},
{
"epoch": 6.270241171403962,
"grad_norm": 0.3680749237537384,
"learning_rate": 0.0005062919079035949,
"loss": 3.2297,
"step": 23300
},
{
"epoch": 6.28369939707149,
"grad_norm": 0.3626226484775543,
"learning_rate": 0.0005060899421031372,
"loss": 3.2332,
"step": 23350
},
{
"epoch": 6.297157622739018,
"grad_norm": 0.3031291961669922,
"learning_rate": 0.0005058879763026793,
"loss": 3.2339,
"step": 23400
},
{
"epoch": 6.310615848406546,
"grad_norm": 0.3484286069869995,
"learning_rate": 0.0005056860105022216,
"loss": 3.2366,
"step": 23450
},
{
"epoch": 6.324074074074074,
"grad_norm": 0.3633332848548889,
"learning_rate": 0.0005054840447017637,
"loss": 3.2369,
"step": 23500
},
{
"epoch": 6.337532299741602,
"grad_norm": 0.384171724319458,
"learning_rate": 0.0005052820789013061,
"loss": 3.2423,
"step": 23550
},
{
"epoch": 6.35099052540913,
"grad_norm": 0.3473833501338959,
"learning_rate": 0.0005050801131008482,
"loss": 3.2322,
"step": 23600
},
{
"epoch": 6.364448751076658,
"grad_norm": 0.3337860107421875,
"learning_rate": 0.0005048781473003905,
"loss": 3.2337,
"step": 23650
},
{
"epoch": 6.377906976744186,
"grad_norm": 0.32066184282302856,
"learning_rate": 0.0005046761814999326,
"loss": 3.2396,
"step": 23700
},
{
"epoch": 6.3913652024117145,
"grad_norm": 0.3586727976799011,
"learning_rate": 0.0005044742156994749,
"loss": 3.2382,
"step": 23750
},
{
"epoch": 6.404823428079242,
"grad_norm": 0.3122645318508148,
"learning_rate": 0.0005042722498990171,
"loss": 3.2338,
"step": 23800
},
{
"epoch": 6.41828165374677,
"grad_norm": 0.3277284502983093,
"learning_rate": 0.0005040702840985593,
"loss": 3.2286,
"step": 23850
},
{
"epoch": 6.431739879414298,
"grad_norm": 0.3412085175514221,
"learning_rate": 0.0005038683182981015,
"loss": 3.2421,
"step": 23900
},
{
"epoch": 6.445198105081826,
"grad_norm": 0.37467247247695923,
"learning_rate": 0.0005036663524976436,
"loss": 3.2218,
"step": 23950
},
{
"epoch": 6.458656330749354,
"grad_norm": 0.37306609749794006,
"learning_rate": 0.0005034643866971859,
"loss": 3.2478,
"step": 24000
},
{
"epoch": 6.458656330749354,
"eval_accuracy": 0.38472438248386703,
"eval_loss": 3.3520491123199463,
"eval_runtime": 87.8028,
"eval_samples_per_second": 205.141,
"eval_steps_per_second": 12.824,
"step": 24000
},
{
"epoch": 6.472114556416882,
"grad_norm": 0.3688383996486664,
"learning_rate": 0.0005032624208967281,
"loss": 3.2365,
"step": 24050
},
{
"epoch": 6.48557278208441,
"grad_norm": 0.38653844594955444,
"learning_rate": 0.0005030604550962703,
"loss": 3.2543,
"step": 24100
},
{
"epoch": 6.499031007751938,
"grad_norm": 0.3507419228553772,
"learning_rate": 0.0005028584892958125,
"loss": 3.2465,
"step": 24150
},
{
"epoch": 6.5124892334194655,
"grad_norm": 0.33346864581108093,
"learning_rate": 0.0005026565234953547,
"loss": 3.2468,
"step": 24200
},
{
"epoch": 6.525947459086994,
"grad_norm": 0.3589414954185486,
"learning_rate": 0.0005024545576948969,
"loss": 3.2423,
"step": 24250
},
{
"epoch": 6.539405684754522,
"grad_norm": 0.33978089690208435,
"learning_rate": 0.0005022525918944392,
"loss": 3.2464,
"step": 24300
},
{
"epoch": 6.55286391042205,
"grad_norm": 0.3351050317287445,
"learning_rate": 0.0005020506260939813,
"loss": 3.2504,
"step": 24350
},
{
"epoch": 6.566322136089578,
"grad_norm": 0.360805869102478,
"learning_rate": 0.0005018486602935236,
"loss": 3.2588,
"step": 24400
},
{
"epoch": 6.579780361757106,
"grad_norm": 0.35320815443992615,
"learning_rate": 0.0005016466944930658,
"loss": 3.2711,
"step": 24450
},
{
"epoch": 6.593238587424634,
"grad_norm": 0.3797634541988373,
"learning_rate": 0.0005014447286926081,
"loss": 3.2452,
"step": 24500
},
{
"epoch": 6.606696813092162,
"grad_norm": 0.33502110838890076,
"learning_rate": 0.0005012427628921502,
"loss": 3.2437,
"step": 24550
},
{
"epoch": 6.62015503875969,
"grad_norm": 0.3418447971343994,
"learning_rate": 0.0005010407970916925,
"loss": 3.2457,
"step": 24600
},
{
"epoch": 6.633613264427218,
"grad_norm": 0.33172404766082764,
"learning_rate": 0.0005008388312912346,
"loss": 3.2407,
"step": 24650
},
{
"epoch": 6.647071490094746,
"grad_norm": 0.3561548888683319,
"learning_rate": 0.0005006368654907769,
"loss": 3.2516,
"step": 24700
},
{
"epoch": 6.660529715762274,
"grad_norm": 0.3367748260498047,
"learning_rate": 0.0005004348996903191,
"loss": 3.2498,
"step": 24750
},
{
"epoch": 6.673987941429802,
"grad_norm": 0.3726949691772461,
"learning_rate": 0.0005002329338898613,
"loss": 3.2442,
"step": 24800
},
{
"epoch": 6.68744616709733,
"grad_norm": 0.33080199360847473,
"learning_rate": 0.0005000309680894035,
"loss": 3.2484,
"step": 24850
},
{
"epoch": 6.7009043927648575,
"grad_norm": 0.3477340340614319,
"learning_rate": 0.0004998290022889456,
"loss": 3.2554,
"step": 24900
},
{
"epoch": 6.714362618432386,
"grad_norm": 0.34718841314315796,
"learning_rate": 0.0004996270364884879,
"loss": 3.249,
"step": 24950
},
{
"epoch": 6.727820844099914,
"grad_norm": 0.3280639946460724,
"learning_rate": 0.0004994250706880301,
"loss": 3.2492,
"step": 25000
},
{
"epoch": 6.727820844099914,
"eval_accuracy": 0.385652552570954,
"eval_loss": 3.342715263366699,
"eval_runtime": 87.6454,
"eval_samples_per_second": 205.51,
"eval_steps_per_second": 12.847,
"step": 25000
},
{
"epoch": 6.741279069767442,
"grad_norm": 0.33485645055770874,
"learning_rate": 0.0004992231048875723,
"loss": 3.2543,
"step": 25050
},
{
"epoch": 6.754737295434969,
"grad_norm": 0.34157443046569824,
"learning_rate": 0.0004990211390871145,
"loss": 3.2516,
"step": 25100
},
{
"epoch": 6.768195521102498,
"grad_norm": 0.33038467168807983,
"learning_rate": 0.0004988191732866567,
"loss": 3.2469,
"step": 25150
},
{
"epoch": 6.781653746770026,
"grad_norm": 0.3318648636341095,
"learning_rate": 0.0004986172074861989,
"loss": 3.2579,
"step": 25200
},
{
"epoch": 6.795111972437554,
"grad_norm": 0.34127184748649597,
"learning_rate": 0.0004984152416857412,
"loss": 3.2505,
"step": 25250
},
{
"epoch": 6.808570198105082,
"grad_norm": 0.35870561003685,
"learning_rate": 0.0004982132758852834,
"loss": 3.2452,
"step": 25300
},
{
"epoch": 6.822028423772609,
"grad_norm": 0.33743295073509216,
"learning_rate": 0.0004980113100848256,
"loss": 3.2506,
"step": 25350
},
{
"epoch": 6.835486649440138,
"grad_norm": 0.3330428898334503,
"learning_rate": 0.0004978093442843678,
"loss": 3.2646,
"step": 25400
},
{
"epoch": 6.848944875107666,
"grad_norm": 0.3399394452571869,
"learning_rate": 0.0004976073784839101,
"loss": 3.2571,
"step": 25450
},
{
"epoch": 6.862403100775194,
"grad_norm": 0.35361889004707336,
"learning_rate": 0.0004974054126834522,
"loss": 3.2598,
"step": 25500
},
{
"epoch": 6.875861326442722,
"grad_norm": 0.3305911421775818,
"learning_rate": 0.0004972034468829945,
"loss": 3.2477,
"step": 25550
},
{
"epoch": 6.8893195521102495,
"grad_norm": 0.3251939117908478,
"learning_rate": 0.0004970014810825366,
"loss": 3.2501,
"step": 25600
},
{
"epoch": 6.902777777777778,
"grad_norm": 0.3742504119873047,
"learning_rate": 0.0004967995152820789,
"loss": 3.2601,
"step": 25650
},
{
"epoch": 6.916236003445306,
"grad_norm": 0.3559335172176361,
"learning_rate": 0.0004965975494816211,
"loss": 3.2633,
"step": 25700
},
{
"epoch": 6.929694229112834,
"grad_norm": 0.31849339604377747,
"learning_rate": 0.0004963955836811633,
"loss": 3.2591,
"step": 25750
},
{
"epoch": 6.943152454780362,
"grad_norm": 0.3387935161590576,
"learning_rate": 0.0004961936178807055,
"loss": 3.2542,
"step": 25800
},
{
"epoch": 6.9566106804478895,
"grad_norm": 0.330844521522522,
"learning_rate": 0.0004959916520802476,
"loss": 3.2516,
"step": 25850
},
{
"epoch": 6.970068906115418,
"grad_norm": 0.3462895154953003,
"learning_rate": 0.0004957896862797899,
"loss": 3.253,
"step": 25900
},
{
"epoch": 6.983527131782946,
"grad_norm": 0.35414084792137146,
"learning_rate": 0.0004955877204793321,
"loss": 3.2574,
"step": 25950
},
{
"epoch": 6.996985357450473,
"grad_norm": 0.32836857438087463,
"learning_rate": 0.0004953857546788743,
"loss": 3.264,
"step": 26000
},
{
"epoch": 6.996985357450473,
"eval_accuracy": 0.38607888283218883,
"eval_loss": 3.3374903202056885,
"eval_runtime": 88.3286,
"eval_samples_per_second": 203.92,
"eval_steps_per_second": 12.748,
"step": 26000
},
{
"epoch": 7.010228251507321,
"grad_norm": 0.34610164165496826,
"learning_rate": 0.0004951837888784165,
"loss": 3.1889,
"step": 26050
},
{
"epoch": 7.0236864771748495,
"grad_norm": 0.35366278886795044,
"learning_rate": 0.0004949818230779587,
"loss": 3.1516,
"step": 26100
},
{
"epoch": 7.037144702842378,
"grad_norm": 0.3489835262298584,
"learning_rate": 0.0004947798572775009,
"loss": 3.1586,
"step": 26150
},
{
"epoch": 7.050602928509905,
"grad_norm": 0.37238720059394836,
"learning_rate": 0.0004945778914770432,
"loss": 3.1644,
"step": 26200
},
{
"epoch": 7.064061154177433,
"grad_norm": 0.33292341232299805,
"learning_rate": 0.0004943759256765854,
"loss": 3.1729,
"step": 26250
},
{
"epoch": 7.077519379844961,
"grad_norm": 0.3589879274368286,
"learning_rate": 0.0004941739598761276,
"loss": 3.1727,
"step": 26300
},
{
"epoch": 7.09097760551249,
"grad_norm": 0.35007166862487793,
"learning_rate": 0.0004939719940756698,
"loss": 3.1729,
"step": 26350
},
{
"epoch": 7.104435831180017,
"grad_norm": 0.36499452590942383,
"learning_rate": 0.0004937700282752121,
"loss": 3.1658,
"step": 26400
},
{
"epoch": 7.117894056847545,
"grad_norm": 0.35566914081573486,
"learning_rate": 0.0004935680624747542,
"loss": 3.1732,
"step": 26450
},
{
"epoch": 7.131352282515073,
"grad_norm": 0.38588660955429077,
"learning_rate": 0.0004933660966742965,
"loss": 3.1778,
"step": 26500
},
{
"epoch": 7.144810508182601,
"grad_norm": 0.3605356514453888,
"learning_rate": 0.0004931641308738386,
"loss": 3.1752,
"step": 26550
},
{
"epoch": 7.15826873385013,
"grad_norm": 0.37103271484375,
"learning_rate": 0.0004929621650733809,
"loss": 3.1925,
"step": 26600
},
{
"epoch": 7.171726959517657,
"grad_norm": 0.3648276627063751,
"learning_rate": 0.0004927601992729231,
"loss": 3.1899,
"step": 26650
},
{
"epoch": 7.185185185185185,
"grad_norm": 0.35377174615859985,
"learning_rate": 0.0004925582334724652,
"loss": 3.1915,
"step": 26700
},
{
"epoch": 7.198643410852713,
"grad_norm": 0.3399772047996521,
"learning_rate": 0.0004923562676720075,
"loss": 3.1972,
"step": 26750
},
{
"epoch": 7.2121016365202415,
"grad_norm": 0.35400402545928955,
"learning_rate": 0.0004921543018715497,
"loss": 3.199,
"step": 26800
},
{
"epoch": 7.225559862187769,
"grad_norm": 0.3436429500579834,
"learning_rate": 0.0004919523360710919,
"loss": 3.1839,
"step": 26850
},
{
"epoch": 7.239018087855297,
"grad_norm": 0.31973743438720703,
"learning_rate": 0.0004917503702706341,
"loss": 3.1854,
"step": 26900
},
{
"epoch": 7.252476313522825,
"grad_norm": 0.3425561487674713,
"learning_rate": 0.0004915484044701763,
"loss": 3.1974,
"step": 26950
},
{
"epoch": 7.265934539190353,
"grad_norm": 0.3549090027809143,
"learning_rate": 0.0004913464386697185,
"loss": 3.2051,
"step": 27000
},
{
"epoch": 7.265934539190353,
"eval_accuracy": 0.38628944043827274,
"eval_loss": 3.3425750732421875,
"eval_runtime": 88.1866,
"eval_samples_per_second": 204.249,
"eval_steps_per_second": 12.768,
"step": 27000
},
{
"epoch": 7.279392764857882,
"grad_norm": 0.34049826860427856,
"learning_rate": 0.0004911444728692608,
"loss": 3.1987,
"step": 27050
},
{
"epoch": 7.292850990525409,
"grad_norm": 0.3492026627063751,
"learning_rate": 0.000490942507068803,
"loss": 3.1879,
"step": 27100
},
{
"epoch": 7.306309216192937,
"grad_norm": 0.33971208333969116,
"learning_rate": 0.0004907405412683452,
"loss": 3.2011,
"step": 27150
},
{
"epoch": 7.319767441860465,
"grad_norm": 0.345683217048645,
"learning_rate": 0.0004905385754678874,
"loss": 3.2004,
"step": 27200
},
{
"epoch": 7.333225667527993,
"grad_norm": 0.37204256653785706,
"learning_rate": 0.0004903366096674297,
"loss": 3.1979,
"step": 27250
},
{
"epoch": 7.346683893195521,
"grad_norm": 0.367231547832489,
"learning_rate": 0.0004901346438669718,
"loss": 3.2097,
"step": 27300
},
{
"epoch": 7.360142118863049,
"grad_norm": 0.33875802159309387,
"learning_rate": 0.0004899326780665141,
"loss": 3.222,
"step": 27350
},
{
"epoch": 7.373600344530577,
"grad_norm": 0.3583943843841553,
"learning_rate": 0.0004897307122660562,
"loss": 3.1964,
"step": 27400
},
{
"epoch": 7.387058570198105,
"grad_norm": 0.34471067786216736,
"learning_rate": 0.0004895287464655985,
"loss": 3.2117,
"step": 27450
},
{
"epoch": 7.4005167958656335,
"grad_norm": 0.335271954536438,
"learning_rate": 0.0004893267806651407,
"loss": 3.201,
"step": 27500
},
{
"epoch": 7.413975021533161,
"grad_norm": 0.36515283584594727,
"learning_rate": 0.0004891248148646829,
"loss": 3.2106,
"step": 27550
},
{
"epoch": 7.427433247200689,
"grad_norm": 0.3432753086090088,
"learning_rate": 0.0004889228490642251,
"loss": 3.2181,
"step": 27600
},
{
"epoch": 7.440891472868217,
"grad_norm": 0.3460571765899658,
"learning_rate": 0.0004887208832637672,
"loss": 3.2091,
"step": 27650
},
{
"epoch": 7.454349698535745,
"grad_norm": 0.366629034280777,
"learning_rate": 0.0004885189174633095,
"loss": 3.2156,
"step": 27700
},
{
"epoch": 7.467807924203273,
"grad_norm": 0.3270829916000366,
"learning_rate": 0.0004883169516628517,
"loss": 3.2139,
"step": 27750
},
{
"epoch": 7.481266149870801,
"grad_norm": 0.3518711030483246,
"learning_rate": 0.0004881149858623939,
"loss": 3.2114,
"step": 27800
},
{
"epoch": 7.494724375538329,
"grad_norm": 0.34649714827537537,
"learning_rate": 0.00048791302006193614,
"loss": 3.2123,
"step": 27850
},
{
"epoch": 7.508182601205857,
"grad_norm": 0.5178732872009277,
"learning_rate": 0.00048771105426147833,
"loss": 3.2167,
"step": 27900
},
{
"epoch": 7.521640826873385,
"grad_norm": 0.3424026370048523,
"learning_rate": 0.00048750908846102053,
"loss": 3.2017,
"step": 27950
},
{
"epoch": 7.535099052540913,
"grad_norm": 0.34950610995292664,
"learning_rate": 0.0004873071226605628,
"loss": 3.2193,
"step": 28000
},
{
"epoch": 7.535099052540913,
"eval_accuracy": 0.38668426311139387,
"eval_loss": 3.3392179012298584,
"eval_runtime": 87.7547,
"eval_samples_per_second": 205.254,
"eval_steps_per_second": 12.831,
"step": 28000
},
{
"epoch": 7.548557278208441,
"grad_norm": 0.34104108810424805,
"learning_rate": 0.00048710515686010503,
"loss": 3.2092,
"step": 28050
},
{
"epoch": 7.562015503875969,
"grad_norm": 0.32871460914611816,
"learning_rate": 0.0004869031910596472,
"loss": 3.2222,
"step": 28100
},
{
"epoch": 7.575473729543497,
"grad_norm": 0.34514591097831726,
"learning_rate": 0.0004867012252591894,
"loss": 3.2164,
"step": 28150
},
{
"epoch": 7.588931955211025,
"grad_norm": 0.33675217628479004,
"learning_rate": 0.0004864992594587316,
"loss": 3.2146,
"step": 28200
},
{
"epoch": 7.602390180878553,
"grad_norm": 0.35956332087516785,
"learning_rate": 0.0004862972936582738,
"loss": 3.2161,
"step": 28250
},
{
"epoch": 7.615848406546081,
"grad_norm": 0.3696342706680298,
"learning_rate": 0.00048609532785781606,
"loss": 3.211,
"step": 28300
},
{
"epoch": 7.629306632213609,
"grad_norm": 0.340660035610199,
"learning_rate": 0.00048589336205735826,
"loss": 3.2156,
"step": 28350
},
{
"epoch": 7.6427648578811365,
"grad_norm": 0.34237396717071533,
"learning_rate": 0.00048569139625690046,
"loss": 3.2297,
"step": 28400
},
{
"epoch": 7.656223083548665,
"grad_norm": 0.37521055340766907,
"learning_rate": 0.00048548943045644265,
"loss": 3.2151,
"step": 28450
},
{
"epoch": 7.669681309216193,
"grad_norm": 0.3448121249675751,
"learning_rate": 0.00048528746465598485,
"loss": 3.201,
"step": 28500
},
{
"epoch": 7.683139534883721,
"grad_norm": 0.34937021136283875,
"learning_rate": 0.0004850854988555271,
"loss": 3.2188,
"step": 28550
},
{
"epoch": 7.696597760551249,
"grad_norm": 0.32854047417640686,
"learning_rate": 0.0004848835330550693,
"loss": 3.2189,
"step": 28600
},
{
"epoch": 7.7100559862187765,
"grad_norm": 0.35428887605667114,
"learning_rate": 0.0004846815672546115,
"loss": 3.2181,
"step": 28650
},
{
"epoch": 7.723514211886305,
"grad_norm": 0.34827983379364014,
"learning_rate": 0.0004844796014541537,
"loss": 3.2252,
"step": 28700
},
{
"epoch": 7.736972437553833,
"grad_norm": 0.3507974147796631,
"learning_rate": 0.0004842776356536959,
"loss": 3.2141,
"step": 28750
},
{
"epoch": 7.750430663221361,
"grad_norm": 0.3425598740577698,
"learning_rate": 0.00048407566985323813,
"loss": 3.2175,
"step": 28800
},
{
"epoch": 7.763888888888889,
"grad_norm": 0.33663254976272583,
"learning_rate": 0.0004838737040527804,
"loss": 3.2409,
"step": 28850
},
{
"epoch": 7.777347114556417,
"grad_norm": 0.33253276348114014,
"learning_rate": 0.0004836717382523226,
"loss": 3.2262,
"step": 28900
},
{
"epoch": 7.790805340223945,
"grad_norm": 0.3696092665195465,
"learning_rate": 0.0004834697724518648,
"loss": 3.2065,
"step": 28950
},
{
"epoch": 7.804263565891473,
"grad_norm": 0.32295504212379456,
"learning_rate": 0.000483267806651407,
"loss": 3.2262,
"step": 29000
},
{
"epoch": 7.804263565891473,
"eval_accuracy": 0.3878248378011093,
"eval_loss": 3.3295774459838867,
"eval_runtime": 86.7214,
"eval_samples_per_second": 207.7,
"eval_steps_per_second": 12.984,
"step": 29000
},
{
"epoch": 7.817721791559001,
"grad_norm": 0.3298531770706177,
"learning_rate": 0.0004830658408509492,
"loss": 3.2213,
"step": 29050
},
{
"epoch": 7.831180017226529,
"grad_norm": 0.3383207619190216,
"learning_rate": 0.0004828638750504914,
"loss": 3.2347,
"step": 29100
},
{
"epoch": 7.844638242894057,
"grad_norm": 0.36010101437568665,
"learning_rate": 0.0004826619092500336,
"loss": 3.2233,
"step": 29150
},
{
"epoch": 7.858096468561585,
"grad_norm": 0.34656408429145813,
"learning_rate": 0.0004824599434495758,
"loss": 3.2259,
"step": 29200
},
{
"epoch": 7.871554694229113,
"grad_norm": 0.3213461637496948,
"learning_rate": 0.00048225797764911806,
"loss": 3.2245,
"step": 29250
},
{
"epoch": 7.885012919896641,
"grad_norm": 0.35127902030944824,
"learning_rate": 0.00048205601184866026,
"loss": 3.2149,
"step": 29300
},
{
"epoch": 7.8984711455641685,
"grad_norm": 0.3342956602573395,
"learning_rate": 0.00048185404604820245,
"loss": 3.2277,
"step": 29350
},
{
"epoch": 7.911929371231697,
"grad_norm": 0.359724760055542,
"learning_rate": 0.00048165208024774465,
"loss": 3.2216,
"step": 29400
},
{
"epoch": 7.925387596899225,
"grad_norm": 0.3557015657424927,
"learning_rate": 0.00048145011444728685,
"loss": 3.2249,
"step": 29450
},
{
"epoch": 7.938845822566753,
"grad_norm": 0.3411562740802765,
"learning_rate": 0.0004812481486468291,
"loss": 3.2336,
"step": 29500
},
{
"epoch": 7.95230404823428,
"grad_norm": 0.3151187598705292,
"learning_rate": 0.0004810461828463713,
"loss": 3.2251,
"step": 29550
},
{
"epoch": 7.965762273901809,
"grad_norm": 0.36215487122535706,
"learning_rate": 0.0004808442170459135,
"loss": 3.2345,
"step": 29600
},
{
"epoch": 7.979220499569337,
"grad_norm": 0.3466815650463104,
"learning_rate": 0.0004806422512454557,
"loss": 3.2336,
"step": 29650
},
{
"epoch": 7.992678725236865,
"grad_norm": 0.340318888425827,
"learning_rate": 0.00048044028544499793,
"loss": 3.2202,
"step": 29700
},
{
"epoch": 8.005921619293712,
"grad_norm": 0.3526352643966675,
"learning_rate": 0.0004802383196445402,
"loss": 3.1826,
"step": 29750
},
{
"epoch": 8.01937984496124,
"grad_norm": 0.3376880884170532,
"learning_rate": 0.0004800363538440824,
"loss": 3.1301,
"step": 29800
},
{
"epoch": 8.032838070628769,
"grad_norm": 0.35756751894950867,
"learning_rate": 0.0004798343880436246,
"loss": 3.1278,
"step": 29850
},
{
"epoch": 8.046296296296296,
"grad_norm": 0.3583872616291046,
"learning_rate": 0.00047963242224316683,
"loss": 3.1379,
"step": 29900
},
{
"epoch": 8.059754521963825,
"grad_norm": 0.352107435464859,
"learning_rate": 0.000479430456442709,
"loss": 3.1389,
"step": 29950
},
{
"epoch": 8.073212747631352,
"grad_norm": 0.32794874906539917,
"learning_rate": 0.0004792284906422512,
"loss": 3.1215,
"step": 30000
},
{
"epoch": 8.073212747631352,
"eval_accuracy": 0.3876267745834154,
"eval_loss": 3.33420991897583,
"eval_runtime": 86.664,
"eval_samples_per_second": 207.837,
"eval_steps_per_second": 12.993,
"step": 30000
},
{
"epoch": 8.08667097329888,
"grad_norm": 0.35623717308044434,
"learning_rate": 0.0004790265248417934,
"loss": 3.1381,
"step": 30050
},
{
"epoch": 8.100129198966409,
"grad_norm": 0.34889814257621765,
"learning_rate": 0.0004788245590413356,
"loss": 3.1485,
"step": 30100
},
{
"epoch": 8.113587424633936,
"grad_norm": 0.3647051155567169,
"learning_rate": 0.00047862259324087786,
"loss": 3.1463,
"step": 30150
},
{
"epoch": 8.127045650301465,
"grad_norm": 0.3717019855976105,
"learning_rate": 0.00047842062744042006,
"loss": 3.1599,
"step": 30200
},
{
"epoch": 8.140503875968992,
"grad_norm": 0.36561229825019836,
"learning_rate": 0.00047821866163996225,
"loss": 3.1543,
"step": 30250
},
{
"epoch": 8.15396210163652,
"grad_norm": 0.37198320031166077,
"learning_rate": 0.00047801669583950445,
"loss": 3.1518,
"step": 30300
},
{
"epoch": 8.167420327304049,
"grad_norm": 0.37193116545677185,
"learning_rate": 0.00047781473003904665,
"loss": 3.1594,
"step": 30350
},
{
"epoch": 8.180878552971576,
"grad_norm": 0.3370911777019501,
"learning_rate": 0.0004776127642385889,
"loss": 3.1515,
"step": 30400
},
{
"epoch": 8.194336778639105,
"grad_norm": 0.3725023865699768,
"learning_rate": 0.0004774107984381311,
"loss": 3.1578,
"step": 30450
},
{
"epoch": 8.207795004306632,
"grad_norm": 0.3886215090751648,
"learning_rate": 0.0004772088326376733,
"loss": 3.1708,
"step": 30500
},
{
"epoch": 8.22125322997416,
"grad_norm": 0.3475479185581207,
"learning_rate": 0.0004770068668372155,
"loss": 3.1582,
"step": 30550
},
{
"epoch": 8.234711455641689,
"grad_norm": 0.36613041162490845,
"learning_rate": 0.0004768049010367577,
"loss": 3.1532,
"step": 30600
},
{
"epoch": 8.248169681309216,
"grad_norm": 0.3678821325302124,
"learning_rate": 0.0004766029352363,
"loss": 3.1717,
"step": 30650
},
{
"epoch": 8.261627906976745,
"grad_norm": 0.3694922924041748,
"learning_rate": 0.0004764009694358422,
"loss": 3.1564,
"step": 30700
},
{
"epoch": 8.275086132644272,
"grad_norm": 0.39171549677848816,
"learning_rate": 0.0004761990036353844,
"loss": 3.1649,
"step": 30750
},
{
"epoch": 8.2885443583118,
"grad_norm": 0.3456292450428009,
"learning_rate": 0.0004759970378349266,
"loss": 3.1692,
"step": 30800
},
{
"epoch": 8.302002583979329,
"grad_norm": 0.37906062602996826,
"learning_rate": 0.0004757950720344688,
"loss": 3.1663,
"step": 30850
},
{
"epoch": 8.315460809646856,
"grad_norm": 0.36503514647483826,
"learning_rate": 0.000475593106234011,
"loss": 3.1734,
"step": 30900
},
{
"epoch": 8.328919035314383,
"grad_norm": 0.3400176167488098,
"learning_rate": 0.0004753911404335532,
"loss": 3.1824,
"step": 30950
},
{
"epoch": 8.342377260981912,
"grad_norm": 0.33662930130958557,
"learning_rate": 0.0004751891746330954,
"loss": 3.1749,
"step": 31000
},
{
"epoch": 8.342377260981912,
"eval_accuracy": 0.3880142092703581,
"eval_loss": 3.3296737670898438,
"eval_runtime": 87.4696,
"eval_samples_per_second": 205.923,
"eval_steps_per_second": 12.873,
"step": 31000
},
{
"epoch": 8.35583548664944,
"grad_norm": 0.3648368716239929,
"learning_rate": 0.0004749872088326376,
"loss": 3.1802,
"step": 31050
},
{
"epoch": 8.369293712316969,
"grad_norm": 0.3359640836715698,
"learning_rate": 0.00047478524303217986,
"loss": 3.1849,
"step": 31100
},
{
"epoch": 8.382751937984496,
"grad_norm": 0.35760825872421265,
"learning_rate": 0.00047458327723172206,
"loss": 3.1772,
"step": 31150
},
{
"epoch": 8.396210163652023,
"grad_norm": 0.382489413022995,
"learning_rate": 0.00047438131143126425,
"loss": 3.1804,
"step": 31200
},
{
"epoch": 8.409668389319553,
"grad_norm": 0.3471967875957489,
"learning_rate": 0.00047417934563080645,
"loss": 3.1865,
"step": 31250
},
{
"epoch": 8.42312661498708,
"grad_norm": 0.3476189374923706,
"learning_rate": 0.00047397737983034864,
"loss": 3.1876,
"step": 31300
},
{
"epoch": 8.436584840654609,
"grad_norm": 0.35890787839889526,
"learning_rate": 0.0004737754140298909,
"loss": 3.183,
"step": 31350
},
{
"epoch": 8.450043066322136,
"grad_norm": 0.33333414793014526,
"learning_rate": 0.0004735734482294331,
"loss": 3.1763,
"step": 31400
},
{
"epoch": 8.463501291989663,
"grad_norm": 0.35873493552207947,
"learning_rate": 0.0004733714824289753,
"loss": 3.1922,
"step": 31450
},
{
"epoch": 8.476959517657193,
"grad_norm": 0.35295209288597107,
"learning_rate": 0.0004731695166285175,
"loss": 3.1904,
"step": 31500
},
{
"epoch": 8.49041774332472,
"grad_norm": 0.3632144629955292,
"learning_rate": 0.0004729675508280598,
"loss": 3.171,
"step": 31550
},
{
"epoch": 8.503875968992247,
"grad_norm": 0.3484487235546112,
"learning_rate": 0.000472765585027602,
"loss": 3.1783,
"step": 31600
},
{
"epoch": 8.517334194659776,
"grad_norm": 0.3353268504142761,
"learning_rate": 0.0004725636192271442,
"loss": 3.1844,
"step": 31650
},
{
"epoch": 8.530792420327304,
"grad_norm": 0.3567664325237274,
"learning_rate": 0.0004723616534266864,
"loss": 3.1806,
"step": 31700
},
{
"epoch": 8.544250645994833,
"grad_norm": 0.35602352023124695,
"learning_rate": 0.0004721596876262286,
"loss": 3.1921,
"step": 31750
},
{
"epoch": 8.55770887166236,
"grad_norm": 0.34894222021102905,
"learning_rate": 0.0004719577218257708,
"loss": 3.1907,
"step": 31800
},
{
"epoch": 8.571167097329887,
"grad_norm": 0.3618151843547821,
"learning_rate": 0.000471755756025313,
"loss": 3.1857,
"step": 31850
},
{
"epoch": 8.584625322997416,
"grad_norm": 0.34802767634391785,
"learning_rate": 0.0004715537902248552,
"loss": 3.1868,
"step": 31900
},
{
"epoch": 8.598083548664944,
"grad_norm": 0.3836779296398163,
"learning_rate": 0.0004713518244243974,
"loss": 3.196,
"step": 31950
},
{
"epoch": 8.611541774332473,
"grad_norm": 0.3747783601284027,
"learning_rate": 0.00047114985862393966,
"loss": 3.1944,
"step": 32000
},
{
"epoch": 8.611541774332473,
"eval_accuracy": 0.38849529754679746,
"eval_loss": 3.32307767868042,
"eval_runtime": 87.6755,
"eval_samples_per_second": 205.439,
"eval_steps_per_second": 12.843,
"step": 32000
},
{
"epoch": 8.625,
"grad_norm": 0.33180832862854004,
"learning_rate": 0.00047094789282348186,
"loss": 3.192,
"step": 32050
},
{
"epoch": 8.638458225667527,
"grad_norm": 0.3259965777397156,
"learning_rate": 0.00047074592702302405,
"loss": 3.1975,
"step": 32100
},
{
"epoch": 8.651916451335056,
"grad_norm": 0.3438642919063568,
"learning_rate": 0.00047054396122256625,
"loss": 3.1899,
"step": 32150
},
{
"epoch": 8.665374677002584,
"grad_norm": 0.4004620313644409,
"learning_rate": 0.00047034199542210844,
"loss": 3.1882,
"step": 32200
},
{
"epoch": 8.678832902670113,
"grad_norm": 0.3403623402118683,
"learning_rate": 0.0004701400296216507,
"loss": 3.1806,
"step": 32250
},
{
"epoch": 8.69229112833764,
"grad_norm": 0.3368901014328003,
"learning_rate": 0.0004699380638211929,
"loss": 3.1914,
"step": 32300
},
{
"epoch": 8.705749354005167,
"grad_norm": 0.3399077355861664,
"learning_rate": 0.0004697360980207351,
"loss": 3.1939,
"step": 32350
},
{
"epoch": 8.719207579672696,
"grad_norm": 0.36910849809646606,
"learning_rate": 0.0004695341322202773,
"loss": 3.1849,
"step": 32400
},
{
"epoch": 8.732665805340224,
"grad_norm": 0.3569251596927643,
"learning_rate": 0.0004693321664198196,
"loss": 3.2088,
"step": 32450
},
{
"epoch": 8.746124031007753,
"grad_norm": 0.33960893750190735,
"learning_rate": 0.0004691302006193618,
"loss": 3.1919,
"step": 32500
},
{
"epoch": 8.75958225667528,
"grad_norm": 0.36516520380973816,
"learning_rate": 0.000468928234818904,
"loss": 3.1945,
"step": 32550
},
{
"epoch": 8.773040482342807,
"grad_norm": 0.3720601499080658,
"learning_rate": 0.0004687262690184462,
"loss": 3.2066,
"step": 32600
},
{
"epoch": 8.786498708010337,
"grad_norm": 0.3640294373035431,
"learning_rate": 0.00046852430321798837,
"loss": 3.1955,
"step": 32650
},
{
"epoch": 8.799956933677864,
"grad_norm": 0.36675596237182617,
"learning_rate": 0.0004683223374175306,
"loss": 3.1951,
"step": 32700
},
{
"epoch": 8.813415159345391,
"grad_norm": 0.34109437465667725,
"learning_rate": 0.0004681203716170728,
"loss": 3.198,
"step": 32750
},
{
"epoch": 8.82687338501292,
"grad_norm": 0.38729915022850037,
"learning_rate": 0.000467918405816615,
"loss": 3.1938,
"step": 32800
},
{
"epoch": 8.840331610680447,
"grad_norm": 0.3687329590320587,
"learning_rate": 0.0004677164400161572,
"loss": 3.1969,
"step": 32850
},
{
"epoch": 8.853789836347977,
"grad_norm": 0.38186848163604736,
"learning_rate": 0.0004675144742156994,
"loss": 3.1982,
"step": 32900
},
{
"epoch": 8.867248062015504,
"grad_norm": 0.3522675037384033,
"learning_rate": 0.00046731250841524166,
"loss": 3.193,
"step": 32950
},
{
"epoch": 8.880706287683031,
"grad_norm": 0.3754071891307831,
"learning_rate": 0.00046711054261478385,
"loss": 3.1866,
"step": 33000
},
{
"epoch": 8.880706287683031,
"eval_accuracy": 0.388980079816326,
"eval_loss": 3.317448616027832,
"eval_runtime": 88.2344,
"eval_samples_per_second": 204.138,
"eval_steps_per_second": 12.761,
"step": 33000
},
{
"epoch": 8.89416451335056,
"grad_norm": 0.3508091866970062,
"learning_rate": 0.00046690857681432605,
"loss": 3.2062,
"step": 33050
},
{
"epoch": 8.907622739018088,
"grad_norm": 0.3021308183670044,
"learning_rate": 0.00046670661101386825,
"loss": 3.2019,
"step": 33100
},
{
"epoch": 8.921080964685617,
"grad_norm": 0.33504754304885864,
"learning_rate": 0.00046650464521341044,
"loss": 3.2104,
"step": 33150
},
{
"epoch": 8.934539190353144,
"grad_norm": 0.32749006152153015,
"learning_rate": 0.0004663026794129527,
"loss": 3.2021,
"step": 33200
},
{
"epoch": 8.947997416020671,
"grad_norm": 0.3676239252090454,
"learning_rate": 0.0004661007136124949,
"loss": 3.1997,
"step": 33250
},
{
"epoch": 8.9614556416882,
"grad_norm": 0.3513721823692322,
"learning_rate": 0.00046589874781203714,
"loss": 3.2092,
"step": 33300
},
{
"epoch": 8.974913867355728,
"grad_norm": 0.3342949450016022,
"learning_rate": 0.0004656967820115794,
"loss": 3.1986,
"step": 33350
},
{
"epoch": 8.988372093023255,
"grad_norm": 0.3422151505947113,
"learning_rate": 0.0004654948162111216,
"loss": 3.204,
"step": 33400
},
{
"epoch": 9.001614987080103,
"grad_norm": 0.3525255620479584,
"learning_rate": 0.0004652928504106638,
"loss": 3.1975,
"step": 33450
},
{
"epoch": 9.015073212747632,
"grad_norm": 0.3380824625492096,
"learning_rate": 0.000465090884610206,
"loss": 3.0995,
"step": 33500
},
{
"epoch": 9.02853143841516,
"grad_norm": 0.3548218309879303,
"learning_rate": 0.00046488891880974817,
"loss": 3.0869,
"step": 33550
},
{
"epoch": 9.041989664082687,
"grad_norm": 0.3807477056980133,
"learning_rate": 0.0004646869530092904,
"loss": 3.1083,
"step": 33600
},
{
"epoch": 9.055447889750216,
"grad_norm": 0.33358967304229736,
"learning_rate": 0.0004644849872088326,
"loss": 3.1118,
"step": 33650
},
{
"epoch": 9.068906115417743,
"grad_norm": 0.3386094272136688,
"learning_rate": 0.0004642830214083748,
"loss": 3.1083,
"step": 33700
},
{
"epoch": 9.082364341085272,
"grad_norm": 0.3573903441429138,
"learning_rate": 0.000464081055607917,
"loss": 3.1117,
"step": 33750
},
{
"epoch": 9.0958225667528,
"grad_norm": 0.33198776841163635,
"learning_rate": 0.0004638790898074592,
"loss": 3.1315,
"step": 33800
},
{
"epoch": 9.109280792420327,
"grad_norm": 0.35434484481811523,
"learning_rate": 0.00046367712400700146,
"loss": 3.1135,
"step": 33850
},
{
"epoch": 9.122739018087856,
"grad_norm": 0.36608824133872986,
"learning_rate": 0.00046347515820654365,
"loss": 3.1209,
"step": 33900
},
{
"epoch": 9.136197243755383,
"grad_norm": 0.3372901380062103,
"learning_rate": 0.00046327319240608585,
"loss": 3.1284,
"step": 33950
},
{
"epoch": 9.14965546942291,
"grad_norm": 0.3538866639137268,
"learning_rate": 0.00046307122660562805,
"loss": 3.1258,
"step": 34000
},
{
"epoch": 9.14965546942291,
"eval_accuracy": 0.3888806679434845,
"eval_loss": 3.325159788131714,
"eval_runtime": 87.3642,
"eval_samples_per_second": 206.171,
"eval_steps_per_second": 12.889,
"step": 34000
},
{
"epoch": 9.16311369509044,
"grad_norm": 0.3983548879623413,
"learning_rate": 0.00046286926080517024,
"loss": 3.1304,
"step": 34050
},
{
"epoch": 9.176571920757967,
"grad_norm": 0.36228519678115845,
"learning_rate": 0.0004626672950047125,
"loss": 3.1338,
"step": 34100
},
{
"epoch": 9.190030146425496,
"grad_norm": 0.3778652846813202,
"learning_rate": 0.0004624653292042547,
"loss": 3.1291,
"step": 34150
},
{
"epoch": 9.203488372093023,
"grad_norm": 0.36719369888305664,
"learning_rate": 0.00046226336340379694,
"loss": 3.1375,
"step": 34200
},
{
"epoch": 9.21694659776055,
"grad_norm": 0.35282188653945923,
"learning_rate": 0.00046206139760333913,
"loss": 3.132,
"step": 34250
},
{
"epoch": 9.23040482342808,
"grad_norm": 0.33646711707115173,
"learning_rate": 0.0004618594318028814,
"loss": 3.1454,
"step": 34300
},
{
"epoch": 9.243863049095607,
"grad_norm": 0.3522685170173645,
"learning_rate": 0.0004616574660024236,
"loss": 3.1459,
"step": 34350
},
{
"epoch": 9.257321274763136,
"grad_norm": 0.36446160078048706,
"learning_rate": 0.0004614555002019658,
"loss": 3.1453,
"step": 34400
},
{
"epoch": 9.270779500430663,
"grad_norm": 0.3586257994174957,
"learning_rate": 0.000461253534401508,
"loss": 3.1475,
"step": 34450
},
{
"epoch": 9.28423772609819,
"grad_norm": 0.4070618748664856,
"learning_rate": 0.00046105156860105017,
"loss": 3.1352,
"step": 34500
},
{
"epoch": 9.29769595176572,
"grad_norm": 0.3892214894294739,
"learning_rate": 0.0004608496028005924,
"loss": 3.1476,
"step": 34550
},
{
"epoch": 9.311154177433247,
"grad_norm": 0.3597724139690399,
"learning_rate": 0.0004606476370001346,
"loss": 3.1403,
"step": 34600
},
{
"epoch": 9.324612403100776,
"grad_norm": 0.36641639471054077,
"learning_rate": 0.0004604456711996768,
"loss": 3.1488,
"step": 34650
},
{
"epoch": 9.338070628768303,
"grad_norm": 0.373392790555954,
"learning_rate": 0.000460243705399219,
"loss": 3.1429,
"step": 34700
},
{
"epoch": 9.35152885443583,
"grad_norm": 0.3421435058116913,
"learning_rate": 0.0004600417395987612,
"loss": 3.1677,
"step": 34750
},
{
"epoch": 9.36498708010336,
"grad_norm": 0.3360295295715332,
"learning_rate": 0.00045983977379830345,
"loss": 3.1539,
"step": 34800
},
{
"epoch": 9.378445305770887,
"grad_norm": 0.37752994894981384,
"learning_rate": 0.00045963780799784565,
"loss": 3.1446,
"step": 34850
},
{
"epoch": 9.391903531438416,
"grad_norm": 0.3226233720779419,
"learning_rate": 0.00045943584219738785,
"loss": 3.1392,
"step": 34900
},
{
"epoch": 9.405361757105943,
"grad_norm": 0.35676461458206177,
"learning_rate": 0.00045923387639693004,
"loss": 3.1527,
"step": 34950
},
{
"epoch": 9.41881998277347,
"grad_norm": 0.3426799178123474,
"learning_rate": 0.00045903191059647224,
"loss": 3.157,
"step": 35000
},
{
"epoch": 9.41881998277347,
"eval_accuracy": 0.3893731641397581,
"eval_loss": 3.3207197189331055,
"eval_runtime": 87.7887,
"eval_samples_per_second": 205.174,
"eval_steps_per_second": 12.826,
"step": 35000
},
{
"epoch": 9.432278208441,
"grad_norm": 0.3415524363517761,
"learning_rate": 0.0004588299447960145,
"loss": 3.1635,
"step": 35050
},
{
"epoch": 9.445736434108527,
"grad_norm": 0.3550635278224945,
"learning_rate": 0.00045862797899555674,
"loss": 3.1633,
"step": 35100
},
{
"epoch": 9.459194659776054,
"grad_norm": 0.33996516466140747,
"learning_rate": 0.00045842601319509894,
"loss": 3.16,
"step": 35150
},
{
"epoch": 9.472652885443583,
"grad_norm": 0.35329610109329224,
"learning_rate": 0.00045822404739464113,
"loss": 3.1616,
"step": 35200
},
{
"epoch": 9.48611111111111,
"grad_norm": 0.33477357029914856,
"learning_rate": 0.0004580220815941834,
"loss": 3.1517,
"step": 35250
},
{
"epoch": 9.49956933677864,
"grad_norm": 0.34619536995887756,
"learning_rate": 0.0004578201157937256,
"loss": 3.1549,
"step": 35300
},
{
"epoch": 9.513027562446167,
"grad_norm": 0.3580850660800934,
"learning_rate": 0.0004576181499932678,
"loss": 3.1615,
"step": 35350
},
{
"epoch": 9.526485788113694,
"grad_norm": 0.33334505558013916,
"learning_rate": 0.00045741618419280997,
"loss": 3.1563,
"step": 35400
},
{
"epoch": 9.539944013781223,
"grad_norm": 0.3636830747127533,
"learning_rate": 0.0004572142183923522,
"loss": 3.1713,
"step": 35450
},
{
"epoch": 9.55340223944875,
"grad_norm": 0.32588592171669006,
"learning_rate": 0.0004570122525918944,
"loss": 3.17,
"step": 35500
},
{
"epoch": 9.56686046511628,
"grad_norm": 0.3375829756259918,
"learning_rate": 0.0004568102867914366,
"loss": 3.1677,
"step": 35550
},
{
"epoch": 9.580318690783807,
"grad_norm": 0.36470961570739746,
"learning_rate": 0.0004566083209909788,
"loss": 3.1621,
"step": 35600
},
{
"epoch": 9.593776916451334,
"grad_norm": 0.37833788990974426,
"learning_rate": 0.000456406355190521,
"loss": 3.166,
"step": 35650
},
{
"epoch": 9.607235142118864,
"grad_norm": 0.3430384397506714,
"learning_rate": 0.00045620438939006326,
"loss": 3.172,
"step": 35700
},
{
"epoch": 9.62069336778639,
"grad_norm": 0.3499787151813507,
"learning_rate": 0.00045600242358960545,
"loss": 3.1631,
"step": 35750
},
{
"epoch": 9.634151593453918,
"grad_norm": 0.3460032343864441,
"learning_rate": 0.00045580045778914765,
"loss": 3.1615,
"step": 35800
},
{
"epoch": 9.647609819121447,
"grad_norm": 0.3551907539367676,
"learning_rate": 0.00045559849198868984,
"loss": 3.167,
"step": 35850
},
{
"epoch": 9.661068044788975,
"grad_norm": 0.3299188017845154,
"learning_rate": 0.00045539652618823204,
"loss": 3.1753,
"step": 35900
},
{
"epoch": 9.674526270456504,
"grad_norm": 0.3797774612903595,
"learning_rate": 0.0004551945603877743,
"loss": 3.1723,
"step": 35950
},
{
"epoch": 9.687984496124031,
"grad_norm": 0.345142126083374,
"learning_rate": 0.00045499259458731654,
"loss": 3.1741,
"step": 36000
},
{
"epoch": 9.687984496124031,
"eval_accuracy": 0.3896962798882067,
"eval_loss": 3.3145992755889893,
"eval_runtime": 87.9367,
"eval_samples_per_second": 204.829,
"eval_steps_per_second": 12.805,
"step": 36000
},
{
"epoch": 9.701442721791558,
"grad_norm": 0.3563687801361084,
"learning_rate": 0.00045479062878685874,
"loss": 3.1675,
"step": 36050
},
{
"epoch": 9.714900947459087,
"grad_norm": 0.3506101071834564,
"learning_rate": 0.00045458866298640093,
"loss": 3.1727,
"step": 36100
},
{
"epoch": 9.728359173126615,
"grad_norm": 0.3728286027908325,
"learning_rate": 0.0004543866971859432,
"loss": 3.1741,
"step": 36150
},
{
"epoch": 9.741817398794144,
"grad_norm": 0.3504398465156555,
"learning_rate": 0.0004541847313854854,
"loss": 3.1642,
"step": 36200
},
{
"epoch": 9.755275624461671,
"grad_norm": 0.34962764382362366,
"learning_rate": 0.0004539827655850276,
"loss": 3.176,
"step": 36250
},
{
"epoch": 9.768733850129198,
"grad_norm": 0.3368580639362335,
"learning_rate": 0.00045378079978456977,
"loss": 3.1726,
"step": 36300
},
{
"epoch": 9.782192075796727,
"grad_norm": 0.3463308811187744,
"learning_rate": 0.00045357883398411197,
"loss": 3.1703,
"step": 36350
},
{
"epoch": 9.795650301464255,
"grad_norm": 0.34614890813827515,
"learning_rate": 0.0004533768681836542,
"loss": 3.169,
"step": 36400
},
{
"epoch": 9.809108527131784,
"grad_norm": 0.3787212371826172,
"learning_rate": 0.0004531749023831964,
"loss": 3.1676,
"step": 36450
},
{
"epoch": 9.822566752799311,
"grad_norm": 0.35479286313056946,
"learning_rate": 0.0004529729365827386,
"loss": 3.1922,
"step": 36500
},
{
"epoch": 9.836024978466838,
"grad_norm": 0.3774144947528839,
"learning_rate": 0.0004527709707822808,
"loss": 3.1752,
"step": 36550
},
{
"epoch": 9.849483204134367,
"grad_norm": 0.35198262333869934,
"learning_rate": 0.000452569004981823,
"loss": 3.17,
"step": 36600
},
{
"epoch": 9.862941429801895,
"grad_norm": 0.3490941524505615,
"learning_rate": 0.00045236703918136525,
"loss": 3.1777,
"step": 36650
},
{
"epoch": 9.876399655469424,
"grad_norm": 0.367546945810318,
"learning_rate": 0.00045216507338090745,
"loss": 3.1761,
"step": 36700
},
{
"epoch": 9.889857881136951,
"grad_norm": 0.3417567312717438,
"learning_rate": 0.00045196310758044964,
"loss": 3.1851,
"step": 36750
},
{
"epoch": 9.903316106804478,
"grad_norm": 0.3448176383972168,
"learning_rate": 0.00045176114177999184,
"loss": 3.1708,
"step": 36800
},
{
"epoch": 9.916774332472007,
"grad_norm": 0.35334399342536926,
"learning_rate": 0.00045155917597953404,
"loss": 3.1809,
"step": 36850
},
{
"epoch": 9.930232558139535,
"grad_norm": 0.36826086044311523,
"learning_rate": 0.00045135721017907634,
"loss": 3.1849,
"step": 36900
},
{
"epoch": 9.943690783807062,
"grad_norm": 0.3243925869464874,
"learning_rate": 0.00045115524437861854,
"loss": 3.1786,
"step": 36950
},
{
"epoch": 9.957149009474591,
"grad_norm": 0.35191184282302856,
"learning_rate": 0.00045095327857816073,
"loss": 3.1805,
"step": 37000
},
{
"epoch": 9.957149009474591,
"eval_accuracy": 0.39071288851572317,
"eval_loss": 3.3055007457733154,
"eval_runtime": 88.172,
"eval_samples_per_second": 204.283,
"eval_steps_per_second": 12.77,
"step": 37000
},
{
"epoch": 9.970607235142118,
"grad_norm": 0.38593989610671997,
"learning_rate": 0.00045075131277770293,
"loss": 3.1912,
"step": 37050
},
{
"epoch": 9.984065460809648,
"grad_norm": 0.36267486214637756,
"learning_rate": 0.0004505493469772452,
"loss": 3.1848,
"step": 37100
},
{
"epoch": 9.997523686477175,
"grad_norm": 0.3603392243385315,
"learning_rate": 0.0004503473811767874,
"loss": 3.1785,
"step": 37150
},
{
"epoch": 10.010766580534023,
"grad_norm": 0.3506993353366852,
"learning_rate": 0.00045014541537632957,
"loss": 3.1058,
"step": 37200
},
{
"epoch": 10.02422480620155,
"grad_norm": 0.348012238740921,
"learning_rate": 0.00044994344957587177,
"loss": 3.0732,
"step": 37250
},
{
"epoch": 10.037683031869078,
"grad_norm": 0.34269097447395325,
"learning_rate": 0.00044974148377541396,
"loss": 3.0742,
"step": 37300
},
{
"epoch": 10.051141257536607,
"grad_norm": 0.3394729495048523,
"learning_rate": 0.0004495395179749562,
"loss": 3.0842,
"step": 37350
},
{
"epoch": 10.064599483204134,
"grad_norm": 0.3448140621185303,
"learning_rate": 0.0004493375521744984,
"loss": 3.0895,
"step": 37400
},
{
"epoch": 10.078057708871663,
"grad_norm": 0.38341960310935974,
"learning_rate": 0.0004491355863740406,
"loss": 3.0889,
"step": 37450
},
{
"epoch": 10.09151593453919,
"grad_norm": 0.3606523871421814,
"learning_rate": 0.0004489336205735828,
"loss": 3.0876,
"step": 37500
},
{
"epoch": 10.104974160206718,
"grad_norm": 0.3559594750404358,
"learning_rate": 0.00044873165477312505,
"loss": 3.0941,
"step": 37550
},
{
"epoch": 10.118432385874247,
"grad_norm": 0.35412347316741943,
"learning_rate": 0.00044852968897266725,
"loss": 3.1045,
"step": 37600
},
{
"epoch": 10.131890611541774,
"grad_norm": 0.3366471529006958,
"learning_rate": 0.00044832772317220945,
"loss": 3.1012,
"step": 37650
},
{
"epoch": 10.145348837209303,
"grad_norm": 0.4085768163204193,
"learning_rate": 0.00044812575737175164,
"loss": 3.1144,
"step": 37700
},
{
"epoch": 10.15880706287683,
"grad_norm": 0.3462596833705902,
"learning_rate": 0.00044792379157129384,
"loss": 3.1093,
"step": 37750
},
{
"epoch": 10.172265288544358,
"grad_norm": 0.40925300121307373,
"learning_rate": 0.00044772182577083614,
"loss": 3.1168,
"step": 37800
},
{
"epoch": 10.185723514211887,
"grad_norm": 0.3763587772846222,
"learning_rate": 0.00044751985997037834,
"loss": 3.1081,
"step": 37850
},
{
"epoch": 10.199181739879414,
"grad_norm": 0.35570356249809265,
"learning_rate": 0.00044731789416992053,
"loss": 3.1153,
"step": 37900
},
{
"epoch": 10.212639965546943,
"grad_norm": 0.35475507378578186,
"learning_rate": 0.00044711592836946273,
"loss": 3.1183,
"step": 37950
},
{
"epoch": 10.22609819121447,
"grad_norm": 0.35603535175323486,
"learning_rate": 0.000446913962569005,
"loss": 3.1222,
"step": 38000
},
{
"epoch": 10.22609819121447,
"eval_accuracy": 0.3902001840043146,
"eval_loss": 3.3156235218048096,
"eval_runtime": 87.8345,
"eval_samples_per_second": 205.068,
"eval_steps_per_second": 12.82,
"step": 38000
},
{
"epoch": 10.239556416881998,
"grad_norm": 0.3575981557369232,
"learning_rate": 0.0004467119967685472,
"loss": 3.1125,
"step": 38050
},
{
"epoch": 10.253014642549527,
"grad_norm": 0.37977248430252075,
"learning_rate": 0.00044651003096808937,
"loss": 3.121,
"step": 38100
},
{
"epoch": 10.266472868217054,
"grad_norm": 0.3563908040523529,
"learning_rate": 0.00044630806516763157,
"loss": 3.1301,
"step": 38150
},
{
"epoch": 10.279931093884581,
"grad_norm": 0.38932687044143677,
"learning_rate": 0.00044610609936717376,
"loss": 3.1217,
"step": 38200
},
{
"epoch": 10.29338931955211,
"grad_norm": 0.36320436000823975,
"learning_rate": 0.000445904133566716,
"loss": 3.1087,
"step": 38250
},
{
"epoch": 10.306847545219638,
"grad_norm": 0.37047919631004333,
"learning_rate": 0.0004457021677662582,
"loss": 3.1263,
"step": 38300
},
{
"epoch": 10.320305770887167,
"grad_norm": 0.3520812690258026,
"learning_rate": 0.0004455002019658004,
"loss": 3.1224,
"step": 38350
},
{
"epoch": 10.333763996554694,
"grad_norm": 0.3531203269958496,
"learning_rate": 0.0004452982361653426,
"loss": 3.1409,
"step": 38400
},
{
"epoch": 10.347222222222221,
"grad_norm": 0.36067909002304077,
"learning_rate": 0.0004450962703648848,
"loss": 3.1286,
"step": 38450
},
{
"epoch": 10.36068044788975,
"grad_norm": 0.34704872965812683,
"learning_rate": 0.00044489430456442705,
"loss": 3.1307,
"step": 38500
},
{
"epoch": 10.374138673557278,
"grad_norm": 0.377408504486084,
"learning_rate": 0.00044469233876396925,
"loss": 3.1355,
"step": 38550
},
{
"epoch": 10.387596899224807,
"grad_norm": 0.3663313686847687,
"learning_rate": 0.00044449037296351144,
"loss": 3.1439,
"step": 38600
},
{
"epoch": 10.401055124892334,
"grad_norm": 0.3542163074016571,
"learning_rate": 0.0004442884071630537,
"loss": 3.1279,
"step": 38650
},
{
"epoch": 10.414513350559861,
"grad_norm": 0.33422955870628357,
"learning_rate": 0.00044408644136259594,
"loss": 3.1333,
"step": 38700
},
{
"epoch": 10.42797157622739,
"grad_norm": 0.3569745421409607,
"learning_rate": 0.00044388447556213814,
"loss": 3.1405,
"step": 38750
},
{
"epoch": 10.441429801894918,
"grad_norm": 0.37442442774772644,
"learning_rate": 0.00044368250976168033,
"loss": 3.1325,
"step": 38800
},
{
"epoch": 10.454888027562447,
"grad_norm": 0.35368022322654724,
"learning_rate": 0.00044348054396122253,
"loss": 3.1391,
"step": 38850
},
{
"epoch": 10.468346253229974,
"grad_norm": 0.3554447293281555,
"learning_rate": 0.0004432785781607647,
"loss": 3.1434,
"step": 38900
},
{
"epoch": 10.481804478897502,
"grad_norm": 0.36405107378959656,
"learning_rate": 0.000443076612360307,
"loss": 3.1422,
"step": 38950
},
{
"epoch": 10.49526270456503,
"grad_norm": 0.34430205821990967,
"learning_rate": 0.0004428746465598492,
"loss": 3.1334,
"step": 39000
},
{
"epoch": 10.49526270456503,
"eval_accuracy": 0.3905743637748785,
"eval_loss": 3.314378023147583,
"eval_runtime": 88.0967,
"eval_samples_per_second": 204.457,
"eval_steps_per_second": 12.781,
"step": 39000
},
{
"epoch": 10.508720930232558,
"grad_norm": 0.33116546273231506,
"learning_rate": 0.00044267268075939137,
"loss": 3.147,
"step": 39050
},
{
"epoch": 10.522179155900087,
"grad_norm": 0.344468355178833,
"learning_rate": 0.00044247071495893357,
"loss": 3.1402,
"step": 39100
},
{
"epoch": 10.535637381567614,
"grad_norm": 0.3650263845920563,
"learning_rate": 0.00044226874915847576,
"loss": 3.1404,
"step": 39150
},
{
"epoch": 10.549095607235142,
"grad_norm": 0.3484593629837036,
"learning_rate": 0.000442066783358018,
"loss": 3.1412,
"step": 39200
},
{
"epoch": 10.56255383290267,
"grad_norm": 0.351365864276886,
"learning_rate": 0.0004418648175575602,
"loss": 3.1512,
"step": 39250
},
{
"epoch": 10.576012058570198,
"grad_norm": 0.3528461456298828,
"learning_rate": 0.0004416628517571024,
"loss": 3.1396,
"step": 39300
},
{
"epoch": 10.589470284237725,
"grad_norm": 0.3310932517051697,
"learning_rate": 0.0004414608859566446,
"loss": 3.1524,
"step": 39350
},
{
"epoch": 10.602928509905254,
"grad_norm": 0.3519033193588257,
"learning_rate": 0.00044125892015618685,
"loss": 3.1505,
"step": 39400
},
{
"epoch": 10.616386735572782,
"grad_norm": 0.3464779257774353,
"learning_rate": 0.00044105695435572905,
"loss": 3.143,
"step": 39450
},
{
"epoch": 10.62984496124031,
"grad_norm": 0.3933066427707672,
"learning_rate": 0.00044085498855527124,
"loss": 3.1493,
"step": 39500
},
{
"epoch": 10.643303186907838,
"grad_norm": 0.3580518364906311,
"learning_rate": 0.0004406530227548135,
"loss": 3.1533,
"step": 39550
},
{
"epoch": 10.656761412575365,
"grad_norm": 0.424843966960907,
"learning_rate": 0.00044045105695435574,
"loss": 3.1308,
"step": 39600
},
{
"epoch": 10.670219638242894,
"grad_norm": 0.35034167766571045,
"learning_rate": 0.00044024909115389794,
"loss": 3.1618,
"step": 39650
},
{
"epoch": 10.683677863910422,
"grad_norm": 0.34508782625198364,
"learning_rate": 0.00044004712535344014,
"loss": 3.1489,
"step": 39700
},
{
"epoch": 10.69713608957795,
"grad_norm": 0.3473265469074249,
"learning_rate": 0.00043984515955298233,
"loss": 3.1419,
"step": 39750
},
{
"epoch": 10.710594315245478,
"grad_norm": 0.3852803409099579,
"learning_rate": 0.00043964319375252453,
"loss": 3.1469,
"step": 39800
},
{
"epoch": 10.724052540913005,
"grad_norm": 0.3702237904071808,
"learning_rate": 0.0004394412279520668,
"loss": 3.1483,
"step": 39850
},
{
"epoch": 10.737510766580534,
"grad_norm": 0.3771539330482483,
"learning_rate": 0.000439239262151609,
"loss": 3.1567,
"step": 39900
},
{
"epoch": 10.750968992248062,
"grad_norm": 0.35787418484687805,
"learning_rate": 0.00043903729635115117,
"loss": 3.1577,
"step": 39950
},
{
"epoch": 10.764427217915589,
"grad_norm": 0.35247600078582764,
"learning_rate": 0.00043883533055069337,
"loss": 3.1497,
"step": 40000
},
{
"epoch": 10.764427217915589,
"eval_accuracy": 0.39091518896078414,
"eval_loss": 3.306220054626465,
"eval_runtime": 88.1301,
"eval_samples_per_second": 204.38,
"eval_steps_per_second": 12.777,
"step": 40000
},
{
"epoch": 10.777885443583118,
"grad_norm": 0.35555049777030945,
"learning_rate": 0.00043863336475023556,
"loss": 3.1444,
"step": 40050
},
{
"epoch": 10.791343669250645,
"grad_norm": 0.3686332702636719,
"learning_rate": 0.0004384313989497778,
"loss": 3.1548,
"step": 40100
},
{
"epoch": 10.804801894918175,
"grad_norm": 0.3736445903778076,
"learning_rate": 0.00043822943314932,
"loss": 3.1487,
"step": 40150
},
{
"epoch": 10.818260120585702,
"grad_norm": 0.3347689211368561,
"learning_rate": 0.0004380274673488622,
"loss": 3.1531,
"step": 40200
},
{
"epoch": 10.83171834625323,
"grad_norm": 0.34181126952171326,
"learning_rate": 0.0004378255015484044,
"loss": 3.1399,
"step": 40250
},
{
"epoch": 10.845176571920758,
"grad_norm": 0.3598361909389496,
"learning_rate": 0.0004376235357479466,
"loss": 3.1542,
"step": 40300
},
{
"epoch": 10.858634797588286,
"grad_norm": 0.34456607699394226,
"learning_rate": 0.00043742156994748885,
"loss": 3.1559,
"step": 40350
},
{
"epoch": 10.872093023255815,
"grad_norm": 0.3753799498081207,
"learning_rate": 0.00043721960414703104,
"loss": 3.1669,
"step": 40400
},
{
"epoch": 10.885551248923342,
"grad_norm": 0.36482125520706177,
"learning_rate": 0.0004370176383465733,
"loss": 3.1523,
"step": 40450
},
{
"epoch": 10.89900947459087,
"grad_norm": 0.38292697072029114,
"learning_rate": 0.0004368156725461155,
"loss": 3.161,
"step": 40500
},
{
"epoch": 10.912467700258398,
"grad_norm": 0.32300931215286255,
"learning_rate": 0.00043661370674565774,
"loss": 3.1547,
"step": 40550
},
{
"epoch": 10.925925925925926,
"grad_norm": 0.35990387201309204,
"learning_rate": 0.00043641174094519994,
"loss": 3.1549,
"step": 40600
},
{
"epoch": 10.939384151593455,
"grad_norm": 0.38015124201774597,
"learning_rate": 0.00043620977514474213,
"loss": 3.1514,
"step": 40650
},
{
"epoch": 10.952842377260982,
"grad_norm": 0.3817886710166931,
"learning_rate": 0.00043600780934428433,
"loss": 3.1636,
"step": 40700
},
{
"epoch": 10.96630060292851,
"grad_norm": 0.3476899564266205,
"learning_rate": 0.0004358058435438265,
"loss": 3.1509,
"step": 40750
},
{
"epoch": 10.979758828596038,
"grad_norm": 0.34082624316215515,
"learning_rate": 0.0004356038777433688,
"loss": 3.1678,
"step": 40800
},
{
"epoch": 10.993217054263566,
"grad_norm": 0.34645843505859375,
"learning_rate": 0.00043540191194291097,
"loss": 3.1662,
"step": 40850
},
{
"epoch": 11.006459948320414,
"grad_norm": 0.3485479950904846,
"learning_rate": 0.00043519994614245317,
"loss": 3.116,
"step": 40900
},
{
"epoch": 11.019918173987941,
"grad_norm": 0.3675366938114166,
"learning_rate": 0.00043499798034199536,
"loss": 3.0632,
"step": 40950
},
{
"epoch": 11.03337639965547,
"grad_norm": 0.3343339264392853,
"learning_rate": 0.00043479601454153756,
"loss": 3.0698,
"step": 41000
},
{
"epoch": 11.03337639965547,
"eval_accuracy": 0.3909793992524227,
"eval_loss": 3.31091046333313,
"eval_runtime": 84.2407,
"eval_samples_per_second": 213.816,
"eval_steps_per_second": 13.366,
"step": 41000
},
{
"epoch": 11.046834625322997,
"grad_norm": 0.36293190717697144,
"learning_rate": 0.0004345940487410798,
"loss": 3.0706,
"step": 41050
},
{
"epoch": 11.060292850990525,
"grad_norm": 0.3535679876804352,
"learning_rate": 0.000434392082940622,
"loss": 3.0633,
"step": 41100
},
{
"epoch": 11.073751076658054,
"grad_norm": 0.33150696754455566,
"learning_rate": 0.0004341901171401642,
"loss": 3.0755,
"step": 41150
},
{
"epoch": 11.087209302325581,
"grad_norm": 0.36507776379585266,
"learning_rate": 0.0004339881513397064,
"loss": 3.0763,
"step": 41200
},
{
"epoch": 11.10066752799311,
"grad_norm": 0.36090418696403503,
"learning_rate": 0.0004337861855392486,
"loss": 3.0905,
"step": 41250
},
{
"epoch": 11.114125753660637,
"grad_norm": 0.38765281438827515,
"learning_rate": 0.00043358421973879084,
"loss": 3.0822,
"step": 41300
},
{
"epoch": 11.127583979328165,
"grad_norm": 0.33757099509239197,
"learning_rate": 0.0004333822539383331,
"loss": 3.0931,
"step": 41350
},
{
"epoch": 11.141042204995694,
"grad_norm": 0.35109296441078186,
"learning_rate": 0.0004331802881378753,
"loss": 3.0877,
"step": 41400
},
{
"epoch": 11.154500430663221,
"grad_norm": 0.3479686677455902,
"learning_rate": 0.00043297832233741754,
"loss": 3.0916,
"step": 41450
},
{
"epoch": 11.16795865633075,
"grad_norm": 0.3615115284919739,
"learning_rate": 0.00043277635653695974,
"loss": 3.0859,
"step": 41500
},
{
"epoch": 11.181416881998278,
"grad_norm": 0.37314116954803467,
"learning_rate": 0.00043257439073650193,
"loss": 3.1043,
"step": 41550
},
{
"epoch": 11.194875107665805,
"grad_norm": 0.3375728726387024,
"learning_rate": 0.00043237242493604413,
"loss": 3.1011,
"step": 41600
},
{
"epoch": 11.208333333333334,
"grad_norm": 0.3412526547908783,
"learning_rate": 0.0004321704591355863,
"loss": 3.0902,
"step": 41650
},
{
"epoch": 11.221791559000861,
"grad_norm": 0.3929307162761688,
"learning_rate": 0.0004319684933351286,
"loss": 3.0889,
"step": 41700
},
{
"epoch": 11.235249784668389,
"grad_norm": 0.36704355478286743,
"learning_rate": 0.00043176652753467077,
"loss": 3.0918,
"step": 41750
},
{
"epoch": 11.248708010335918,
"grad_norm": 0.3677278161048889,
"learning_rate": 0.00043156456173421297,
"loss": 3.1011,
"step": 41800
},
{
"epoch": 11.262166236003445,
"grad_norm": 0.37482133507728577,
"learning_rate": 0.00043136259593375516,
"loss": 3.0974,
"step": 41850
},
{
"epoch": 11.275624461670974,
"grad_norm": 0.36803311109542847,
"learning_rate": 0.00043116063013329736,
"loss": 3.1111,
"step": 41900
},
{
"epoch": 11.289082687338501,
"grad_norm": 0.4043966829776764,
"learning_rate": 0.0004309586643328396,
"loss": 3.1013,
"step": 41950
},
{
"epoch": 11.302540913006029,
"grad_norm": 0.37155815958976746,
"learning_rate": 0.0004307566985323818,
"loss": 3.1033,
"step": 42000
},
{
"epoch": 11.302540913006029,
"eval_accuracy": 0.39119245573618455,
"eval_loss": 3.309682846069336,
"eval_runtime": 84.3256,
"eval_samples_per_second": 213.601,
"eval_steps_per_second": 13.353,
"step": 42000
},
{
"epoch": 11.315999138673558,
"grad_norm": 0.3757474422454834,
"learning_rate": 0.000430554732731924,
"loss": 3.1042,
"step": 42050
},
{
"epoch": 11.329457364341085,
"grad_norm": 0.3633876144886017,
"learning_rate": 0.0004303527669314662,
"loss": 3.0951,
"step": 42100
},
{
"epoch": 11.342915590008614,
"grad_norm": 0.34023556113243103,
"learning_rate": 0.0004301508011310084,
"loss": 3.102,
"step": 42150
},
{
"epoch": 11.356373815676141,
"grad_norm": 0.3511642515659332,
"learning_rate": 0.00042994883533055065,
"loss": 3.103,
"step": 42200
},
{
"epoch": 11.369832041343669,
"grad_norm": 0.37258774042129517,
"learning_rate": 0.0004297468695300929,
"loss": 3.1049,
"step": 42250
},
{
"epoch": 11.383290267011198,
"grad_norm": 0.3569978177547455,
"learning_rate": 0.0004295449037296351,
"loss": 3.1066,
"step": 42300
},
{
"epoch": 11.396748492678725,
"grad_norm": 0.36627477407455444,
"learning_rate": 0.0004293429379291773,
"loss": 3.1104,
"step": 42350
},
{
"epoch": 11.410206718346252,
"grad_norm": 0.33913758397102356,
"learning_rate": 0.00042914097212871954,
"loss": 3.1198,
"step": 42400
},
{
"epoch": 11.423664944013781,
"grad_norm": 0.3415125608444214,
"learning_rate": 0.00042893900632826173,
"loss": 3.1135,
"step": 42450
},
{
"epoch": 11.437123169681309,
"grad_norm": 0.38717105984687805,
"learning_rate": 0.00042873704052780393,
"loss": 3.1106,
"step": 42500
},
{
"epoch": 11.450581395348838,
"grad_norm": 0.3576693534851074,
"learning_rate": 0.0004285350747273461,
"loss": 3.108,
"step": 42550
},
{
"epoch": 11.464039621016365,
"grad_norm": 0.3778170943260193,
"learning_rate": 0.0004283331089268883,
"loss": 3.1131,
"step": 42600
},
{
"epoch": 11.477497846683892,
"grad_norm": 0.3914255201816559,
"learning_rate": 0.00042813114312643057,
"loss": 3.1215,
"step": 42650
},
{
"epoch": 11.490956072351421,
"grad_norm": 0.3620469570159912,
"learning_rate": 0.00042792917732597277,
"loss": 3.1206,
"step": 42700
},
{
"epoch": 11.504414298018949,
"grad_norm": 0.36856579780578613,
"learning_rate": 0.00042772721152551496,
"loss": 3.1164,
"step": 42750
},
{
"epoch": 11.517872523686478,
"grad_norm": 0.35470420122146606,
"learning_rate": 0.00042752524572505716,
"loss": 3.1205,
"step": 42800
},
{
"epoch": 11.531330749354005,
"grad_norm": 0.3629094362258911,
"learning_rate": 0.00042732327992459936,
"loss": 3.1215,
"step": 42850
},
{
"epoch": 11.544788975021532,
"grad_norm": 0.3661877512931824,
"learning_rate": 0.0004271213141241416,
"loss": 3.1351,
"step": 42900
},
{
"epoch": 11.558247200689062,
"grad_norm": 0.3606022000312805,
"learning_rate": 0.0004269193483236838,
"loss": 3.1315,
"step": 42950
},
{
"epoch": 11.571705426356589,
"grad_norm": 0.3552758991718292,
"learning_rate": 0.000426717382523226,
"loss": 3.1317,
"step": 43000
},
{
"epoch": 11.571705426356589,
"eval_accuracy": 0.3915242632330784,
"eval_loss": 3.3031299114227295,
"eval_runtime": 84.278,
"eval_samples_per_second": 213.721,
"eval_steps_per_second": 13.361,
"step": 43000
},
{
"epoch": 11.585163652024118,
"grad_norm": 0.3457125127315521,
"learning_rate": 0.0004265154167227682,
"loss": 3.127,
"step": 43050
},
{
"epoch": 11.598621877691645,
"grad_norm": 0.36335304379463196,
"learning_rate": 0.0004263134509223105,
"loss": 3.1305,
"step": 43100
},
{
"epoch": 11.612080103359173,
"grad_norm": 0.3702634572982788,
"learning_rate": 0.0004261114851218527,
"loss": 3.1314,
"step": 43150
},
{
"epoch": 11.625538329026702,
"grad_norm": 0.3538024425506592,
"learning_rate": 0.0004259095193213949,
"loss": 3.1253,
"step": 43200
},
{
"epoch": 11.638996554694229,
"grad_norm": 0.3644096851348877,
"learning_rate": 0.0004257075535209371,
"loss": 3.1377,
"step": 43250
},
{
"epoch": 11.652454780361758,
"grad_norm": 0.3496798574924469,
"learning_rate": 0.00042550558772047934,
"loss": 3.1354,
"step": 43300
},
{
"epoch": 11.665913006029285,
"grad_norm": 0.3464163839817047,
"learning_rate": 0.00042530362192002153,
"loss": 3.1224,
"step": 43350
},
{
"epoch": 11.679371231696813,
"grad_norm": 0.3821980953216553,
"learning_rate": 0.00042510165611956373,
"loss": 3.1225,
"step": 43400
},
{
"epoch": 11.692829457364342,
"grad_norm": 0.3555978238582611,
"learning_rate": 0.0004248996903191059,
"loss": 3.1304,
"step": 43450
},
{
"epoch": 11.706287683031869,
"grad_norm": 0.34647437930107117,
"learning_rate": 0.0004246977245186481,
"loss": 3.1398,
"step": 43500
},
{
"epoch": 11.719745908699396,
"grad_norm": 0.3689185678958893,
"learning_rate": 0.0004244957587181904,
"loss": 3.1278,
"step": 43550
},
{
"epoch": 11.733204134366925,
"grad_norm": 0.3581877648830414,
"learning_rate": 0.00042429379291773257,
"loss": 3.1262,
"step": 43600
},
{
"epoch": 11.746662360034453,
"grad_norm": 0.37423980236053467,
"learning_rate": 0.00042409182711727477,
"loss": 3.1397,
"step": 43650
},
{
"epoch": 11.760120585701982,
"grad_norm": 0.3919681906700134,
"learning_rate": 0.00042388986131681696,
"loss": 3.1285,
"step": 43700
},
{
"epoch": 11.773578811369509,
"grad_norm": 0.348802387714386,
"learning_rate": 0.00042368789551635916,
"loss": 3.1447,
"step": 43750
},
{
"epoch": 11.787037037037036,
"grad_norm": 0.3656277358531952,
"learning_rate": 0.0004234859297159014,
"loss": 3.1322,
"step": 43800
},
{
"epoch": 11.800495262704565,
"grad_norm": 0.36101657152175903,
"learning_rate": 0.0004232839639154436,
"loss": 3.1468,
"step": 43850
},
{
"epoch": 11.813953488372093,
"grad_norm": 0.35467052459716797,
"learning_rate": 0.0004230819981149858,
"loss": 3.1376,
"step": 43900
},
{
"epoch": 11.827411714039622,
"grad_norm": 0.35919854044914246,
"learning_rate": 0.000422880032314528,
"loss": 3.1309,
"step": 43950
},
{
"epoch": 11.840869939707149,
"grad_norm": 0.33611345291137695,
"learning_rate": 0.0004226780665140703,
"loss": 3.1351,
"step": 44000
},
{
"epoch": 11.840869939707149,
"eval_accuracy": 0.3920807524272794,
"eval_loss": 3.2992093563079834,
"eval_runtime": 84.2871,
"eval_samples_per_second": 213.698,
"eval_steps_per_second": 13.359,
"step": 44000
},
{
"epoch": 11.854328165374676,
"grad_norm": 0.36410412192344666,
"learning_rate": 0.0004224761007136125,
"loss": 3.1421,
"step": 44050
},
{
"epoch": 11.867786391042205,
"grad_norm": 0.3480066657066345,
"learning_rate": 0.0004222741349131547,
"loss": 3.137,
"step": 44100
},
{
"epoch": 11.881244616709733,
"grad_norm": 0.36214300990104675,
"learning_rate": 0.0004220721691126969,
"loss": 3.1478,
"step": 44150
},
{
"epoch": 11.89470284237726,
"grad_norm": 0.3660474419593811,
"learning_rate": 0.0004218702033122391,
"loss": 3.1228,
"step": 44200
},
{
"epoch": 11.90816106804479,
"grad_norm": 0.3615580201148987,
"learning_rate": 0.00042166823751178134,
"loss": 3.1344,
"step": 44250
},
{
"epoch": 11.921619293712316,
"grad_norm": 0.3521454632282257,
"learning_rate": 0.00042146627171132353,
"loss": 3.1452,
"step": 44300
},
{
"epoch": 11.935077519379846,
"grad_norm": 0.3320244252681732,
"learning_rate": 0.00042126430591086573,
"loss": 3.133,
"step": 44350
},
{
"epoch": 11.948535745047373,
"grad_norm": 0.3705757260322571,
"learning_rate": 0.0004210623401104079,
"loss": 3.1322,
"step": 44400
},
{
"epoch": 11.9619939707149,
"grad_norm": 0.34349524974823,
"learning_rate": 0.0004208603743099501,
"loss": 3.1307,
"step": 44450
},
{
"epoch": 11.97545219638243,
"grad_norm": 0.3370743691921234,
"learning_rate": 0.00042065840850949237,
"loss": 3.1353,
"step": 44500
},
{
"epoch": 11.988910422049956,
"grad_norm": 0.3711462616920471,
"learning_rate": 0.00042045644270903457,
"loss": 3.1402,
"step": 44550
},
{
"epoch": 12.002153316106805,
"grad_norm": 0.3837710916996002,
"learning_rate": 0.00042025447690857676,
"loss": 3.1314,
"step": 44600
},
{
"epoch": 12.015611541774332,
"grad_norm": 0.38589784502983093,
"learning_rate": 0.00042005251110811896,
"loss": 3.0303,
"step": 44650
},
{
"epoch": 12.029069767441861,
"grad_norm": 0.3585261106491089,
"learning_rate": 0.00041985054530766115,
"loss": 3.0456,
"step": 44700
},
{
"epoch": 12.042527993109388,
"grad_norm": 0.3696274161338806,
"learning_rate": 0.0004196485795072034,
"loss": 3.0529,
"step": 44750
},
{
"epoch": 12.055986218776916,
"grad_norm": 0.3691428005695343,
"learning_rate": 0.0004194466137067456,
"loss": 3.0489,
"step": 44800
},
{
"epoch": 12.069444444444445,
"grad_norm": 0.35204964876174927,
"learning_rate": 0.0004192446479062878,
"loss": 3.0474,
"step": 44850
},
{
"epoch": 12.082902670111972,
"grad_norm": 0.38218945264816284,
"learning_rate": 0.00041904268210583005,
"loss": 3.0588,
"step": 44900
},
{
"epoch": 12.096360895779501,
"grad_norm": 0.3627367913722992,
"learning_rate": 0.0004188407163053723,
"loss": 3.0597,
"step": 44950
},
{
"epoch": 12.109819121447028,
"grad_norm": 0.3626028597354889,
"learning_rate": 0.0004186387505049145,
"loss": 3.0598,
"step": 45000
},
{
"epoch": 12.109819121447028,
"eval_accuracy": 0.39129393189928174,
"eval_loss": 3.307757616043091,
"eval_runtime": 84.2296,
"eval_samples_per_second": 213.844,
"eval_steps_per_second": 13.368,
"step": 45000
},
{
"epoch": 12.123277347114556,
"grad_norm": 0.39511799812316895,
"learning_rate": 0.0004184367847044567,
"loss": 3.0587,
"step": 45050
},
{
"epoch": 12.136735572782085,
"grad_norm": 0.4090312719345093,
"learning_rate": 0.0004182348189039989,
"loss": 3.0599,
"step": 45100
},
{
"epoch": 12.150193798449612,
"grad_norm": 0.39701277017593384,
"learning_rate": 0.00041803285310354114,
"loss": 3.0677,
"step": 45150
},
{
"epoch": 12.163652024117141,
"grad_norm": 0.37128493189811707,
"learning_rate": 0.00041783088730308333,
"loss": 3.082,
"step": 45200
},
{
"epoch": 12.177110249784668,
"grad_norm": 0.39566662907600403,
"learning_rate": 0.00041762892150262553,
"loss": 3.0731,
"step": 45250
},
{
"epoch": 12.190568475452196,
"grad_norm": 0.32580286264419556,
"learning_rate": 0.0004174269557021677,
"loss": 3.0668,
"step": 45300
},
{
"epoch": 12.204026701119725,
"grad_norm": 0.3477148115634918,
"learning_rate": 0.0004172249899017099,
"loss": 3.0767,
"step": 45350
},
{
"epoch": 12.217484926787252,
"grad_norm": 0.3254866600036621,
"learning_rate": 0.00041702302410125217,
"loss": 3.0774,
"step": 45400
},
{
"epoch": 12.230943152454781,
"grad_norm": 0.37678778171539307,
"learning_rate": 0.00041682105830079437,
"loss": 3.0851,
"step": 45450
},
{
"epoch": 12.244401378122308,
"grad_norm": 0.38609495759010315,
"learning_rate": 0.00041661909250033656,
"loss": 3.0808,
"step": 45500
},
{
"epoch": 12.257859603789836,
"grad_norm": 0.3586466610431671,
"learning_rate": 0.00041641712669987876,
"loss": 3.0884,
"step": 45550
},
{
"epoch": 12.271317829457365,
"grad_norm": 0.37479081749916077,
"learning_rate": 0.00041621516089942096,
"loss": 3.0768,
"step": 45600
},
{
"epoch": 12.284776055124892,
"grad_norm": 0.3411691188812256,
"learning_rate": 0.0004160131950989632,
"loss": 3.0837,
"step": 45650
},
{
"epoch": 12.298234280792421,
"grad_norm": 0.36653265357017517,
"learning_rate": 0.0004158112292985054,
"loss": 3.0925,
"step": 45700
},
{
"epoch": 12.311692506459949,
"grad_norm": 0.3570050001144409,
"learning_rate": 0.0004156092634980476,
"loss": 3.0891,
"step": 45750
},
{
"epoch": 12.325150732127476,
"grad_norm": 0.37854400277137756,
"learning_rate": 0.00041540729769758985,
"loss": 3.0932,
"step": 45800
},
{
"epoch": 12.338608957795005,
"grad_norm": 0.34331968426704407,
"learning_rate": 0.0004152053318971321,
"loss": 3.1047,
"step": 45850
},
{
"epoch": 12.352067183462532,
"grad_norm": 0.38357841968536377,
"learning_rate": 0.0004150033660966743,
"loss": 3.0985,
"step": 45900
},
{
"epoch": 12.36552540913006,
"grad_norm": 0.3992276191711426,
"learning_rate": 0.0004148014002962165,
"loss": 3.1048,
"step": 45950
},
{
"epoch": 12.378983634797589,
"grad_norm": 0.4186655580997467,
"learning_rate": 0.0004145994344957587,
"loss": 3.0973,
"step": 46000
},
{
"epoch": 12.378983634797589,
"eval_accuracy": 0.39201675942935194,
"eval_loss": 3.3033642768859863,
"eval_runtime": 84.2981,
"eval_samples_per_second": 213.67,
"eval_steps_per_second": 13.357,
"step": 46000
},
{
"epoch": 12.392441860465116,
"grad_norm": 0.36977341771125793,
"learning_rate": 0.0004143974686953009,
"loss": 3.092,
"step": 46050
},
{
"epoch": 12.405900086132645,
"grad_norm": 0.34746086597442627,
"learning_rate": 0.00041419550289484313,
"loss": 3.0894,
"step": 46100
},
{
"epoch": 12.419358311800172,
"grad_norm": 0.37170976400375366,
"learning_rate": 0.00041399353709438533,
"loss": 3.0844,
"step": 46150
},
{
"epoch": 12.4328165374677,
"grad_norm": 0.37848731875419617,
"learning_rate": 0.0004137915712939275,
"loss": 3.0849,
"step": 46200
},
{
"epoch": 12.446274763135229,
"grad_norm": 0.3569401204586029,
"learning_rate": 0.0004135896054934697,
"loss": 3.1005,
"step": 46250
},
{
"epoch": 12.459732988802756,
"grad_norm": 0.3888532519340515,
"learning_rate": 0.0004133876396930119,
"loss": 3.0948,
"step": 46300
},
{
"epoch": 12.473191214470285,
"grad_norm": 0.3701384365558624,
"learning_rate": 0.00041318567389255417,
"loss": 3.1017,
"step": 46350
},
{
"epoch": 12.486649440137812,
"grad_norm": 0.3526800572872162,
"learning_rate": 0.00041298370809209636,
"loss": 3.1008,
"step": 46400
},
{
"epoch": 12.50010766580534,
"grad_norm": 0.3700055778026581,
"learning_rate": 0.00041278174229163856,
"loss": 3.1032,
"step": 46450
},
{
"epoch": 12.513565891472869,
"grad_norm": 0.36047184467315674,
"learning_rate": 0.00041257977649118076,
"loss": 3.1158,
"step": 46500
},
{
"epoch": 12.527024117140396,
"grad_norm": 0.359749436378479,
"learning_rate": 0.00041237781069072295,
"loss": 3.1017,
"step": 46550
},
{
"epoch": 12.540482342807923,
"grad_norm": 0.35695669054985046,
"learning_rate": 0.0004121758448902652,
"loss": 3.0996,
"step": 46600
},
{
"epoch": 12.553940568475452,
"grad_norm": 0.36963951587677,
"learning_rate": 0.0004119738790898074,
"loss": 3.1095,
"step": 46650
},
{
"epoch": 12.56739879414298,
"grad_norm": 0.3756096661090851,
"learning_rate": 0.00041177191328934965,
"loss": 3.1052,
"step": 46700
},
{
"epoch": 12.580857019810509,
"grad_norm": 0.3856138586997986,
"learning_rate": 0.00041156994748889185,
"loss": 3.1089,
"step": 46750
},
{
"epoch": 12.594315245478036,
"grad_norm": 0.3549295663833618,
"learning_rate": 0.0004113679816884341,
"loss": 3.0999,
"step": 46800
},
{
"epoch": 12.607773471145563,
"grad_norm": 0.36051085591316223,
"learning_rate": 0.0004111660158879763,
"loss": 3.1071,
"step": 46850
},
{
"epoch": 12.621231696813092,
"grad_norm": 0.37265828251838684,
"learning_rate": 0.0004109640500875185,
"loss": 3.1097,
"step": 46900
},
{
"epoch": 12.63468992248062,
"grad_norm": 0.3527121841907501,
"learning_rate": 0.0004107620842870607,
"loss": 3.1085,
"step": 46950
},
{
"epoch": 12.648148148148149,
"grad_norm": 0.35205259919166565,
"learning_rate": 0.0004105601184866029,
"loss": 3.1139,
"step": 47000
},
{
"epoch": 12.648148148148149,
"eval_accuracy": 0.39267146538098324,
"eval_loss": 3.2952685356140137,
"eval_runtime": 84.2558,
"eval_samples_per_second": 213.778,
"eval_steps_per_second": 13.364,
"step": 47000
},
{
"epoch": 12.661606373815676,
"grad_norm": 0.36217617988586426,
"learning_rate": 0.00041035815268614513,
"loss": 3.1118,
"step": 47050
},
{
"epoch": 12.675064599483203,
"grad_norm": 0.37881895899772644,
"learning_rate": 0.0004101561868856873,
"loss": 3.1218,
"step": 47100
},
{
"epoch": 12.688522825150732,
"grad_norm": 0.3426401913166046,
"learning_rate": 0.0004099542210852295,
"loss": 3.1114,
"step": 47150
},
{
"epoch": 12.70198105081826,
"grad_norm": 0.34635311365127563,
"learning_rate": 0.0004097522552847717,
"loss": 3.1118,
"step": 47200
},
{
"epoch": 12.715439276485789,
"grad_norm": 0.34759101271629333,
"learning_rate": 0.00040955028948431397,
"loss": 3.1153,
"step": 47250
},
{
"epoch": 12.728897502153316,
"grad_norm": 0.39212140440940857,
"learning_rate": 0.00040934832368385616,
"loss": 3.1197,
"step": 47300
},
{
"epoch": 12.742355727820843,
"grad_norm": 0.34549346566200256,
"learning_rate": 0.00040914635788339836,
"loss": 3.125,
"step": 47350
},
{
"epoch": 12.755813953488373,
"grad_norm": 0.33954575657844543,
"learning_rate": 0.00040894439208294056,
"loss": 3.1183,
"step": 47400
},
{
"epoch": 12.7692721791559,
"grad_norm": 0.34983354806900024,
"learning_rate": 0.00040874242628248275,
"loss": 3.1252,
"step": 47450
},
{
"epoch": 12.782730404823429,
"grad_norm": 0.3618473410606384,
"learning_rate": 0.000408540460482025,
"loss": 3.1039,
"step": 47500
},
{
"epoch": 12.796188630490956,
"grad_norm": 0.3492349088191986,
"learning_rate": 0.00040833849468156725,
"loss": 3.1332,
"step": 47550
},
{
"epoch": 12.809646856158484,
"grad_norm": 0.3821626603603363,
"learning_rate": 0.00040813652888110945,
"loss": 3.1107,
"step": 47600
},
{
"epoch": 12.823105081826013,
"grad_norm": 0.3705442547798157,
"learning_rate": 0.00040793456308065165,
"loss": 3.1254,
"step": 47650
},
{
"epoch": 12.83656330749354,
"grad_norm": 0.3588872253894806,
"learning_rate": 0.0004077325972801939,
"loss": 3.1226,
"step": 47700
},
{
"epoch": 12.850021533161069,
"grad_norm": 0.37265342473983765,
"learning_rate": 0.0004075306314797361,
"loss": 3.1101,
"step": 47750
},
{
"epoch": 12.863479758828596,
"grad_norm": 0.3652234673500061,
"learning_rate": 0.0004073286656792783,
"loss": 3.12,
"step": 47800
},
{
"epoch": 12.876937984496124,
"grad_norm": 0.3789381980895996,
"learning_rate": 0.0004071266998788205,
"loss": 3.1207,
"step": 47850
},
{
"epoch": 12.890396210163653,
"grad_norm": 0.34515896439552307,
"learning_rate": 0.0004069247340783627,
"loss": 3.1286,
"step": 47900
},
{
"epoch": 12.90385443583118,
"grad_norm": 0.37213557958602905,
"learning_rate": 0.00040672276827790493,
"loss": 3.1279,
"step": 47950
},
{
"epoch": 12.917312661498707,
"grad_norm": 0.3628597557544708,
"learning_rate": 0.0004065208024774471,
"loss": 3.1098,
"step": 48000
},
{
"epoch": 12.917312661498707,
"eval_accuracy": 0.39299349466087624,
"eval_loss": 3.292800188064575,
"eval_runtime": 84.4074,
"eval_samples_per_second": 213.394,
"eval_steps_per_second": 13.34,
"step": 48000
},
{
"epoch": 12.930770887166236,
"grad_norm": 0.35086748003959656,
"learning_rate": 0.0004063188366769893,
"loss": 3.1178,
"step": 48050
},
{
"epoch": 12.944229112833764,
"grad_norm": 0.40841004252433777,
"learning_rate": 0.0004061168708765315,
"loss": 3.1226,
"step": 48100
},
{
"epoch": 12.957687338501293,
"grad_norm": 0.3559737205505371,
"learning_rate": 0.0004059149050760737,
"loss": 3.1358,
"step": 48150
},
{
"epoch": 12.97114556416882,
"grad_norm": 0.3919322192668915,
"learning_rate": 0.00040571293927561597,
"loss": 3.1335,
"step": 48200
},
{
"epoch": 12.984603789836347,
"grad_norm": 0.35989293456077576,
"learning_rate": 0.00040551097347515816,
"loss": 3.1246,
"step": 48250
},
{
"epoch": 12.998062015503876,
"grad_norm": 0.3764020800590515,
"learning_rate": 0.00040530900767470036,
"loss": 3.1221,
"step": 48300
},
{
"epoch": 13.011304909560723,
"grad_norm": 0.3399815559387207,
"learning_rate": 0.00040510704187424255,
"loss": 3.0507,
"step": 48350
},
{
"epoch": 13.024763135228252,
"grad_norm": 0.3811633884906769,
"learning_rate": 0.00040490507607378475,
"loss": 3.0216,
"step": 48400
},
{
"epoch": 13.038221360895779,
"grad_norm": 0.35439440608024597,
"learning_rate": 0.00040470311027332705,
"loss": 3.0306,
"step": 48450
},
{
"epoch": 13.051679586563308,
"grad_norm": 0.3569784164428711,
"learning_rate": 0.00040450114447286925,
"loss": 3.0333,
"step": 48500
},
{
"epoch": 13.065137812230835,
"grad_norm": 0.37488240003585815,
"learning_rate": 0.00040429917867241145,
"loss": 3.0394,
"step": 48550
},
{
"epoch": 13.078596037898363,
"grad_norm": 0.37975722551345825,
"learning_rate": 0.00040409721287195364,
"loss": 3.0402,
"step": 48600
},
{
"epoch": 13.092054263565892,
"grad_norm": 0.37718072533607483,
"learning_rate": 0.0004038952470714959,
"loss": 3.0411,
"step": 48650
},
{
"epoch": 13.10551248923342,
"grad_norm": 0.35373955965042114,
"learning_rate": 0.0004036932812710381,
"loss": 3.0521,
"step": 48700
},
{
"epoch": 13.118970714900948,
"grad_norm": 0.39553093910217285,
"learning_rate": 0.0004034913154705803,
"loss": 3.045,
"step": 48750
},
{
"epoch": 13.132428940568476,
"grad_norm": 0.3530154526233673,
"learning_rate": 0.0004032893496701225,
"loss": 3.0471,
"step": 48800
},
{
"epoch": 13.145887166236003,
"grad_norm": 0.39944958686828613,
"learning_rate": 0.0004030873838696647,
"loss": 3.0552,
"step": 48850
},
{
"epoch": 13.159345391903532,
"grad_norm": 0.3472399115562439,
"learning_rate": 0.00040288541806920693,
"loss": 3.0566,
"step": 48900
},
{
"epoch": 13.17280361757106,
"grad_norm": 0.3732159733772278,
"learning_rate": 0.0004026834522687491,
"loss": 3.0581,
"step": 48950
},
{
"epoch": 13.186261843238588,
"grad_norm": 0.3669090270996094,
"learning_rate": 0.0004024814864682913,
"loss": 3.0614,
"step": 49000
},
{
"epoch": 13.186261843238588,
"eval_accuracy": 0.39207771031532357,
"eval_loss": 3.3035924434661865,
"eval_runtime": 84.3763,
"eval_samples_per_second": 213.472,
"eval_steps_per_second": 13.345,
"step": 49000
},
{
"epoch": 13.199720068906116,
"grad_norm": 0.3668637275695801,
"learning_rate": 0.0004022795206678335,
"loss": 3.0574,
"step": 49050
},
{
"epoch": 13.213178294573643,
"grad_norm": 0.3803250193595886,
"learning_rate": 0.00040207755486737577,
"loss": 3.0672,
"step": 49100
},
{
"epoch": 13.226636520241172,
"grad_norm": 0.4007713794708252,
"learning_rate": 0.00040187558906691796,
"loss": 3.0597,
"step": 49150
},
{
"epoch": 13.2400947459087,
"grad_norm": 0.37357866764068604,
"learning_rate": 0.00040167362326646016,
"loss": 3.0649,
"step": 49200
},
{
"epoch": 13.253552971576227,
"grad_norm": 0.3848036825656891,
"learning_rate": 0.00040147165746600235,
"loss": 3.0663,
"step": 49250
},
{
"epoch": 13.267011197243756,
"grad_norm": 0.3878563344478607,
"learning_rate": 0.00040126969166554455,
"loss": 3.068,
"step": 49300
},
{
"epoch": 13.280469422911283,
"grad_norm": 0.3646444082260132,
"learning_rate": 0.00040106772586508686,
"loss": 3.0814,
"step": 49350
},
{
"epoch": 13.293927648578812,
"grad_norm": 0.37095150351524353,
"learning_rate": 0.00040086576006462905,
"loss": 3.0718,
"step": 49400
},
{
"epoch": 13.30738587424634,
"grad_norm": 0.38114747405052185,
"learning_rate": 0.00040066379426417125,
"loss": 3.0783,
"step": 49450
},
{
"epoch": 13.320844099913867,
"grad_norm": 0.38861939311027527,
"learning_rate": 0.00040046182846371344,
"loss": 3.0669,
"step": 49500
},
{
"epoch": 13.334302325581396,
"grad_norm": 0.3983568847179413,
"learning_rate": 0.0004002598626632557,
"loss": 3.0697,
"step": 49550
},
{
"epoch": 13.347760551248923,
"grad_norm": 0.3889234662055969,
"learning_rate": 0.0004000578968627979,
"loss": 3.0612,
"step": 49600
},
{
"epoch": 13.361218776916452,
"grad_norm": 0.35765254497528076,
"learning_rate": 0.0003998559310623401,
"loss": 3.0659,
"step": 49650
},
{
"epoch": 13.37467700258398,
"grad_norm": 0.39406129717826843,
"learning_rate": 0.0003996539652618823,
"loss": 3.077,
"step": 49700
},
{
"epoch": 13.388135228251507,
"grad_norm": 0.40474364161491394,
"learning_rate": 0.0003994519994614245,
"loss": 3.0741,
"step": 49750
},
{
"epoch": 13.401593453919036,
"grad_norm": 0.38321611285209656,
"learning_rate": 0.00039925003366096673,
"loss": 3.0795,
"step": 49800
},
{
"epoch": 13.415051679586563,
"grad_norm": 0.37154221534729004,
"learning_rate": 0.0003990480678605089,
"loss": 3.0756,
"step": 49850
},
{
"epoch": 13.428509905254092,
"grad_norm": 0.38573193550109863,
"learning_rate": 0.0003988461020600511,
"loss": 3.074,
"step": 49900
},
{
"epoch": 13.44196813092162,
"grad_norm": 0.3848041892051697,
"learning_rate": 0.0003986441362595933,
"loss": 3.0801,
"step": 49950
},
{
"epoch": 13.455426356589147,
"grad_norm": 0.35528790950775146,
"learning_rate": 0.0003984421704591355,
"loss": 3.0893,
"step": 50000
},
{
"epoch": 13.455426356589147,
"eval_accuracy": 0.39282367962562903,
"eval_loss": 3.2962677478790283,
"eval_runtime": 84.322,
"eval_samples_per_second": 213.61,
"eval_steps_per_second": 13.354,
"step": 50000
},
{
"epoch": 13.468884582256676,
"grad_norm": 0.3837132453918457,
"learning_rate": 0.00039824020465867776,
"loss": 3.0822,
"step": 50050
},
{
"epoch": 13.482342807924203,
"grad_norm": 0.3933063745498657,
"learning_rate": 0.00039803823885821996,
"loss": 3.0871,
"step": 50100
},
{
"epoch": 13.49580103359173,
"grad_norm": 0.3604951798915863,
"learning_rate": 0.00039783627305776216,
"loss": 3.0849,
"step": 50150
},
{
"epoch": 13.50925925925926,
"grad_norm": 0.3549090027809143,
"learning_rate": 0.00039763430725730435,
"loss": 3.0842,
"step": 50200
},
{
"epoch": 13.522717484926787,
"grad_norm": 0.3664722442626953,
"learning_rate": 0.00039743234145684666,
"loss": 3.099,
"step": 50250
},
{
"epoch": 13.536175710594316,
"grad_norm": 0.35317978262901306,
"learning_rate": 0.00039723037565638885,
"loss": 3.0917,
"step": 50300
},
{
"epoch": 13.549633936261843,
"grad_norm": 0.3661048412322998,
"learning_rate": 0.00039702840985593105,
"loss": 3.0861,
"step": 50350
},
{
"epoch": 13.56309216192937,
"grad_norm": 0.38903483748435974,
"learning_rate": 0.00039682644405547324,
"loss": 3.0975,
"step": 50400
},
{
"epoch": 13.5765503875969,
"grad_norm": 0.3472830057144165,
"learning_rate": 0.00039662447825501544,
"loss": 3.0983,
"step": 50450
},
{
"epoch": 13.590008613264427,
"grad_norm": 0.40977710485458374,
"learning_rate": 0.0003964225124545577,
"loss": 3.0923,
"step": 50500
},
{
"epoch": 13.603466838931956,
"grad_norm": 0.37898826599121094,
"learning_rate": 0.0003962205466540999,
"loss": 3.1029,
"step": 50550
},
{
"epoch": 13.616925064599483,
"grad_norm": 0.36426326632499695,
"learning_rate": 0.0003960185808536421,
"loss": 3.0972,
"step": 50600
},
{
"epoch": 13.63038329026701,
"grad_norm": 0.3560165464878082,
"learning_rate": 0.0003958166150531843,
"loss": 3.1104,
"step": 50650
},
{
"epoch": 13.64384151593454,
"grad_norm": 0.38337618112564087,
"learning_rate": 0.0003956146492527265,
"loss": 3.099,
"step": 50700
},
{
"epoch": 13.657299741602067,
"grad_norm": 0.373165488243103,
"learning_rate": 0.0003954126834522687,
"loss": 3.0936,
"step": 50750
},
{
"epoch": 13.670757967269594,
"grad_norm": 0.35410720109939575,
"learning_rate": 0.0003952107176518109,
"loss": 3.1071,
"step": 50800
},
{
"epoch": 13.684216192937123,
"grad_norm": 0.4100600481033325,
"learning_rate": 0.0003950087518513531,
"loss": 3.0973,
"step": 50850
},
{
"epoch": 13.69767441860465,
"grad_norm": 0.38382619619369507,
"learning_rate": 0.0003948067860508953,
"loss": 3.1021,
"step": 50900
},
{
"epoch": 13.71113264427218,
"grad_norm": 0.36865368485450745,
"learning_rate": 0.0003946048202504375,
"loss": 3.0984,
"step": 50950
},
{
"epoch": 13.724590869939707,
"grad_norm": 0.35743772983551025,
"learning_rate": 0.00039440285444997976,
"loss": 3.0893,
"step": 51000
},
{
"epoch": 13.724590869939707,
"eval_accuracy": 0.39332182545839195,
"eval_loss": 3.289945363998413,
"eval_runtime": 84.2626,
"eval_samples_per_second": 213.76,
"eval_steps_per_second": 13.363,
"step": 51000
},
{
"epoch": 13.738049095607234,
"grad_norm": 0.3683944344520569,
"learning_rate": 0.00039420088864952196,
"loss": 3.1162,
"step": 51050
},
{
"epoch": 13.751507321274763,
"grad_norm": 0.36566293239593506,
"learning_rate": 0.00039399892284906415,
"loss": 3.0974,
"step": 51100
},
{
"epoch": 13.76496554694229,
"grad_norm": 0.34378859400749207,
"learning_rate": 0.00039379695704860646,
"loss": 3.1029,
"step": 51150
},
{
"epoch": 13.77842377260982,
"grad_norm": 0.409297913312912,
"learning_rate": 0.00039359499124814865,
"loss": 3.1031,
"step": 51200
},
{
"epoch": 13.791881998277347,
"grad_norm": 0.3918900191783905,
"learning_rate": 0.00039339302544769085,
"loss": 3.1081,
"step": 51250
},
{
"epoch": 13.805340223944874,
"grad_norm": 0.3867114782333374,
"learning_rate": 0.00039319105964723305,
"loss": 3.1026,
"step": 51300
},
{
"epoch": 13.818798449612403,
"grad_norm": 0.38156697154045105,
"learning_rate": 0.00039298909384677524,
"loss": 3.1067,
"step": 51350
},
{
"epoch": 13.83225667527993,
"grad_norm": 0.381967693567276,
"learning_rate": 0.0003927871280463175,
"loss": 3.1001,
"step": 51400
},
{
"epoch": 13.84571490094746,
"grad_norm": 0.37300676107406616,
"learning_rate": 0.0003925851622458597,
"loss": 3.0924,
"step": 51450
},
{
"epoch": 13.859173126614987,
"grad_norm": 0.36449840664863586,
"learning_rate": 0.0003923831964454019,
"loss": 3.1068,
"step": 51500
},
{
"epoch": 13.872631352282514,
"grad_norm": 0.37150195240974426,
"learning_rate": 0.0003921812306449441,
"loss": 3.0981,
"step": 51550
},
{
"epoch": 13.886089577950044,
"grad_norm": 0.36262959241867065,
"learning_rate": 0.0003919792648444863,
"loss": 3.0958,
"step": 51600
},
{
"epoch": 13.89954780361757,
"grad_norm": 0.37757211923599243,
"learning_rate": 0.0003917772990440285,
"loss": 3.1027,
"step": 51650
},
{
"epoch": 13.9130060292851,
"grad_norm": 0.3373667597770691,
"learning_rate": 0.0003915753332435707,
"loss": 3.1131,
"step": 51700
},
{
"epoch": 13.926464254952627,
"grad_norm": 0.35641902685165405,
"learning_rate": 0.0003913733674431129,
"loss": 3.1163,
"step": 51750
},
{
"epoch": 13.939922480620154,
"grad_norm": 0.35720914602279663,
"learning_rate": 0.0003911714016426551,
"loss": 3.0914,
"step": 51800
},
{
"epoch": 13.953380706287684,
"grad_norm": 0.37049901485443115,
"learning_rate": 0.0003909694358421973,
"loss": 3.1012,
"step": 51850
},
{
"epoch": 13.96683893195521,
"grad_norm": 0.3717890679836273,
"learning_rate": 0.00039076747004173956,
"loss": 3.1198,
"step": 51900
},
{
"epoch": 13.98029715762274,
"grad_norm": 0.3679327070713043,
"learning_rate": 0.00039056550424128176,
"loss": 3.1053,
"step": 51950
},
{
"epoch": 13.993755383290267,
"grad_norm": 0.36129581928253174,
"learning_rate": 0.00039036353844082395,
"loss": 3.108,
"step": 52000
},
{
"epoch": 13.993755383290267,
"eval_accuracy": 0.3937257744673805,
"eval_loss": 3.2866153717041016,
"eval_runtime": 84.429,
"eval_samples_per_second": 213.339,
"eval_steps_per_second": 13.337,
"step": 52000
},
{
"epoch": 14.006998277347115,
"grad_norm": 0.38459154963493347,
"learning_rate": 0.0003901615726403662,
"loss": 3.054,
"step": 52050
},
{
"epoch": 14.020456503014643,
"grad_norm": 0.3999425768852234,
"learning_rate": 0.00038995960683990845,
"loss": 3.007,
"step": 52100
},
{
"epoch": 14.03391472868217,
"grad_norm": 0.35180553793907166,
"learning_rate": 0.00038975764103945065,
"loss": 3.0228,
"step": 52150
},
{
"epoch": 14.047372954349699,
"grad_norm": 0.407040536403656,
"learning_rate": 0.00038955567523899285,
"loss": 3.0114,
"step": 52200
},
{
"epoch": 14.060831180017226,
"grad_norm": 0.36089593172073364,
"learning_rate": 0.00038935370943853504,
"loss": 3.0107,
"step": 52250
},
{
"epoch": 14.074289405684755,
"grad_norm": 0.36350467801094055,
"learning_rate": 0.00038915174363807724,
"loss": 3.0155,
"step": 52300
},
{
"epoch": 14.087747631352283,
"grad_norm": 0.36970725655555725,
"learning_rate": 0.0003889497778376195,
"loss": 3.0151,
"step": 52350
},
{
"epoch": 14.10120585701981,
"grad_norm": 0.38831764459609985,
"learning_rate": 0.0003887478120371617,
"loss": 3.0335,
"step": 52400
},
{
"epoch": 14.114664082687339,
"grad_norm": 0.38094422221183777,
"learning_rate": 0.0003885458462367039,
"loss": 3.031,
"step": 52450
},
{
"epoch": 14.128122308354866,
"grad_norm": 0.3756919205188751,
"learning_rate": 0.0003883438804362461,
"loss": 3.0327,
"step": 52500
},
{
"epoch": 14.141580534022394,
"grad_norm": 0.372146874666214,
"learning_rate": 0.0003881419146357883,
"loss": 3.0398,
"step": 52550
},
{
"epoch": 14.155038759689923,
"grad_norm": 0.36529073119163513,
"learning_rate": 0.0003879399488353305,
"loss": 3.0487,
"step": 52600
},
{
"epoch": 14.16849698535745,
"grad_norm": 0.3786604404449463,
"learning_rate": 0.0003877379830348727,
"loss": 3.0371,
"step": 52650
},
{
"epoch": 14.18195521102498,
"grad_norm": 0.4139573276042938,
"learning_rate": 0.0003875360172344149,
"loss": 3.0559,
"step": 52700
},
{
"epoch": 14.195413436692506,
"grad_norm": 0.35359033942222595,
"learning_rate": 0.0003873340514339571,
"loss": 3.0373,
"step": 52750
},
{
"epoch": 14.208871662360034,
"grad_norm": 0.37699979543685913,
"learning_rate": 0.0003871320856334993,
"loss": 3.0482,
"step": 52800
},
{
"epoch": 14.222329888027563,
"grad_norm": 0.3546140193939209,
"learning_rate": 0.00038693011983304156,
"loss": 3.0478,
"step": 52850
},
{
"epoch": 14.23578811369509,
"grad_norm": 0.3665039837360382,
"learning_rate": 0.0003867281540325838,
"loss": 3.0516,
"step": 52900
},
{
"epoch": 14.24924633936262,
"grad_norm": 0.3570476174354553,
"learning_rate": 0.000386526188232126,
"loss": 3.0523,
"step": 52950
},
{
"epoch": 14.262704565030146,
"grad_norm": 0.40596699714660645,
"learning_rate": 0.00038632422243166825,
"loss": 3.0564,
"step": 53000
},
{
"epoch": 14.262704565030146,
"eval_accuracy": 0.39316233187442334,
"eval_loss": 3.299149513244629,
"eval_runtime": 84.3356,
"eval_samples_per_second": 213.575,
"eval_steps_per_second": 13.351,
"step": 53000
},
{
"epoch": 14.276162790697674,
"grad_norm": 0.36422741413116455,
"learning_rate": 0.00038612225663121045,
"loss": 3.0472,
"step": 53050
},
{
"epoch": 14.289621016365203,
"grad_norm": 0.36780548095703125,
"learning_rate": 0.00038592029083075265,
"loss": 3.0387,
"step": 53100
},
{
"epoch": 14.30307924203273,
"grad_norm": 0.3683931827545166,
"learning_rate": 0.00038571832503029484,
"loss": 3.0483,
"step": 53150
},
{
"epoch": 14.31653746770026,
"grad_norm": 0.38417869806289673,
"learning_rate": 0.00038551635922983704,
"loss": 3.0559,
"step": 53200
},
{
"epoch": 14.329995693367787,
"grad_norm": 0.39943137764930725,
"learning_rate": 0.0003853143934293793,
"loss": 3.0479,
"step": 53250
},
{
"epoch": 14.343453919035314,
"grad_norm": 0.4140488803386688,
"learning_rate": 0.0003851124276289215,
"loss": 3.0612,
"step": 53300
},
{
"epoch": 14.356912144702843,
"grad_norm": 0.3700057864189148,
"learning_rate": 0.0003849104618284637,
"loss": 3.0584,
"step": 53350
},
{
"epoch": 14.37037037037037,
"grad_norm": 0.36353835463523865,
"learning_rate": 0.0003847084960280059,
"loss": 3.0683,
"step": 53400
},
{
"epoch": 14.383828596037898,
"grad_norm": 0.3969934284687042,
"learning_rate": 0.0003845065302275481,
"loss": 3.0688,
"step": 53450
},
{
"epoch": 14.397286821705427,
"grad_norm": 0.38807758688926697,
"learning_rate": 0.0003843045644270903,
"loss": 3.0707,
"step": 53500
},
{
"epoch": 14.410745047372954,
"grad_norm": 0.36600351333618164,
"learning_rate": 0.0003841025986266325,
"loss": 3.08,
"step": 53550
},
{
"epoch": 14.424203273040483,
"grad_norm": 0.36152181029319763,
"learning_rate": 0.0003839006328261747,
"loss": 3.0594,
"step": 53600
},
{
"epoch": 14.43766149870801,
"grad_norm": 0.38738057017326355,
"learning_rate": 0.0003836986670257169,
"loss": 3.0623,
"step": 53650
},
{
"epoch": 14.451119724375538,
"grad_norm": 0.3771466314792633,
"learning_rate": 0.0003834967012252591,
"loss": 3.0713,
"step": 53700
},
{
"epoch": 14.464577950043067,
"grad_norm": 0.3636995851993561,
"learning_rate": 0.00038329473542480136,
"loss": 3.0727,
"step": 53750
},
{
"epoch": 14.478036175710594,
"grad_norm": 0.3603183329105377,
"learning_rate": 0.0003830927696243436,
"loss": 3.0736,
"step": 53800
},
{
"epoch": 14.491494401378123,
"grad_norm": 0.4068652093410492,
"learning_rate": 0.0003828908038238858,
"loss": 3.0775,
"step": 53850
},
{
"epoch": 14.50495262704565,
"grad_norm": 0.3683650493621826,
"learning_rate": 0.000382688838023428,
"loss": 3.0743,
"step": 53900
},
{
"epoch": 14.518410852713178,
"grad_norm": 0.34459036588668823,
"learning_rate": 0.00038248687222297025,
"loss": 3.0744,
"step": 53950
},
{
"epoch": 14.531869078380707,
"grad_norm": 0.37111741304397583,
"learning_rate": 0.00038228490642251245,
"loss": 3.0737,
"step": 54000
},
{
"epoch": 14.531869078380707,
"eval_accuracy": 0.39345415732846944,
"eval_loss": 3.292693853378296,
"eval_runtime": 84.4283,
"eval_samples_per_second": 213.341,
"eval_steps_per_second": 13.337,
"step": 54000
},
{
"epoch": 14.545327304048234,
"grad_norm": 0.3831128478050232,
"learning_rate": 0.00038208294062205464,
"loss": 3.0687,
"step": 54050
},
{
"epoch": 14.558785529715763,
"grad_norm": 0.3837164044380188,
"learning_rate": 0.00038188097482159684,
"loss": 3.0722,
"step": 54100
},
{
"epoch": 14.57224375538329,
"grad_norm": 0.362723171710968,
"learning_rate": 0.00038167900902113904,
"loss": 3.0852,
"step": 54150
},
{
"epoch": 14.585701981050818,
"grad_norm": 0.37301892042160034,
"learning_rate": 0.0003814770432206813,
"loss": 3.0733,
"step": 54200
},
{
"epoch": 14.599160206718347,
"grad_norm": 0.3915317952632904,
"learning_rate": 0.0003812750774202235,
"loss": 3.0799,
"step": 54250
},
{
"epoch": 14.612618432385874,
"grad_norm": 0.3872342109680176,
"learning_rate": 0.0003810731116197657,
"loss": 3.0755,
"step": 54300
},
{
"epoch": 14.626076658053403,
"grad_norm": 0.367057740688324,
"learning_rate": 0.0003808711458193079,
"loss": 3.0825,
"step": 54350
},
{
"epoch": 14.63953488372093,
"grad_norm": 0.42809247970581055,
"learning_rate": 0.00038066918001885007,
"loss": 3.0796,
"step": 54400
},
{
"epoch": 14.652993109388458,
"grad_norm": 0.39945393800735474,
"learning_rate": 0.0003804672142183923,
"loss": 3.0829,
"step": 54450
},
{
"epoch": 14.666451335055987,
"grad_norm": 0.3408666253089905,
"learning_rate": 0.0003802652484179345,
"loss": 3.0862,
"step": 54500
},
{
"epoch": 14.679909560723514,
"grad_norm": 0.3764728307723999,
"learning_rate": 0.0003800632826174767,
"loss": 3.0856,
"step": 54550
},
{
"epoch": 14.693367786391041,
"grad_norm": 0.3803330957889557,
"learning_rate": 0.0003798613168170189,
"loss": 3.0751,
"step": 54600
},
{
"epoch": 14.70682601205857,
"grad_norm": 0.39075732231140137,
"learning_rate": 0.0003796593510165611,
"loss": 3.0867,
"step": 54650
},
{
"epoch": 14.720284237726098,
"grad_norm": 0.3951171636581421,
"learning_rate": 0.0003794573852161034,
"loss": 3.0824,
"step": 54700
},
{
"epoch": 14.733742463393627,
"grad_norm": 0.37119993567466736,
"learning_rate": 0.0003792554194156456,
"loss": 3.0845,
"step": 54750
},
{
"epoch": 14.747200689061154,
"grad_norm": 0.37848755717277527,
"learning_rate": 0.0003790534536151878,
"loss": 3.0861,
"step": 54800
},
{
"epoch": 14.760658914728682,
"grad_norm": 0.4070167541503906,
"learning_rate": 0.00037885148781473005,
"loss": 3.0856,
"step": 54850
},
{
"epoch": 14.77411714039621,
"grad_norm": 0.3541718125343323,
"learning_rate": 0.00037864952201427225,
"loss": 3.0917,
"step": 54900
},
{
"epoch": 14.787575366063738,
"grad_norm": 0.38448378443717957,
"learning_rate": 0.00037844755621381444,
"loss": 3.0865,
"step": 54950
},
{
"epoch": 14.801033591731267,
"grad_norm": 0.36622485518455505,
"learning_rate": 0.00037824559041335664,
"loss": 3.0934,
"step": 55000
},
{
"epoch": 14.801033591731267,
"eval_accuracy": 0.39399760890000274,
"eval_loss": 3.2891037464141846,
"eval_runtime": 84.0862,
"eval_samples_per_second": 214.209,
"eval_steps_per_second": 13.391,
"step": 55000
},
{
"epoch": 14.814491817398794,
"grad_norm": 0.35305047035217285,
"learning_rate": 0.00037804362461289884,
"loss": 3.0932,
"step": 55050
},
{
"epoch": 14.827950043066322,
"grad_norm": 0.3791576325893402,
"learning_rate": 0.0003778416588124411,
"loss": 3.0863,
"step": 55100
},
{
"epoch": 14.84140826873385,
"grad_norm": 0.36797282099723816,
"learning_rate": 0.0003776396930119833,
"loss": 3.0889,
"step": 55150
},
{
"epoch": 14.854866494401378,
"grad_norm": 0.3456723988056183,
"learning_rate": 0.0003774377272115255,
"loss": 3.0958,
"step": 55200
},
{
"epoch": 14.868324720068905,
"grad_norm": 0.38494226336479187,
"learning_rate": 0.0003772357614110677,
"loss": 3.0857,
"step": 55250
},
{
"epoch": 14.881782945736434,
"grad_norm": 0.35462987422943115,
"learning_rate": 0.00037703379561060987,
"loss": 3.0942,
"step": 55300
},
{
"epoch": 14.895241171403962,
"grad_norm": 0.36904287338256836,
"learning_rate": 0.0003768318298101521,
"loss": 3.1013,
"step": 55350
},
{
"epoch": 14.90869939707149,
"grad_norm": 0.36798912286758423,
"learning_rate": 0.0003766298640096943,
"loss": 3.093,
"step": 55400
},
{
"epoch": 14.922157622739018,
"grad_norm": 0.3697493076324463,
"learning_rate": 0.0003764278982092365,
"loss": 3.0977,
"step": 55450
},
{
"epoch": 14.935615848406545,
"grad_norm": 0.39716044068336487,
"learning_rate": 0.0003762259324087787,
"loss": 3.1009,
"step": 55500
},
{
"epoch": 14.949074074074074,
"grad_norm": 0.39862293004989624,
"learning_rate": 0.0003760239666083209,
"loss": 3.0913,
"step": 55550
},
{
"epoch": 14.962532299741602,
"grad_norm": 0.426544725894928,
"learning_rate": 0.0003758220008078632,
"loss": 3.089,
"step": 55600
},
{
"epoch": 14.97599052540913,
"grad_norm": 0.3789792060852051,
"learning_rate": 0.0003756200350074054,
"loss": 3.0964,
"step": 55650
},
{
"epoch": 14.989448751076658,
"grad_norm": 0.3739278316497803,
"learning_rate": 0.0003754180692069476,
"loss": 3.0979,
"step": 55700
},
{
"epoch": 15.002691645133506,
"grad_norm": 0.4127860367298126,
"learning_rate": 0.0003752161034064898,
"loss": 3.0805,
"step": 55750
},
{
"epoch": 15.016149870801033,
"grad_norm": 0.3938711881637573,
"learning_rate": 0.00037501413760603205,
"loss": 2.9973,
"step": 55800
},
{
"epoch": 15.02960809646856,
"grad_norm": 0.3406141400337219,
"learning_rate": 0.00037481217180557425,
"loss": 2.9984,
"step": 55850
},
{
"epoch": 15.04306632213609,
"grad_norm": 0.3878343105316162,
"learning_rate": 0.00037461020600511644,
"loss": 2.9946,
"step": 55900
},
{
"epoch": 15.056524547803617,
"grad_norm": 0.42925557494163513,
"learning_rate": 0.00037440824020465864,
"loss": 3.0021,
"step": 55950
},
{
"epoch": 15.069982773471146,
"grad_norm": 0.36447229981422424,
"learning_rate": 0.00037420627440420083,
"loss": 3.0086,
"step": 56000
},
{
"epoch": 15.069982773471146,
"eval_accuracy": 0.39346197990207005,
"eval_loss": 3.2978291511535645,
"eval_runtime": 84.1272,
"eval_samples_per_second": 214.104,
"eval_steps_per_second": 13.384,
"step": 56000
},
{
"epoch": 15.083440999138674,
"grad_norm": 0.4149514436721802,
"learning_rate": 0.0003740043086037431,
"loss": 3.0051,
"step": 56050
},
{
"epoch": 15.0968992248062,
"grad_norm": 0.4040587246417999,
"learning_rate": 0.0003738023428032853,
"loss": 3.0164,
"step": 56100
},
{
"epoch": 15.11035745047373,
"grad_norm": 0.41578409075737,
"learning_rate": 0.0003736003770028275,
"loss": 3.0111,
"step": 56150
},
{
"epoch": 15.123815676141257,
"grad_norm": 0.4122296869754791,
"learning_rate": 0.00037339841120236967,
"loss": 3.0214,
"step": 56200
},
{
"epoch": 15.137273901808786,
"grad_norm": 0.4029271900653839,
"learning_rate": 0.00037319644540191187,
"loss": 3.0259,
"step": 56250
},
{
"epoch": 15.150732127476314,
"grad_norm": 0.3622616231441498,
"learning_rate": 0.0003729944796014541,
"loss": 3.0213,
"step": 56300
},
{
"epoch": 15.164190353143841,
"grad_norm": 0.39520934224128723,
"learning_rate": 0.0003727925138009963,
"loss": 3.034,
"step": 56350
},
{
"epoch": 15.17764857881137,
"grad_norm": 0.3711996376514435,
"learning_rate": 0.0003725905480005385,
"loss": 3.029,
"step": 56400
},
{
"epoch": 15.191106804478897,
"grad_norm": 0.4158594608306885,
"learning_rate": 0.0003723885822000807,
"loss": 3.033,
"step": 56450
},
{
"epoch": 15.204565030146426,
"grad_norm": 0.4163890480995178,
"learning_rate": 0.000372186616399623,
"loss": 3.023,
"step": 56500
},
{
"epoch": 15.218023255813954,
"grad_norm": 0.4126832187175751,
"learning_rate": 0.0003719846505991652,
"loss": 3.0363,
"step": 56550
},
{
"epoch": 15.231481481481481,
"grad_norm": 0.3895432651042938,
"learning_rate": 0.0003717826847987074,
"loss": 3.0311,
"step": 56600
},
{
"epoch": 15.24493970714901,
"grad_norm": 0.3966125547885895,
"learning_rate": 0.0003715807189982496,
"loss": 3.0431,
"step": 56650
},
{
"epoch": 15.258397932816537,
"grad_norm": 0.3741343021392822,
"learning_rate": 0.00037137875319779185,
"loss": 3.0379,
"step": 56700
},
{
"epoch": 15.271856158484065,
"grad_norm": 0.37849706411361694,
"learning_rate": 0.00037117678739733405,
"loss": 3.0511,
"step": 56750
},
{
"epoch": 15.285314384151594,
"grad_norm": 0.38693898916244507,
"learning_rate": 0.00037097482159687624,
"loss": 3.0524,
"step": 56800
},
{
"epoch": 15.298772609819121,
"grad_norm": 0.3879419267177582,
"learning_rate": 0.00037077285579641844,
"loss": 3.0444,
"step": 56850
},
{
"epoch": 15.31223083548665,
"grad_norm": 0.38886481523513794,
"learning_rate": 0.00037057088999596063,
"loss": 3.0338,
"step": 56900
},
{
"epoch": 15.325689061154177,
"grad_norm": 0.36853650212287903,
"learning_rate": 0.0003703689241955029,
"loss": 3.0466,
"step": 56950
},
{
"epoch": 15.339147286821705,
"grad_norm": 0.383457750082016,
"learning_rate": 0.0003701669583950451,
"loss": 3.047,
"step": 57000
},
{
"epoch": 15.339147286821705,
"eval_accuracy": 0.39383724614118965,
"eval_loss": 3.2929184436798096,
"eval_runtime": 84.2703,
"eval_samples_per_second": 213.741,
"eval_steps_per_second": 13.362,
"step": 57000
},
{
"epoch": 15.352605512489234,
"grad_norm": 0.3808010220527649,
"learning_rate": 0.0003699649925945873,
"loss": 3.0444,
"step": 57050
},
{
"epoch": 15.366063738156761,
"grad_norm": 0.4095914363861084,
"learning_rate": 0.0003697630267941295,
"loss": 3.0514,
"step": 57100
},
{
"epoch": 15.37952196382429,
"grad_norm": 0.3669900894165039,
"learning_rate": 0.00036956106099367167,
"loss": 3.0462,
"step": 57150
},
{
"epoch": 15.392980189491817,
"grad_norm": 0.3716629147529602,
"learning_rate": 0.0003693590951932139,
"loss": 3.0458,
"step": 57200
},
{
"epoch": 15.406438415159345,
"grad_norm": 0.38203051686286926,
"learning_rate": 0.0003691571293927561,
"loss": 3.0479,
"step": 57250
},
{
"epoch": 15.419896640826874,
"grad_norm": 0.37157347798347473,
"learning_rate": 0.0003689551635922983,
"loss": 3.0497,
"step": 57300
},
{
"epoch": 15.433354866494401,
"grad_norm": 0.36887258291244507,
"learning_rate": 0.00036875319779184056,
"loss": 3.0448,
"step": 57350
},
{
"epoch": 15.44681309216193,
"grad_norm": 0.3627953827381134,
"learning_rate": 0.0003685512319913828,
"loss": 3.0498,
"step": 57400
},
{
"epoch": 15.460271317829458,
"grad_norm": 0.3988632559776306,
"learning_rate": 0.000368349266190925,
"loss": 3.0679,
"step": 57450
},
{
"epoch": 15.473729543496985,
"grad_norm": 0.41351136565208435,
"learning_rate": 0.0003681473003904672,
"loss": 3.045,
"step": 57500
},
{
"epoch": 15.487187769164514,
"grad_norm": 0.3687242865562439,
"learning_rate": 0.0003679453345900094,
"loss": 3.0479,
"step": 57550
},
{
"epoch": 15.500645994832041,
"grad_norm": 0.3737011253833771,
"learning_rate": 0.0003677433687895516,
"loss": 3.0599,
"step": 57600
},
{
"epoch": 15.514104220499568,
"grad_norm": 0.4048716127872467,
"learning_rate": 0.00036754140298909385,
"loss": 3.0586,
"step": 57650
},
{
"epoch": 15.527562446167098,
"grad_norm": 0.36465123295783997,
"learning_rate": 0.00036733943718863604,
"loss": 3.0744,
"step": 57700
},
{
"epoch": 15.541020671834625,
"grad_norm": 0.3666316866874695,
"learning_rate": 0.00036713747138817824,
"loss": 3.0563,
"step": 57750
},
{
"epoch": 15.554478897502154,
"grad_norm": 0.37599480152130127,
"learning_rate": 0.00036693550558772044,
"loss": 3.0602,
"step": 57800
},
{
"epoch": 15.567937123169681,
"grad_norm": 0.38387683033943176,
"learning_rate": 0.00036673353978726263,
"loss": 3.0745,
"step": 57850
},
{
"epoch": 15.581395348837209,
"grad_norm": 0.3832012116909027,
"learning_rate": 0.0003665315739868049,
"loss": 3.0711,
"step": 57900
},
{
"epoch": 15.594853574504738,
"grad_norm": 0.3958407938480377,
"learning_rate": 0.0003663296081863471,
"loss": 3.0672,
"step": 57950
},
{
"epoch": 15.608311800172265,
"grad_norm": 0.36599913239479065,
"learning_rate": 0.0003661276423858893,
"loss": 3.063,
"step": 58000
},
{
"epoch": 15.608311800172265,
"eval_accuracy": 0.39374978542246025,
"eval_loss": 3.2895333766937256,
"eval_runtime": 84.3811,
"eval_samples_per_second": 213.46,
"eval_steps_per_second": 13.344,
"step": 58000
},
{
"epoch": 15.621770025839794,
"grad_norm": 0.39657357335090637,
"learning_rate": 0.00036592567658543147,
"loss": 3.0706,
"step": 58050
},
{
"epoch": 15.635228251507321,
"grad_norm": 0.36927181482315063,
"learning_rate": 0.00036572371078497367,
"loss": 3.0651,
"step": 58100
},
{
"epoch": 15.648686477174849,
"grad_norm": 0.38070324063301086,
"learning_rate": 0.0003655217449845159,
"loss": 3.0712,
"step": 58150
},
{
"epoch": 15.662144702842378,
"grad_norm": 0.36729058623313904,
"learning_rate": 0.0003653197791840581,
"loss": 3.0727,
"step": 58200
},
{
"epoch": 15.675602928509905,
"grad_norm": 0.3709213137626648,
"learning_rate": 0.00036511781338360036,
"loss": 3.0636,
"step": 58250
},
{
"epoch": 15.689061154177434,
"grad_norm": 0.34084853529930115,
"learning_rate": 0.00036491584758314256,
"loss": 3.0752,
"step": 58300
},
{
"epoch": 15.702519379844961,
"grad_norm": 0.4009411036968231,
"learning_rate": 0.0003647138817826848,
"loss": 3.0757,
"step": 58350
},
{
"epoch": 15.715977605512489,
"grad_norm": 0.394245445728302,
"learning_rate": 0.000364511915982227,
"loss": 3.0708,
"step": 58400
},
{
"epoch": 15.729435831180018,
"grad_norm": 0.3695777356624603,
"learning_rate": 0.0003643099501817692,
"loss": 3.0635,
"step": 58450
},
{
"epoch": 15.742894056847545,
"grad_norm": 0.3706803023815155,
"learning_rate": 0.0003641079843813114,
"loss": 3.078,
"step": 58500
},
{
"epoch": 15.756352282515074,
"grad_norm": 0.38254979252815247,
"learning_rate": 0.0003639060185808536,
"loss": 3.0782,
"step": 58550
},
{
"epoch": 15.769810508182601,
"grad_norm": 0.38933348655700684,
"learning_rate": 0.00036370405278039584,
"loss": 3.0668,
"step": 58600
},
{
"epoch": 15.783268733850129,
"grad_norm": 0.36985987424850464,
"learning_rate": 0.00036350208697993804,
"loss": 3.0669,
"step": 58650
},
{
"epoch": 15.796726959517658,
"grad_norm": 0.40725845098495483,
"learning_rate": 0.00036330012117948024,
"loss": 3.0812,
"step": 58700
},
{
"epoch": 15.810185185185185,
"grad_norm": 0.3564852476119995,
"learning_rate": 0.00036309815537902243,
"loss": 3.0722,
"step": 58750
},
{
"epoch": 15.823643410852712,
"grad_norm": 0.3941057026386261,
"learning_rate": 0.0003628961895785647,
"loss": 3.0807,
"step": 58800
},
{
"epoch": 15.837101636520241,
"grad_norm": 0.41038039326667786,
"learning_rate": 0.0003626942237781069,
"loss": 3.0783,
"step": 58850
},
{
"epoch": 15.850559862187769,
"grad_norm": 0.3650042712688446,
"learning_rate": 0.0003624922579776491,
"loss": 3.0741,
"step": 58900
},
{
"epoch": 15.864018087855298,
"grad_norm": 0.38325804471969604,
"learning_rate": 0.00036229029217719127,
"loss": 3.0692,
"step": 58950
},
{
"epoch": 15.877476313522825,
"grad_norm": 0.38196754455566406,
"learning_rate": 0.00036208832637673347,
"loss": 3.0813,
"step": 59000
},
{
"epoch": 15.877476313522825,
"eval_accuracy": 0.3946187429732646,
"eval_loss": 3.2819015979766846,
"eval_runtime": 84.302,
"eval_samples_per_second": 213.66,
"eval_steps_per_second": 13.357,
"step": 59000
},
{
"epoch": 15.890934539190352,
"grad_norm": 0.3796331286430359,
"learning_rate": 0.0003618863605762757,
"loss": 3.0757,
"step": 59050
},
{
"epoch": 15.904392764857882,
"grad_norm": 0.38216689229011536,
"learning_rate": 0.0003616843947758179,
"loss": 3.0867,
"step": 59100
},
{
"epoch": 15.917850990525409,
"grad_norm": 0.3886622190475464,
"learning_rate": 0.00036148242897536016,
"loss": 3.0757,
"step": 59150
},
{
"epoch": 15.931309216192938,
"grad_norm": 0.37623468041419983,
"learning_rate": 0.00036128046317490236,
"loss": 3.0783,
"step": 59200
},
{
"epoch": 15.944767441860465,
"grad_norm": 0.36371952295303345,
"learning_rate": 0.0003610784973744446,
"loss": 3.0838,
"step": 59250
},
{
"epoch": 15.958225667527993,
"grad_norm": 0.3721584975719452,
"learning_rate": 0.0003608765315739868,
"loss": 3.0743,
"step": 59300
},
{
"epoch": 15.971683893195522,
"grad_norm": 0.41483673453330994,
"learning_rate": 0.000360674565773529,
"loss": 3.088,
"step": 59350
},
{
"epoch": 15.985142118863049,
"grad_norm": 0.39113107323646545,
"learning_rate": 0.0003604725999730712,
"loss": 3.0794,
"step": 59400
},
{
"epoch": 15.998600344530576,
"grad_norm": 0.3843020796775818,
"learning_rate": 0.0003602706341726134,
"loss": 3.0755,
"step": 59450
},
{
"epoch": 16.011843238587424,
"grad_norm": 0.3941507935523987,
"learning_rate": 0.00036006866837215564,
"loss": 2.9901,
"step": 59500
},
{
"epoch": 16.02530146425495,
"grad_norm": 0.4168238043785095,
"learning_rate": 0.00035986670257169784,
"loss": 2.9901,
"step": 59550
},
{
"epoch": 16.03875968992248,
"grad_norm": 0.4108584225177765,
"learning_rate": 0.00035966473677124004,
"loss": 2.9998,
"step": 59600
},
{
"epoch": 16.05221791559001,
"grad_norm": 0.3931739032268524,
"learning_rate": 0.00035946277097078223,
"loss": 2.9857,
"step": 59650
},
{
"epoch": 16.065676141257537,
"grad_norm": 0.41490697860717773,
"learning_rate": 0.00035926080517032443,
"loss": 2.9893,
"step": 59700
},
{
"epoch": 16.079134366925064,
"grad_norm": 0.3986837565898895,
"learning_rate": 0.0003590588393698667,
"loss": 3.006,
"step": 59750
},
{
"epoch": 16.09259259259259,
"grad_norm": 0.4169183373451233,
"learning_rate": 0.0003588568735694089,
"loss": 3.0026,
"step": 59800
},
{
"epoch": 16.10605081826012,
"grad_norm": 0.3991803228855133,
"learning_rate": 0.00035865490776895107,
"loss": 2.9958,
"step": 59850
},
{
"epoch": 16.11950904392765,
"grad_norm": 0.438391774892807,
"learning_rate": 0.00035845294196849327,
"loss": 3.0077,
"step": 59900
},
{
"epoch": 16.132967269595177,
"grad_norm": 0.3962257504463196,
"learning_rate": 0.00035825097616803546,
"loss": 3.0028,
"step": 59950
},
{
"epoch": 16.146425495262704,
"grad_norm": 0.3833325505256653,
"learning_rate": 0.0003580490103675777,
"loss": 3.0017,
"step": 60000
},
{
"epoch": 16.146425495262704,
"eval_accuracy": 0.3941621002393273,
"eval_loss": 3.294508934020996,
"eval_runtime": 84.2172,
"eval_samples_per_second": 213.876,
"eval_steps_per_second": 13.37,
"step": 60000
},
{
"epoch": 16.15988372093023,
"grad_norm": 0.3867778480052948,
"learning_rate": 0.00035784704456711996,
"loss": 3.0079,
"step": 60050
},
{
"epoch": 16.17334194659776,
"grad_norm": 0.40779435634613037,
"learning_rate": 0.00035764507876666216,
"loss": 3.0106,
"step": 60100
},
{
"epoch": 16.18680017226529,
"grad_norm": 0.40491026639938354,
"learning_rate": 0.00035744311296620436,
"loss": 3.0132,
"step": 60150
},
{
"epoch": 16.200258397932817,
"grad_norm": 0.37903210520744324,
"learning_rate": 0.0003572411471657466,
"loss": 3.0205,
"step": 60200
},
{
"epoch": 16.213716623600344,
"grad_norm": 0.41829851269721985,
"learning_rate": 0.0003570391813652888,
"loss": 3.0283,
"step": 60250
},
{
"epoch": 16.227174849267872,
"grad_norm": 0.3932286500930786,
"learning_rate": 0.000356837215564831,
"loss": 3.0208,
"step": 60300
},
{
"epoch": 16.2406330749354,
"grad_norm": 0.4128912389278412,
"learning_rate": 0.0003566352497643732,
"loss": 3.0246,
"step": 60350
},
{
"epoch": 16.25409130060293,
"grad_norm": 0.3961387872695923,
"learning_rate": 0.0003564332839639154,
"loss": 3.0255,
"step": 60400
},
{
"epoch": 16.267549526270457,
"grad_norm": 0.40615180134773254,
"learning_rate": 0.00035623131816345764,
"loss": 3.0274,
"step": 60450
},
{
"epoch": 16.281007751937985,
"grad_norm": 0.38457584381103516,
"learning_rate": 0.00035602935236299984,
"loss": 3.0314,
"step": 60500
},
{
"epoch": 16.294465977605512,
"grad_norm": 0.4016266167163849,
"learning_rate": 0.00035582738656254203,
"loss": 3.0388,
"step": 60550
},
{
"epoch": 16.30792420327304,
"grad_norm": 0.4087277054786682,
"learning_rate": 0.00035562542076208423,
"loss": 3.0362,
"step": 60600
},
{
"epoch": 16.32138242894057,
"grad_norm": 0.3765519857406616,
"learning_rate": 0.0003554234549616264,
"loss": 3.0337,
"step": 60650
},
{
"epoch": 16.334840654608097,
"grad_norm": 0.3905170261859894,
"learning_rate": 0.0003552214891611687,
"loss": 3.0391,
"step": 60700
},
{
"epoch": 16.348298880275625,
"grad_norm": 0.404747873544693,
"learning_rate": 0.00035501952336071087,
"loss": 3.0303,
"step": 60750
},
{
"epoch": 16.361757105943152,
"grad_norm": 0.41902950406074524,
"learning_rate": 0.00035481755756025307,
"loss": 3.0401,
"step": 60800
},
{
"epoch": 16.37521533161068,
"grad_norm": 0.3708718419075012,
"learning_rate": 0.00035461559175979526,
"loss": 3.0386,
"step": 60850
},
{
"epoch": 16.38867355727821,
"grad_norm": 0.38788917660713196,
"learning_rate": 0.0003544136259593375,
"loss": 3.0364,
"step": 60900
},
{
"epoch": 16.402131782945737,
"grad_norm": 0.4070287048816681,
"learning_rate": 0.00035421166015887977,
"loss": 3.0424,
"step": 60950
},
{
"epoch": 16.415590008613265,
"grad_norm": 0.3842083513736725,
"learning_rate": 0.00035400969435842196,
"loss": 3.038,
"step": 61000
},
{
"epoch": 16.415590008613265,
"eval_accuracy": 0.3939595825005552,
"eval_loss": 3.2929747104644775,
"eval_runtime": 84.1825,
"eval_samples_per_second": 213.964,
"eval_steps_per_second": 13.376,
"step": 61000
},
{
"epoch": 16.429048234280792,
"grad_norm": 0.3588254749774933,
"learning_rate": 0.00035380772855796416,
"loss": 3.0327,
"step": 61050
},
{
"epoch": 16.44250645994832,
"grad_norm": 0.4259098470211029,
"learning_rate": 0.0003536057627575064,
"loss": 3.0472,
"step": 61100
},
{
"epoch": 16.45596468561585,
"grad_norm": 0.38360491394996643,
"learning_rate": 0.0003534037969570486,
"loss": 3.0426,
"step": 61150
},
{
"epoch": 16.469422911283377,
"grad_norm": 0.40665701031684875,
"learning_rate": 0.0003532018311565908,
"loss": 3.0399,
"step": 61200
},
{
"epoch": 16.482881136950905,
"grad_norm": 0.39631187915802,
"learning_rate": 0.000352999865356133,
"loss": 3.0323,
"step": 61250
},
{
"epoch": 16.496339362618432,
"grad_norm": 0.3738632798194885,
"learning_rate": 0.0003527978995556752,
"loss": 3.0373,
"step": 61300
},
{
"epoch": 16.50979758828596,
"grad_norm": 0.4072265028953552,
"learning_rate": 0.00035259593375521744,
"loss": 3.0502,
"step": 61350
},
{
"epoch": 16.52325581395349,
"grad_norm": 0.39553919434547424,
"learning_rate": 0.00035239396795475964,
"loss": 3.0483,
"step": 61400
},
{
"epoch": 16.536714039621017,
"grad_norm": 0.4070907235145569,
"learning_rate": 0.00035219200215430183,
"loss": 3.0439,
"step": 61450
},
{
"epoch": 16.550172265288545,
"grad_norm": 0.4034261405467987,
"learning_rate": 0.00035199003635384403,
"loss": 3.047,
"step": 61500
},
{
"epoch": 16.563630490956072,
"grad_norm": 0.3746528923511505,
"learning_rate": 0.0003517880705533862,
"loss": 3.0452,
"step": 61550
},
{
"epoch": 16.5770887166236,
"grad_norm": 0.37463000416755676,
"learning_rate": 0.0003515861047529285,
"loss": 3.0557,
"step": 61600
},
{
"epoch": 16.590546942291127,
"grad_norm": 0.3937366008758545,
"learning_rate": 0.0003513841389524707,
"loss": 3.0455,
"step": 61650
},
{
"epoch": 16.604005167958658,
"grad_norm": 0.3887689411640167,
"learning_rate": 0.00035118217315201287,
"loss": 3.0628,
"step": 61700
},
{
"epoch": 16.617463393626185,
"grad_norm": 0.3895772695541382,
"learning_rate": 0.00035098020735155507,
"loss": 3.0501,
"step": 61750
},
{
"epoch": 16.630921619293712,
"grad_norm": 0.3983083665370941,
"learning_rate": 0.00035077824155109737,
"loss": 3.0507,
"step": 61800
},
{
"epoch": 16.64437984496124,
"grad_norm": 0.40093982219696045,
"learning_rate": 0.00035057627575063957,
"loss": 3.0563,
"step": 61850
},
{
"epoch": 16.657838070628767,
"grad_norm": 0.38137128949165344,
"learning_rate": 0.00035037430995018176,
"loss": 3.0613,
"step": 61900
},
{
"epoch": 16.671296296296298,
"grad_norm": 0.3852699398994446,
"learning_rate": 0.00035017234414972396,
"loss": 3.0527,
"step": 61950
},
{
"epoch": 16.684754521963825,
"grad_norm": 0.39291974902153015,
"learning_rate": 0.00034997037834926615,
"loss": 3.0502,
"step": 62000
},
{
"epoch": 16.684754521963825,
"eval_accuracy": 0.39450640212461097,
"eval_loss": 3.2875256538391113,
"eval_runtime": 84.1718,
"eval_samples_per_second": 213.991,
"eval_steps_per_second": 13.377,
"step": 62000
},
{
"epoch": 16.698212747631352,
"grad_norm": 0.3874092102050781,
"learning_rate": 0.0003497684125488084,
"loss": 3.061,
"step": 62050
},
{
"epoch": 16.71167097329888,
"grad_norm": 0.40655258297920227,
"learning_rate": 0.0003495664467483506,
"loss": 3.0631,
"step": 62100
},
{
"epoch": 16.725129198966407,
"grad_norm": 0.3778446614742279,
"learning_rate": 0.0003493644809478928,
"loss": 3.0609,
"step": 62150
},
{
"epoch": 16.738587424633938,
"grad_norm": 0.36551788449287415,
"learning_rate": 0.000349162515147435,
"loss": 3.0705,
"step": 62200
},
{
"epoch": 16.752045650301465,
"grad_norm": 0.4111713469028473,
"learning_rate": 0.0003489605493469772,
"loss": 3.0618,
"step": 62250
},
{
"epoch": 16.765503875968992,
"grad_norm": 0.4054068326950073,
"learning_rate": 0.00034875858354651944,
"loss": 3.0539,
"step": 62300
},
{
"epoch": 16.77896210163652,
"grad_norm": 0.39005810022354126,
"learning_rate": 0.00034855661774606164,
"loss": 3.049,
"step": 62350
},
{
"epoch": 16.792420327304047,
"grad_norm": 0.3782937228679657,
"learning_rate": 0.00034835465194560383,
"loss": 3.0594,
"step": 62400
},
{
"epoch": 16.805878552971578,
"grad_norm": 0.3960946798324585,
"learning_rate": 0.00034815268614514603,
"loss": 3.0617,
"step": 62450
},
{
"epoch": 16.819336778639105,
"grad_norm": 0.3699169158935547,
"learning_rate": 0.0003479507203446882,
"loss": 3.0675,
"step": 62500
},
{
"epoch": 16.832795004306632,
"grad_norm": 0.38070520758628845,
"learning_rate": 0.0003477487545442305,
"loss": 3.0642,
"step": 62550
},
{
"epoch": 16.84625322997416,
"grad_norm": 0.3870474100112915,
"learning_rate": 0.00034754678874377267,
"loss": 3.0738,
"step": 62600
},
{
"epoch": 16.859711455641687,
"grad_norm": 0.38762134313583374,
"learning_rate": 0.00034734482294331487,
"loss": 3.0511,
"step": 62650
},
{
"epoch": 16.873169681309218,
"grad_norm": 0.3905826807022095,
"learning_rate": 0.00034714285714285717,
"loss": 3.0606,
"step": 62700
},
{
"epoch": 16.886627906976745,
"grad_norm": 0.3941311538219452,
"learning_rate": 0.00034694089134239937,
"loss": 3.0604,
"step": 62750
},
{
"epoch": 16.900086132644272,
"grad_norm": 0.3685128390789032,
"learning_rate": 0.00034673892554194156,
"loss": 3.0612,
"step": 62800
},
{
"epoch": 16.9135443583118,
"grad_norm": 0.403667688369751,
"learning_rate": 0.00034653695974148376,
"loss": 3.0636,
"step": 62850
},
{
"epoch": 16.927002583979327,
"grad_norm": 0.41439446806907654,
"learning_rate": 0.00034633499394102595,
"loss": 3.076,
"step": 62900
},
{
"epoch": 16.940460809646858,
"grad_norm": 0.4269420802593231,
"learning_rate": 0.0003461330281405682,
"loss": 3.064,
"step": 62950
},
{
"epoch": 16.953919035314385,
"grad_norm": 0.4000409245491028,
"learning_rate": 0.0003459310623401104,
"loss": 3.0795,
"step": 63000
},
{
"epoch": 16.953919035314385,
"eval_accuracy": 0.39524346239276015,
"eval_loss": 3.2783825397491455,
"eval_runtime": 84.6772,
"eval_samples_per_second": 212.714,
"eval_steps_per_second": 13.298,
"step": 63000
},
{
"epoch": 16.967377260981912,
"grad_norm": 0.4015685021877289,
"learning_rate": 0.0003457290965396526,
"loss": 3.0622,
"step": 63050
},
{
"epoch": 16.98083548664944,
"grad_norm": 0.40094172954559326,
"learning_rate": 0.0003455271307391948,
"loss": 3.0641,
"step": 63100
},
{
"epoch": 16.994293712316967,
"grad_norm": 0.40834230184555054,
"learning_rate": 0.000345325164938737,
"loss": 3.0729,
"step": 63150
},
{
"epoch": 17.007536606373815,
"grad_norm": 0.43471765518188477,
"learning_rate": 0.00034512319913827924,
"loss": 3.0287,
"step": 63200
},
{
"epoch": 17.020994832041342,
"grad_norm": 0.40283820033073425,
"learning_rate": 0.00034492123333782144,
"loss": 2.9724,
"step": 63250
},
{
"epoch": 17.034453057708873,
"grad_norm": 0.39246246218681335,
"learning_rate": 0.00034471926753736363,
"loss": 2.9799,
"step": 63300
},
{
"epoch": 17.0479112833764,
"grad_norm": 0.38070741295814514,
"learning_rate": 0.00034451730173690583,
"loss": 2.9707,
"step": 63350
},
{
"epoch": 17.061369509043928,
"grad_norm": 0.3874499797821045,
"learning_rate": 0.000344315335936448,
"loss": 2.9896,
"step": 63400
},
{
"epoch": 17.074827734711455,
"grad_norm": 0.411945641040802,
"learning_rate": 0.0003441133701359903,
"loss": 2.9876,
"step": 63450
},
{
"epoch": 17.088285960378983,
"grad_norm": 0.38966816663742065,
"learning_rate": 0.00034391140433553247,
"loss": 2.9961,
"step": 63500
},
{
"epoch": 17.101744186046513,
"grad_norm": 0.3948839604854584,
"learning_rate": 0.00034370943853507467,
"loss": 2.9908,
"step": 63550
},
{
"epoch": 17.11520241171404,
"grad_norm": 0.44139349460601807,
"learning_rate": 0.0003435074727346169,
"loss": 3.0079,
"step": 63600
},
{
"epoch": 17.128660637381568,
"grad_norm": 0.40904638171195984,
"learning_rate": 0.00034330550693415917,
"loss": 2.9926,
"step": 63650
},
{
"epoch": 17.142118863049095,
"grad_norm": 0.37809765338897705,
"learning_rate": 0.00034310354113370136,
"loss": 2.9871,
"step": 63700
},
{
"epoch": 17.155577088716623,
"grad_norm": 0.4005953371524811,
"learning_rate": 0.00034290157533324356,
"loss": 2.9962,
"step": 63750
},
{
"epoch": 17.16903531438415,
"grad_norm": 0.3982625901699066,
"learning_rate": 0.00034269960953278576,
"loss": 3.0034,
"step": 63800
},
{
"epoch": 17.18249354005168,
"grad_norm": 0.37310752272605896,
"learning_rate": 0.00034249764373232795,
"loss": 2.9922,
"step": 63850
},
{
"epoch": 17.195951765719208,
"grad_norm": 0.35717523097991943,
"learning_rate": 0.0003422956779318702,
"loss": 3.0183,
"step": 63900
},
{
"epoch": 17.209409991386735,
"grad_norm": 0.41922739148139954,
"learning_rate": 0.0003420937121314124,
"loss": 3.0097,
"step": 63950
},
{
"epoch": 17.222868217054263,
"grad_norm": 0.4463370740413666,
"learning_rate": 0.0003418917463309546,
"loss": 3.0119,
"step": 64000
},
{
"epoch": 17.222868217054263,
"eval_accuracy": 0.39430344979841664,
"eval_loss": 3.293164014816284,
"eval_runtime": 84.1962,
"eval_samples_per_second": 213.929,
"eval_steps_per_second": 13.374,
"step": 64000
},
{
"epoch": 17.23632644272179,
"grad_norm": 0.4146665334701538,
"learning_rate": 0.0003416897805304968,
"loss": 3.0098,
"step": 64050
},
{
"epoch": 17.24978466838932,
"grad_norm": 0.39131778478622437,
"learning_rate": 0.000341487814730039,
"loss": 3.0131,
"step": 64100
},
{
"epoch": 17.263242894056848,
"grad_norm": 0.3792233467102051,
"learning_rate": 0.00034128584892958124,
"loss": 3.0138,
"step": 64150
},
{
"epoch": 17.276701119724375,
"grad_norm": 0.38367384672164917,
"learning_rate": 0.00034108388312912343,
"loss": 3.0241,
"step": 64200
},
{
"epoch": 17.290159345391903,
"grad_norm": 0.40969136357307434,
"learning_rate": 0.00034088191732866563,
"loss": 3.0249,
"step": 64250
},
{
"epoch": 17.30361757105943,
"grad_norm": 0.41047418117523193,
"learning_rate": 0.0003406799515282078,
"loss": 3.0121,
"step": 64300
},
{
"epoch": 17.31707579672696,
"grad_norm": 0.4021008610725403,
"learning_rate": 0.00034047798572775,
"loss": 3.0179,
"step": 64350
},
{
"epoch": 17.330534022394488,
"grad_norm": 0.41244468092918396,
"learning_rate": 0.00034027601992729227,
"loss": 3.025,
"step": 64400
},
{
"epoch": 17.343992248062015,
"grad_norm": 0.41508427262306213,
"learning_rate": 0.00034007405412683447,
"loss": 3.0094,
"step": 64450
},
{
"epoch": 17.357450473729543,
"grad_norm": 0.3844899833202362,
"learning_rate": 0.0003398720883263767,
"loss": 3.0283,
"step": 64500
},
{
"epoch": 17.37090869939707,
"grad_norm": 0.3954097628593445,
"learning_rate": 0.00033967012252591897,
"loss": 3.029,
"step": 64550
},
{
"epoch": 17.3843669250646,
"grad_norm": 0.4179207682609558,
"learning_rate": 0.00033946815672546116,
"loss": 3.025,
"step": 64600
},
{
"epoch": 17.397825150732128,
"grad_norm": 0.45622459053993225,
"learning_rate": 0.00033926619092500336,
"loss": 3.0297,
"step": 64650
},
{
"epoch": 17.411283376399656,
"grad_norm": 0.383299857378006,
"learning_rate": 0.00033906422512454556,
"loss": 3.0269,
"step": 64700
},
{
"epoch": 17.424741602067183,
"grad_norm": 0.42966628074645996,
"learning_rate": 0.00033886225932408775,
"loss": 3.0307,
"step": 64750
},
{
"epoch": 17.43819982773471,
"grad_norm": 0.41395410895347595,
"learning_rate": 0.00033866029352363,
"loss": 3.0371,
"step": 64800
},
{
"epoch": 17.45165805340224,
"grad_norm": 0.3621746897697449,
"learning_rate": 0.0003384583277231722,
"loss": 3.0224,
"step": 64850
},
{
"epoch": 17.46511627906977,
"grad_norm": 0.4138607382774353,
"learning_rate": 0.0003382563619227144,
"loss": 3.0178,
"step": 64900
},
{
"epoch": 17.478574504737296,
"grad_norm": 0.39790982007980347,
"learning_rate": 0.0003380543961222566,
"loss": 3.0318,
"step": 64950
},
{
"epoch": 17.492032730404823,
"grad_norm": 0.4042600989341736,
"learning_rate": 0.0003378524303217988,
"loss": 3.0361,
"step": 65000
},
{
"epoch": 17.492032730404823,
"eval_accuracy": 0.39438504358694554,
"eval_loss": 3.2896738052368164,
"eval_runtime": 84.2441,
"eval_samples_per_second": 213.807,
"eval_steps_per_second": 13.366,
"step": 65000
},
{
"epoch": 17.50549095607235,
"grad_norm": 0.4193043112754822,
"learning_rate": 0.00033765046452134104,
"loss": 3.0426,
"step": 65050
},
{
"epoch": 17.51894918173988,
"grad_norm": 0.40196123719215393,
"learning_rate": 0.00033744849872088323,
"loss": 3.0348,
"step": 65100
},
{
"epoch": 17.53240740740741,
"grad_norm": 0.40320277214050293,
"learning_rate": 0.00033724653292042543,
"loss": 3.0389,
"step": 65150
},
{
"epoch": 17.545865633074936,
"grad_norm": 0.3964909613132477,
"learning_rate": 0.0003370445671199676,
"loss": 3.0265,
"step": 65200
},
{
"epoch": 17.559323858742463,
"grad_norm": 0.40705111622810364,
"learning_rate": 0.0003368426013195098,
"loss": 3.0402,
"step": 65250
},
{
"epoch": 17.57278208440999,
"grad_norm": 0.38822999596595764,
"learning_rate": 0.00033664063551905207,
"loss": 3.0294,
"step": 65300
},
{
"epoch": 17.58624031007752,
"grad_norm": 0.3868526518344879,
"learning_rate": 0.00033643866971859427,
"loss": 3.0417,
"step": 65350
},
{
"epoch": 17.59969853574505,
"grad_norm": 0.3836658000946045,
"learning_rate": 0.0003362367039181365,
"loss": 3.0319,
"step": 65400
},
{
"epoch": 17.613156761412576,
"grad_norm": 0.39039117097854614,
"learning_rate": 0.0003360347381176787,
"loss": 3.0439,
"step": 65450
},
{
"epoch": 17.626614987080103,
"grad_norm": 0.3786974251270294,
"learning_rate": 0.00033583277231722097,
"loss": 3.0384,
"step": 65500
},
{
"epoch": 17.64007321274763,
"grad_norm": 0.38470643758773804,
"learning_rate": 0.00033563080651676316,
"loss": 3.0402,
"step": 65550
},
{
"epoch": 17.653531438415158,
"grad_norm": 0.3940637707710266,
"learning_rate": 0.00033542884071630536,
"loss": 3.0493,
"step": 65600
},
{
"epoch": 17.66698966408269,
"grad_norm": 0.3953503370285034,
"learning_rate": 0.00033522687491584755,
"loss": 3.0447,
"step": 65650
},
{
"epoch": 17.680447889750216,
"grad_norm": 0.38392022252082825,
"learning_rate": 0.00033502490911538975,
"loss": 3.0463,
"step": 65700
},
{
"epoch": 17.693906115417743,
"grad_norm": 0.41958823800086975,
"learning_rate": 0.000334822943314932,
"loss": 3.0397,
"step": 65750
},
{
"epoch": 17.70736434108527,
"grad_norm": 0.40301191806793213,
"learning_rate": 0.0003346209775144742,
"loss": 3.057,
"step": 65800
},
{
"epoch": 17.720822566752798,
"grad_norm": 0.4074375033378601,
"learning_rate": 0.0003344190117140164,
"loss": 3.0343,
"step": 65850
},
{
"epoch": 17.73428079242033,
"grad_norm": 0.4151737689971924,
"learning_rate": 0.0003342170459135586,
"loss": 3.0405,
"step": 65900
},
{
"epoch": 17.747739018087856,
"grad_norm": 0.43711110949516296,
"learning_rate": 0.0003340150801131008,
"loss": 3.0309,
"step": 65950
},
{
"epoch": 17.761197243755383,
"grad_norm": 0.3818199634552002,
"learning_rate": 0.00033381311431264303,
"loss": 3.0541,
"step": 66000
},
{
"epoch": 17.761197243755383,
"eval_accuracy": 0.39482212988688126,
"eval_loss": 3.2832953929901123,
"eval_runtime": 84.4291,
"eval_samples_per_second": 213.339,
"eval_steps_per_second": 13.337,
"step": 66000
},
{
"epoch": 17.77465546942291,
"grad_norm": 0.3675343990325928,
"learning_rate": 0.00033361114851218523,
"loss": 3.0484,
"step": 66050
},
{
"epoch": 17.788113695090438,
"grad_norm": 0.40552109479904175,
"learning_rate": 0.0003334091827117274,
"loss": 3.0509,
"step": 66100
},
{
"epoch": 17.80157192075797,
"grad_norm": 0.38828209042549133,
"learning_rate": 0.0003332072169112696,
"loss": 3.0509,
"step": 66150
},
{
"epoch": 17.815030146425496,
"grad_norm": 0.40719807147979736,
"learning_rate": 0.0003330052511108118,
"loss": 3.0486,
"step": 66200
},
{
"epoch": 17.828488372093023,
"grad_norm": 0.4311251938343048,
"learning_rate": 0.0003328032853103541,
"loss": 3.05,
"step": 66250
},
{
"epoch": 17.84194659776055,
"grad_norm": 0.39757847785949707,
"learning_rate": 0.0003326013195098963,
"loss": 3.045,
"step": 66300
},
{
"epoch": 17.855404823428078,
"grad_norm": 0.38673529028892517,
"learning_rate": 0.0003323993537094385,
"loss": 3.0503,
"step": 66350
},
{
"epoch": 17.86886304909561,
"grad_norm": 0.4126622676849365,
"learning_rate": 0.00033219738790898077,
"loss": 3.0606,
"step": 66400
},
{
"epoch": 17.882321274763136,
"grad_norm": 0.41488194465637207,
"learning_rate": 0.00033199542210852296,
"loss": 3.0484,
"step": 66450
},
{
"epoch": 17.895779500430663,
"grad_norm": 0.39812037348747253,
"learning_rate": 0.00033179345630806516,
"loss": 3.0533,
"step": 66500
},
{
"epoch": 17.90923772609819,
"grad_norm": 0.40347662568092346,
"learning_rate": 0.00033159149050760735,
"loss": 3.0444,
"step": 66550
},
{
"epoch": 17.922695951765718,
"grad_norm": 0.38835588097572327,
"learning_rate": 0.00033138952470714955,
"loss": 3.0562,
"step": 66600
},
{
"epoch": 17.93615417743325,
"grad_norm": 0.38559824228286743,
"learning_rate": 0.0003311875589066918,
"loss": 3.0597,
"step": 66650
},
{
"epoch": 17.949612403100776,
"grad_norm": 0.39280909299850464,
"learning_rate": 0.000330985593106234,
"loss": 3.0506,
"step": 66700
},
{
"epoch": 17.963070628768303,
"grad_norm": 0.4078952968120575,
"learning_rate": 0.0003307836273057762,
"loss": 3.0493,
"step": 66750
},
{
"epoch": 17.97652885443583,
"grad_norm": 0.3855295479297638,
"learning_rate": 0.0003305816615053184,
"loss": 3.0515,
"step": 66800
},
{
"epoch": 17.989987080103358,
"grad_norm": 0.3947485387325287,
"learning_rate": 0.0003303796957048606,
"loss": 3.0457,
"step": 66850
},
{
"epoch": 18.003229974160206,
"grad_norm": 0.3720707297325134,
"learning_rate": 0.00033017772990440284,
"loss": 3.0407,
"step": 66900
},
{
"epoch": 18.016688199827733,
"grad_norm": 0.39281925559043884,
"learning_rate": 0.00032997576410394503,
"loss": 2.9672,
"step": 66950
},
{
"epoch": 18.030146425495264,
"grad_norm": 0.40296903252601624,
"learning_rate": 0.00032977379830348723,
"loss": 2.9625,
"step": 67000
},
{
"epoch": 18.030146425495264,
"eval_accuracy": 0.3947613962946207,
"eval_loss": 3.2923240661621094,
"eval_runtime": 84.3224,
"eval_samples_per_second": 213.609,
"eval_steps_per_second": 13.354,
"step": 67000
},
{
"epoch": 18.04360465116279,
"grad_norm": 0.39170727133750916,
"learning_rate": 0.0003295718325030294,
"loss": 2.9558,
"step": 67050
},
{
"epoch": 18.05706287683032,
"grad_norm": 0.42225125432014465,
"learning_rate": 0.0003293698667025716,
"loss": 2.9637,
"step": 67100
},
{
"epoch": 18.070521102497846,
"grad_norm": 0.38866063952445984,
"learning_rate": 0.0003291679009021139,
"loss": 2.979,
"step": 67150
},
{
"epoch": 18.083979328165373,
"grad_norm": 0.4313035309314728,
"learning_rate": 0.0003289659351016561,
"loss": 2.9725,
"step": 67200
},
{
"epoch": 18.097437553832904,
"grad_norm": 0.43296217918395996,
"learning_rate": 0.0003287639693011983,
"loss": 2.9914,
"step": 67250
},
{
"epoch": 18.11089577950043,
"grad_norm": 0.39378491044044495,
"learning_rate": 0.0003285620035007405,
"loss": 2.9781,
"step": 67300
},
{
"epoch": 18.12435400516796,
"grad_norm": 0.4113011062145233,
"learning_rate": 0.00032836003770028276,
"loss": 2.9787,
"step": 67350
},
{
"epoch": 18.137812230835486,
"grad_norm": 0.4032084345817566,
"learning_rate": 0.00032815807189982496,
"loss": 2.9797,
"step": 67400
},
{
"epoch": 18.151270456503013,
"grad_norm": 0.405706524848938,
"learning_rate": 0.00032795610609936715,
"loss": 2.9983,
"step": 67450
},
{
"epoch": 18.164728682170544,
"grad_norm": 0.40440014004707336,
"learning_rate": 0.00032775414029890935,
"loss": 2.9931,
"step": 67500
},
{
"epoch": 18.17818690783807,
"grad_norm": 0.4321618378162384,
"learning_rate": 0.00032755217449845155,
"loss": 2.9937,
"step": 67550
},
{
"epoch": 18.1916451335056,
"grad_norm": 0.42441773414611816,
"learning_rate": 0.0003273502086979938,
"loss": 2.99,
"step": 67600
},
{
"epoch": 18.205103359173126,
"grad_norm": 0.38702791929244995,
"learning_rate": 0.000327148242897536,
"loss": 2.991,
"step": 67650
},
{
"epoch": 18.218561584840653,
"grad_norm": 0.4204156696796417,
"learning_rate": 0.0003269462770970782,
"loss": 2.9891,
"step": 67700
},
{
"epoch": 18.232019810508184,
"grad_norm": 0.4095945656299591,
"learning_rate": 0.0003267443112966204,
"loss": 3.0104,
"step": 67750
},
{
"epoch": 18.24547803617571,
"grad_norm": 0.395253449678421,
"learning_rate": 0.0003265423454961626,
"loss": 3.0061,
"step": 67800
},
{
"epoch": 18.25893626184324,
"grad_norm": 0.40738797187805176,
"learning_rate": 0.00032634037969570483,
"loss": 2.9976,
"step": 67850
},
{
"epoch": 18.272394487510766,
"grad_norm": 0.4268544912338257,
"learning_rate": 0.00032613841389524703,
"loss": 3.0001,
"step": 67900
},
{
"epoch": 18.285852713178294,
"grad_norm": 0.40578269958496094,
"learning_rate": 0.0003259364480947892,
"loss": 3.0027,
"step": 67950
},
{
"epoch": 18.29931093884582,
"grad_norm": 0.3866397440433502,
"learning_rate": 0.0003257344822943314,
"loss": 3.0122,
"step": 68000
},
{
"epoch": 18.29931093884582,
"eval_accuracy": 0.3947345605212963,
"eval_loss": 3.292531967163086,
"eval_runtime": 84.2705,
"eval_samples_per_second": 213.74,
"eval_steps_per_second": 13.362,
"step": 68000
},
{
"epoch": 18.31276916451335,
"grad_norm": 0.434059739112854,
"learning_rate": 0.0003255325164938737,
"loss": 3.0145,
"step": 68050
},
{
"epoch": 18.32622739018088,
"grad_norm": 0.40752777457237244,
"learning_rate": 0.0003253305506934159,
"loss": 3.0132,
"step": 68100
},
{
"epoch": 18.339685615848406,
"grad_norm": 0.3976019620895386,
"learning_rate": 0.0003251285848929581,
"loss": 3.0111,
"step": 68150
},
{
"epoch": 18.353143841515934,
"grad_norm": 0.4011041820049286,
"learning_rate": 0.0003249266190925003,
"loss": 3.0114,
"step": 68200
},
{
"epoch": 18.36660206718346,
"grad_norm": 0.42141130566596985,
"learning_rate": 0.0003247246532920425,
"loss": 3.0157,
"step": 68250
},
{
"epoch": 18.38006029285099,
"grad_norm": 0.44433796405792236,
"learning_rate": 0.00032452268749158476,
"loss": 3.0075,
"step": 68300
},
{
"epoch": 18.39351851851852,
"grad_norm": 0.39202117919921875,
"learning_rate": 0.00032432072169112696,
"loss": 3.0201,
"step": 68350
},
{
"epoch": 18.406976744186046,
"grad_norm": 0.41736364364624023,
"learning_rate": 0.00032411875589066915,
"loss": 3.0086,
"step": 68400
},
{
"epoch": 18.420434969853574,
"grad_norm": 0.39685291051864624,
"learning_rate": 0.00032391679009021135,
"loss": 3.016,
"step": 68450
},
{
"epoch": 18.4338931955211,
"grad_norm": 0.43723633885383606,
"learning_rate": 0.0003237148242897536,
"loss": 3.0146,
"step": 68500
},
{
"epoch": 18.447351421188632,
"grad_norm": 0.42096295952796936,
"learning_rate": 0.0003235128584892958,
"loss": 3.0179,
"step": 68550
},
{
"epoch": 18.46080964685616,
"grad_norm": 0.41369014978408813,
"learning_rate": 0.000323310892688838,
"loss": 3.0155,
"step": 68600
},
{
"epoch": 18.474267872523686,
"grad_norm": 0.41110730171203613,
"learning_rate": 0.0003231089268883802,
"loss": 3.015,
"step": 68650
},
{
"epoch": 18.487726098191214,
"grad_norm": 0.39935365319252014,
"learning_rate": 0.0003229069610879224,
"loss": 3.0315,
"step": 68700
},
{
"epoch": 18.50118432385874,
"grad_norm": 0.3989832401275635,
"learning_rate": 0.00032270499528746463,
"loss": 3.0217,
"step": 68750
},
{
"epoch": 18.514642549526272,
"grad_norm": 0.4078866243362427,
"learning_rate": 0.00032250302948700683,
"loss": 3.0264,
"step": 68800
},
{
"epoch": 18.5281007751938,
"grad_norm": 0.4013637900352478,
"learning_rate": 0.000322301063686549,
"loss": 3.0267,
"step": 68850
},
{
"epoch": 18.541559000861326,
"grad_norm": 0.38905027508735657,
"learning_rate": 0.0003220990978860912,
"loss": 3.0212,
"step": 68900
},
{
"epoch": 18.555017226528854,
"grad_norm": 0.4122450351715088,
"learning_rate": 0.0003218971320856335,
"loss": 3.0263,
"step": 68950
},
{
"epoch": 18.56847545219638,
"grad_norm": 0.40349528193473816,
"learning_rate": 0.0003216951662851757,
"loss": 3.0234,
"step": 69000
},
{
"epoch": 18.56847545219638,
"eval_accuracy": 0.39492480116538964,
"eval_loss": 3.2897045612335205,
"eval_runtime": 84.2999,
"eval_samples_per_second": 213.666,
"eval_steps_per_second": 13.357,
"step": 69000
},
{
"epoch": 18.581933677863912,
"grad_norm": 0.3932398557662964,
"learning_rate": 0.0003214932004847179,
"loss": 3.0295,
"step": 69050
},
{
"epoch": 18.59539190353144,
"grad_norm": 0.4231753349304199,
"learning_rate": 0.0003212912346842601,
"loss": 3.022,
"step": 69100
},
{
"epoch": 18.608850129198967,
"grad_norm": 0.41071921586990356,
"learning_rate": 0.0003210892688838023,
"loss": 3.0148,
"step": 69150
},
{
"epoch": 18.622308354866494,
"grad_norm": 0.4104444980621338,
"learning_rate": 0.00032088730308334456,
"loss": 3.0233,
"step": 69200
},
{
"epoch": 18.63576658053402,
"grad_norm": 0.4138637185096741,
"learning_rate": 0.00032068533728288676,
"loss": 3.0323,
"step": 69250
},
{
"epoch": 18.649224806201552,
"grad_norm": 0.3882623016834259,
"learning_rate": 0.00032048337148242895,
"loss": 3.0267,
"step": 69300
},
{
"epoch": 18.66268303186908,
"grad_norm": 0.41541221737861633,
"learning_rate": 0.00032028140568197115,
"loss": 3.0318,
"step": 69350
},
{
"epoch": 18.676141257536607,
"grad_norm": 0.4507473111152649,
"learning_rate": 0.00032007943988151334,
"loss": 3.0323,
"step": 69400
},
{
"epoch": 18.689599483204134,
"grad_norm": 0.4193718433380127,
"learning_rate": 0.0003198774740810556,
"loss": 3.0344,
"step": 69450
},
{
"epoch": 18.70305770887166,
"grad_norm": 0.42218124866485596,
"learning_rate": 0.0003196755082805978,
"loss": 3.0344,
"step": 69500
},
{
"epoch": 18.716515934539192,
"grad_norm": 0.3834584653377533,
"learning_rate": 0.00031947354248014,
"loss": 3.0258,
"step": 69550
},
{
"epoch": 18.72997416020672,
"grad_norm": 0.410442978143692,
"learning_rate": 0.0003192715766796822,
"loss": 3.0256,
"step": 69600
},
{
"epoch": 18.743432385874247,
"grad_norm": 0.42289379239082336,
"learning_rate": 0.0003190696108792244,
"loss": 3.0309,
"step": 69650
},
{
"epoch": 18.756890611541774,
"grad_norm": 0.4159040153026581,
"learning_rate": 0.00031886764507876663,
"loss": 3.0447,
"step": 69700
},
{
"epoch": 18.7703488372093,
"grad_norm": 0.39476460218429565,
"learning_rate": 0.0003186656792783088,
"loss": 3.0321,
"step": 69750
},
{
"epoch": 18.783807062876832,
"grad_norm": 0.4341265559196472,
"learning_rate": 0.000318463713477851,
"loss": 3.0538,
"step": 69800
},
{
"epoch": 18.79726528854436,
"grad_norm": 0.3935094475746155,
"learning_rate": 0.00031826174767739327,
"loss": 3.0307,
"step": 69850
},
{
"epoch": 18.810723514211887,
"grad_norm": 0.41315150260925293,
"learning_rate": 0.0003180597818769355,
"loss": 3.0372,
"step": 69900
},
{
"epoch": 18.824181739879414,
"grad_norm": 0.43439584970474243,
"learning_rate": 0.0003178578160764777,
"loss": 3.0453,
"step": 69950
},
{
"epoch": 18.83763996554694,
"grad_norm": 0.3953023850917816,
"learning_rate": 0.0003176558502760199,
"loss": 3.0303,
"step": 70000
},
{
"epoch": 18.83763996554694,
"eval_accuracy": 0.3956587106747274,
"eval_loss": 3.2794418334960938,
"eval_runtime": 84.2367,
"eval_samples_per_second": 213.826,
"eval_steps_per_second": 13.367,
"step": 70000
},
{
"epoch": 18.85109819121447,
"grad_norm": 0.3870137631893158,
"learning_rate": 0.0003174538844755621,
"loss": 3.0376,
"step": 70050
},
{
"epoch": 18.864556416882,
"grad_norm": 0.4227631688117981,
"learning_rate": 0.0003172519186751043,
"loss": 3.041,
"step": 70100
},
{
"epoch": 18.878014642549527,
"grad_norm": 0.3939836919307709,
"learning_rate": 0.00031704995287464656,
"loss": 3.0363,
"step": 70150
},
{
"epoch": 18.891472868217054,
"grad_norm": 0.40550845861434937,
"learning_rate": 0.00031684798707418875,
"loss": 3.0482,
"step": 70200
},
{
"epoch": 18.90493109388458,
"grad_norm": 0.3825607895851135,
"learning_rate": 0.00031664602127373095,
"loss": 3.0388,
"step": 70250
},
{
"epoch": 18.91838931955211,
"grad_norm": 0.39878538250923157,
"learning_rate": 0.00031644405547327315,
"loss": 3.0499,
"step": 70300
},
{
"epoch": 18.93184754521964,
"grad_norm": 0.42560845613479614,
"learning_rate": 0.00031624208967281534,
"loss": 3.0404,
"step": 70350
},
{
"epoch": 18.945305770887167,
"grad_norm": 0.39959248900413513,
"learning_rate": 0.0003160401238723576,
"loss": 3.0438,
"step": 70400
},
{
"epoch": 18.958763996554694,
"grad_norm": 0.3780984580516815,
"learning_rate": 0.0003158381580718998,
"loss": 3.0429,
"step": 70450
},
{
"epoch": 18.97222222222222,
"grad_norm": 0.4092783033847809,
"learning_rate": 0.000315636192271442,
"loss": 3.053,
"step": 70500
},
{
"epoch": 18.98568044788975,
"grad_norm": 0.3804001808166504,
"learning_rate": 0.0003154342264709842,
"loss": 3.0393,
"step": 70550
},
{
"epoch": 18.99913867355728,
"grad_norm": 0.44005870819091797,
"learning_rate": 0.00031523226067052643,
"loss": 3.0403,
"step": 70600
},
{
"epoch": 19.012381567614124,
"grad_norm": 0.39213648438453674,
"learning_rate": 0.0003150302948700686,
"loss": 2.9572,
"step": 70650
},
{
"epoch": 19.025839793281655,
"grad_norm": 0.40990149974823,
"learning_rate": 0.0003148283290696108,
"loss": 2.9514,
"step": 70700
},
{
"epoch": 19.039298018949182,
"grad_norm": 0.43813619017601013,
"learning_rate": 0.0003146263632691531,
"loss": 2.956,
"step": 70750
},
{
"epoch": 19.05275624461671,
"grad_norm": 0.4364962577819824,
"learning_rate": 0.0003144243974686953,
"loss": 2.9542,
"step": 70800
},
{
"epoch": 19.066214470284237,
"grad_norm": 0.44929248094558716,
"learning_rate": 0.0003142224316682375,
"loss": 2.9621,
"step": 70850
},
{
"epoch": 19.079672695951764,
"grad_norm": 0.4099480211734772,
"learning_rate": 0.0003140204658677797,
"loss": 2.9712,
"step": 70900
},
{
"epoch": 19.093130921619295,
"grad_norm": 0.393709659576416,
"learning_rate": 0.0003138185000673219,
"loss": 2.9694,
"step": 70950
},
{
"epoch": 19.106589147286822,
"grad_norm": 0.40786850452423096,
"learning_rate": 0.0003136165342668641,
"loss": 2.9725,
"step": 71000
},
{
"epoch": 19.106589147286822,
"eval_accuracy": 0.3949505504701584,
"eval_loss": 3.2920520305633545,
"eval_runtime": 84.1457,
"eval_samples_per_second": 214.057,
"eval_steps_per_second": 13.382,
"step": 71000
},
{
"epoch": 19.12004737295435,
"grad_norm": 0.4190482199192047,
"learning_rate": 0.00031341456846640636,
"loss": 2.9667,
"step": 71050
},
{
"epoch": 19.133505598621877,
"grad_norm": 0.4100850820541382,
"learning_rate": 0.00031321260266594855,
"loss": 2.9747,
"step": 71100
},
{
"epoch": 19.146963824289404,
"grad_norm": 0.4308258295059204,
"learning_rate": 0.00031301063686549075,
"loss": 2.9763,
"step": 71150
},
{
"epoch": 19.160422049956935,
"grad_norm": 0.41372618079185486,
"learning_rate": 0.00031280867106503295,
"loss": 2.9849,
"step": 71200
},
{
"epoch": 19.173880275624462,
"grad_norm": 0.40586134791374207,
"learning_rate": 0.00031260670526457514,
"loss": 2.9846,
"step": 71250
},
{
"epoch": 19.18733850129199,
"grad_norm": 0.41523006558418274,
"learning_rate": 0.0003124047394641174,
"loss": 2.9732,
"step": 71300
},
{
"epoch": 19.200796726959517,
"grad_norm": 0.4177076518535614,
"learning_rate": 0.0003122027736636596,
"loss": 2.9882,
"step": 71350
},
{
"epoch": 19.214254952627044,
"grad_norm": 0.4282616674900055,
"learning_rate": 0.0003120008078632018,
"loss": 2.9876,
"step": 71400
},
{
"epoch": 19.227713178294575,
"grad_norm": 0.40223127603530884,
"learning_rate": 0.000311798842062744,
"loss": 2.993,
"step": 71450
},
{
"epoch": 19.241171403962102,
"grad_norm": 0.4155823886394501,
"learning_rate": 0.0003115968762622862,
"loss": 2.9904,
"step": 71500
},
{
"epoch": 19.25462962962963,
"grad_norm": 0.4424571096897125,
"learning_rate": 0.00031139491046182843,
"loss": 2.9923,
"step": 71550
},
{
"epoch": 19.268087855297157,
"grad_norm": 0.42144522070884705,
"learning_rate": 0.0003111929446613707,
"loss": 2.9912,
"step": 71600
},
{
"epoch": 19.281546080964684,
"grad_norm": 0.4306216835975647,
"learning_rate": 0.0003109909788609129,
"loss": 2.9929,
"step": 71650
},
{
"epoch": 19.295004306632215,
"grad_norm": 0.43049681186676025,
"learning_rate": 0.00031078901306045507,
"loss": 3.0012,
"step": 71700
},
{
"epoch": 19.308462532299743,
"grad_norm": 0.4196663498878479,
"learning_rate": 0.0003105870472599973,
"loss": 2.9855,
"step": 71750
},
{
"epoch": 19.32192075796727,
"grad_norm": 0.4187632203102112,
"learning_rate": 0.0003103850814595395,
"loss": 3.0017,
"step": 71800
},
{
"epoch": 19.335378983634797,
"grad_norm": 0.3931756615638733,
"learning_rate": 0.0003101831156590817,
"loss": 3.0001,
"step": 71850
},
{
"epoch": 19.348837209302324,
"grad_norm": 0.4297729432582855,
"learning_rate": 0.0003099811498586239,
"loss": 3.0043,
"step": 71900
},
{
"epoch": 19.362295434969855,
"grad_norm": 0.4008636772632599,
"learning_rate": 0.0003097791840581661,
"loss": 3.0034,
"step": 71950
},
{
"epoch": 19.375753660637383,
"grad_norm": 0.4368478059768677,
"learning_rate": 0.00030957721825770836,
"loss": 3.0149,
"step": 72000
},
{
"epoch": 19.375753660637383,
"eval_accuracy": 0.39520391493733464,
"eval_loss": 3.289865493774414,
"eval_runtime": 84.3044,
"eval_samples_per_second": 213.654,
"eval_steps_per_second": 13.356,
"step": 72000
},
{
"epoch": 19.38921188630491,
"grad_norm": 0.4222554564476013,
"learning_rate": 0.00030937525245725055,
"loss": 3.004,
"step": 72050
},
{
"epoch": 19.402670111972437,
"grad_norm": 0.4352582097053528,
"learning_rate": 0.00030917328665679275,
"loss": 3.0041,
"step": 72100
},
{
"epoch": 19.416128337639964,
"grad_norm": 0.41281789541244507,
"learning_rate": 0.00030897132085633494,
"loss": 3.001,
"step": 72150
},
{
"epoch": 19.429586563307495,
"grad_norm": 0.4322177469730377,
"learning_rate": 0.00030876935505587714,
"loss": 3.0007,
"step": 72200
},
{
"epoch": 19.443044788975023,
"grad_norm": 0.4109971225261688,
"learning_rate": 0.0003085673892554194,
"loss": 2.9968,
"step": 72250
},
{
"epoch": 19.45650301464255,
"grad_norm": 0.4037865400314331,
"learning_rate": 0.0003083654234549616,
"loss": 3.0081,
"step": 72300
},
{
"epoch": 19.469961240310077,
"grad_norm": 0.41936466097831726,
"learning_rate": 0.0003081634576545038,
"loss": 3.0122,
"step": 72350
},
{
"epoch": 19.483419465977605,
"grad_norm": 0.41000697016716003,
"learning_rate": 0.000307961491854046,
"loss": 3.0012,
"step": 72400
},
{
"epoch": 19.496877691645132,
"grad_norm": 0.40724891424179077,
"learning_rate": 0.00030775952605358823,
"loss": 3.0076,
"step": 72450
},
{
"epoch": 19.510335917312663,
"grad_norm": 0.4325968325138092,
"learning_rate": 0.0003075575602531305,
"loss": 3.0025,
"step": 72500
},
{
"epoch": 19.52379414298019,
"grad_norm": 0.4281940162181854,
"learning_rate": 0.0003073555944526727,
"loss": 3.0107,
"step": 72550
},
{
"epoch": 19.537252368647717,
"grad_norm": 0.42506474256515503,
"learning_rate": 0.00030715362865221487,
"loss": 3.0072,
"step": 72600
},
{
"epoch": 19.550710594315245,
"grad_norm": 0.43724924325942993,
"learning_rate": 0.0003069516628517571,
"loss": 3.0121,
"step": 72650
},
{
"epoch": 19.564168819982772,
"grad_norm": 0.4503478705883026,
"learning_rate": 0.0003067496970512993,
"loss": 3.0069,
"step": 72700
},
{
"epoch": 19.577627045650303,
"grad_norm": 0.4112701416015625,
"learning_rate": 0.0003065477312508415,
"loss": 3.0038,
"step": 72750
},
{
"epoch": 19.59108527131783,
"grad_norm": 0.428783655166626,
"learning_rate": 0.0003063457654503837,
"loss": 3.0042,
"step": 72800
},
{
"epoch": 19.604543496985357,
"grad_norm": 0.4140222668647766,
"learning_rate": 0.0003061437996499259,
"loss": 3.0208,
"step": 72850
},
{
"epoch": 19.618001722652885,
"grad_norm": 0.39155247807502747,
"learning_rate": 0.00030594183384946816,
"loss": 3.0183,
"step": 72900
},
{
"epoch": 19.631459948320412,
"grad_norm": 0.44719892740249634,
"learning_rate": 0.00030573986804901035,
"loss": 3.0273,
"step": 72950
},
{
"epoch": 19.644918173987943,
"grad_norm": 0.42863699793815613,
"learning_rate": 0.00030553790224855255,
"loss": 3.0114,
"step": 73000
},
{
"epoch": 19.644918173987943,
"eval_accuracy": 0.3954941106885473,
"eval_loss": 3.282496690750122,
"eval_runtime": 84.0925,
"eval_samples_per_second": 214.193,
"eval_steps_per_second": 13.39,
"step": 73000
},
{
"epoch": 19.65837639965547,
"grad_norm": 0.40204405784606934,
"learning_rate": 0.00030533593644809474,
"loss": 3.0191,
"step": 73050
},
{
"epoch": 19.671834625322997,
"grad_norm": 0.5403961539268494,
"learning_rate": 0.00030513397064763694,
"loss": 3.0225,
"step": 73100
},
{
"epoch": 19.685292850990525,
"grad_norm": 0.3981429636478424,
"learning_rate": 0.0003049320048471792,
"loss": 3.017,
"step": 73150
},
{
"epoch": 19.698751076658052,
"grad_norm": 0.41193249821662903,
"learning_rate": 0.0003047300390467214,
"loss": 3.0141,
"step": 73200
},
{
"epoch": 19.712209302325583,
"grad_norm": 0.4144224226474762,
"learning_rate": 0.0003045280732462636,
"loss": 3.0261,
"step": 73250
},
{
"epoch": 19.72566752799311,
"grad_norm": 0.41177770495414734,
"learning_rate": 0.0003043261074458058,
"loss": 3.0294,
"step": 73300
},
{
"epoch": 19.739125753660637,
"grad_norm": 0.41590163111686707,
"learning_rate": 0.000304124141645348,
"loss": 3.0213,
"step": 73350
},
{
"epoch": 19.752583979328165,
"grad_norm": 0.41663703322410583,
"learning_rate": 0.0003039221758448903,
"loss": 3.0266,
"step": 73400
},
{
"epoch": 19.766042204995692,
"grad_norm": 0.4177576005458832,
"learning_rate": 0.0003037202100444325,
"loss": 3.0302,
"step": 73450
},
{
"epoch": 19.779500430663223,
"grad_norm": 0.4391637444496155,
"learning_rate": 0.00030351824424397467,
"loss": 3.0236,
"step": 73500
},
{
"epoch": 19.79295865633075,
"grad_norm": 0.42288199067115784,
"learning_rate": 0.00030331627844351687,
"loss": 3.0299,
"step": 73550
},
{
"epoch": 19.806416881998278,
"grad_norm": 0.4287734925746918,
"learning_rate": 0.0003031143126430591,
"loss": 3.0197,
"step": 73600
},
{
"epoch": 19.819875107665805,
"grad_norm": 0.4228683114051819,
"learning_rate": 0.0003029123468426013,
"loss": 3.0321,
"step": 73650
},
{
"epoch": 19.833333333333332,
"grad_norm": 0.41015538573265076,
"learning_rate": 0.0003027103810421435,
"loss": 3.0173,
"step": 73700
},
{
"epoch": 19.846791559000863,
"grad_norm": 0.3986530005931854,
"learning_rate": 0.0003025084152416857,
"loss": 3.0236,
"step": 73750
},
{
"epoch": 19.86024978466839,
"grad_norm": 0.42268338799476624,
"learning_rate": 0.0003023064494412279,
"loss": 3.0255,
"step": 73800
},
{
"epoch": 19.873708010335918,
"grad_norm": 0.4277915954589844,
"learning_rate": 0.00030210448364077015,
"loss": 3.0403,
"step": 73850
},
{
"epoch": 19.887166236003445,
"grad_norm": 0.4032815992832184,
"learning_rate": 0.00030190251784031235,
"loss": 3.035,
"step": 73900
},
{
"epoch": 19.900624461670972,
"grad_norm": 0.40952691435813904,
"learning_rate": 0.00030170055203985454,
"loss": 3.0269,
"step": 73950
},
{
"epoch": 19.9140826873385,
"grad_norm": 0.42225638031959534,
"learning_rate": 0.00030149858623939674,
"loss": 3.0345,
"step": 74000
},
{
"epoch": 19.9140826873385,
"eval_accuracy": 0.39611828687376494,
"eval_loss": 3.2770566940307617,
"eval_runtime": 84.1856,
"eval_samples_per_second": 213.956,
"eval_steps_per_second": 13.375,
"step": 74000
},
{
"epoch": 19.92754091300603,
"grad_norm": 0.4844612181186676,
"learning_rate": 0.00030129662043893894,
"loss": 3.0382,
"step": 74050
},
{
"epoch": 19.940999138673558,
"grad_norm": 0.46322786808013916,
"learning_rate": 0.0003010946546384812,
"loss": 3.0347,
"step": 74100
},
{
"epoch": 19.954457364341085,
"grad_norm": 0.38965100049972534,
"learning_rate": 0.0003008926888380234,
"loss": 3.0335,
"step": 74150
},
{
"epoch": 19.967915590008612,
"grad_norm": 0.40263253450393677,
"learning_rate": 0.0003006907230375656,
"loss": 3.0308,
"step": 74200
},
{
"epoch": 19.98137381567614,
"grad_norm": 0.42235392332077026,
"learning_rate": 0.0003004887572371078,
"loss": 3.0277,
"step": 74250
},
{
"epoch": 19.99483204134367,
"grad_norm": 0.4125952422618866,
"learning_rate": 0.0003002867914366501,
"loss": 3.0265,
"step": 74300
},
{
"epoch": 20.00807493540052,
"grad_norm": 0.4285377860069275,
"learning_rate": 0.0003000848256361923,
"loss": 2.9725,
"step": 74350
},
{
"epoch": 20.021533161068046,
"grad_norm": 0.42794254422187805,
"learning_rate": 0.00029988285983573447,
"loss": 2.9399,
"step": 74400
},
{
"epoch": 20.034991386735573,
"grad_norm": 0.42025497555732727,
"learning_rate": 0.00029968089403527667,
"loss": 2.9645,
"step": 74450
},
{
"epoch": 20.0484496124031,
"grad_norm": 0.4233931303024292,
"learning_rate": 0.0002994789282348189,
"loss": 2.9521,
"step": 74500
},
{
"epoch": 20.061907838070628,
"grad_norm": 0.4307868480682373,
"learning_rate": 0.0002992769624343611,
"loss": 2.9538,
"step": 74550
},
{
"epoch": 20.075366063738155,
"grad_norm": 0.40296119451522827,
"learning_rate": 0.0002990749966339033,
"loss": 2.9526,
"step": 74600
},
{
"epoch": 20.088824289405686,
"grad_norm": 0.4423048794269562,
"learning_rate": 0.0002988730308334455,
"loss": 2.9594,
"step": 74650
},
{
"epoch": 20.102282515073213,
"grad_norm": 0.40786945819854736,
"learning_rate": 0.0002986710650329877,
"loss": 2.9683,
"step": 74700
},
{
"epoch": 20.11574074074074,
"grad_norm": 0.4542340040206909,
"learning_rate": 0.00029846909923252995,
"loss": 2.9531,
"step": 74750
},
{
"epoch": 20.129198966408268,
"grad_norm": 0.39865925908088684,
"learning_rate": 0.00029826713343207215,
"loss": 2.9618,
"step": 74800
},
{
"epoch": 20.142657192075795,
"grad_norm": 0.4336836040019989,
"learning_rate": 0.00029806516763161435,
"loss": 2.9662,
"step": 74850
},
{
"epoch": 20.156115417743326,
"grad_norm": 0.434036523103714,
"learning_rate": 0.00029786320183115654,
"loss": 2.9768,
"step": 74900
},
{
"epoch": 20.169573643410853,
"grad_norm": 0.4439670145511627,
"learning_rate": 0.0002976612360306988,
"loss": 2.9656,
"step": 74950
},
{
"epoch": 20.18303186907838,
"grad_norm": 0.4146876633167267,
"learning_rate": 0.000297459270230241,
"loss": 2.9698,
"step": 75000
},
{
"epoch": 20.18303186907838,
"eval_accuracy": 0.39523020747638127,
"eval_loss": 3.2923033237457275,
"eval_runtime": 84.1703,
"eval_samples_per_second": 213.995,
"eval_steps_per_second": 13.378,
"step": 75000
},
{
"epoch": 20.196490094745908,
"grad_norm": 0.40108510851860046,
"learning_rate": 0.0002972573044297832,
"loss": 2.9759,
"step": 75050
},
{
"epoch": 20.209948320413435,
"grad_norm": 0.4314274191856384,
"learning_rate": 0.00029705533862932543,
"loss": 2.9713,
"step": 75100
},
{
"epoch": 20.223406546080966,
"grad_norm": 0.42117205262184143,
"learning_rate": 0.00029685337282886763,
"loss": 2.9687,
"step": 75150
},
{
"epoch": 20.236864771748493,
"grad_norm": 0.41496866941452026,
"learning_rate": 0.0002966514070284098,
"loss": 2.9855,
"step": 75200
},
{
"epoch": 20.25032299741602,
"grad_norm": 0.4257923364639282,
"learning_rate": 0.000296449441227952,
"loss": 2.9734,
"step": 75250
},
{
"epoch": 20.263781223083548,
"grad_norm": 0.44260990619659424,
"learning_rate": 0.0002962474754274942,
"loss": 2.9859,
"step": 75300
},
{
"epoch": 20.277239448751075,
"grad_norm": 0.4316161274909973,
"learning_rate": 0.00029604550962703647,
"loss": 2.9723,
"step": 75350
},
{
"epoch": 20.290697674418606,
"grad_norm": 0.4551236927509308,
"learning_rate": 0.00029584354382657867,
"loss": 2.9815,
"step": 75400
},
{
"epoch": 20.304155900086133,
"grad_norm": 0.4505977928638458,
"learning_rate": 0.0002956415780261209,
"loss": 2.9736,
"step": 75450
},
{
"epoch": 20.31761412575366,
"grad_norm": 0.417833536863327,
"learning_rate": 0.0002954396122256631,
"loss": 2.9779,
"step": 75500
},
{
"epoch": 20.331072351421188,
"grad_norm": 0.4278238117694855,
"learning_rate": 0.0002952376464252053,
"loss": 2.9783,
"step": 75550
},
{
"epoch": 20.344530577088715,
"grad_norm": 0.4067491292953491,
"learning_rate": 0.0002950356806247475,
"loss": 2.9846,
"step": 75600
},
{
"epoch": 20.357988802756246,
"grad_norm": 0.46258091926574707,
"learning_rate": 0.0002948337148242897,
"loss": 2.9934,
"step": 75650
},
{
"epoch": 20.371447028423773,
"grad_norm": 0.46640563011169434,
"learning_rate": 0.00029463174902383195,
"loss": 2.9984,
"step": 75700
},
{
"epoch": 20.3849052540913,
"grad_norm": 0.4281347393989563,
"learning_rate": 0.00029442978322337415,
"loss": 2.9832,
"step": 75750
},
{
"epoch": 20.398363479758828,
"grad_norm": 0.40202075242996216,
"learning_rate": 0.00029422781742291634,
"loss": 2.9862,
"step": 75800
},
{
"epoch": 20.411821705426355,
"grad_norm": 0.43491074442863464,
"learning_rate": 0.0002940258516224586,
"loss": 3.0028,
"step": 75850
},
{
"epoch": 20.425279931093886,
"grad_norm": 0.4118116796016693,
"learning_rate": 0.0002938238858220008,
"loss": 2.9898,
"step": 75900
},
{
"epoch": 20.438738156761413,
"grad_norm": 0.4505225718021393,
"learning_rate": 0.000293621920021543,
"loss": 2.9878,
"step": 75950
},
{
"epoch": 20.45219638242894,
"grad_norm": 0.4443102777004242,
"learning_rate": 0.0002934199542210852,
"loss": 3.0003,
"step": 76000
},
{
"epoch": 20.45219638242894,
"eval_accuracy": 0.39568022275212916,
"eval_loss": 3.2855124473571777,
"eval_runtime": 84.2204,
"eval_samples_per_second": 213.867,
"eval_steps_per_second": 13.37,
"step": 76000
},
{
"epoch": 20.465654608096468,
"grad_norm": 0.4270143210887909,
"learning_rate": 0.00029321798842062743,
"loss": 3.0053,
"step": 76050
},
{
"epoch": 20.479112833763995,
"grad_norm": 0.44338861107826233,
"learning_rate": 0.00029301602262016963,
"loss": 3.0084,
"step": 76100
},
{
"epoch": 20.492571059431526,
"grad_norm": 0.4215359091758728,
"learning_rate": 0.0002928140568197118,
"loss": 3.0041,
"step": 76150
},
{
"epoch": 20.506029285099054,
"grad_norm": 0.4466627240180969,
"learning_rate": 0.000292612091019254,
"loss": 3.001,
"step": 76200
},
{
"epoch": 20.51948751076658,
"grad_norm": 0.4711362421512604,
"learning_rate": 0.0002924101252187962,
"loss": 2.9919,
"step": 76250
},
{
"epoch": 20.532945736434108,
"grad_norm": 0.4064037501811981,
"learning_rate": 0.00029220815941833847,
"loss": 2.9949,
"step": 76300
},
{
"epoch": 20.546403962101635,
"grad_norm": 0.45425111055374146,
"learning_rate": 0.0002920061936178807,
"loss": 3.0046,
"step": 76350
},
{
"epoch": 20.559862187769163,
"grad_norm": 0.4240078032016754,
"learning_rate": 0.0002918042278174229,
"loss": 3.0034,
"step": 76400
},
{
"epoch": 20.573320413436694,
"grad_norm": 0.41471272706985474,
"learning_rate": 0.0002916022620169651,
"loss": 3.0037,
"step": 76450
},
{
"epoch": 20.58677863910422,
"grad_norm": 0.44904863834381104,
"learning_rate": 0.0002914002962165073,
"loss": 3.0002,
"step": 76500
},
{
"epoch": 20.600236864771748,
"grad_norm": 0.40947166085243225,
"learning_rate": 0.0002911983304160495,
"loss": 3.0096,
"step": 76550
},
{
"epoch": 20.613695090439276,
"grad_norm": 0.45504602789878845,
"learning_rate": 0.00029099636461559175,
"loss": 3.0149,
"step": 76600
},
{
"epoch": 20.627153316106803,
"grad_norm": 0.44130855798721313,
"learning_rate": 0.00029079439881513395,
"loss": 3.0,
"step": 76650
},
{
"epoch": 20.640611541774334,
"grad_norm": 0.41574594378471375,
"learning_rate": 0.00029059243301467614,
"loss": 3.0135,
"step": 76700
},
{
"epoch": 20.65406976744186,
"grad_norm": 0.4264014661312103,
"learning_rate": 0.0002903904672142184,
"loss": 3.0095,
"step": 76750
},
{
"epoch": 20.66752799310939,
"grad_norm": 0.3829843997955322,
"learning_rate": 0.0002901885014137606,
"loss": 3.0188,
"step": 76800
},
{
"epoch": 20.680986218776916,
"grad_norm": 0.41488978266716003,
"learning_rate": 0.0002899865356133028,
"loss": 3.0143,
"step": 76850
},
{
"epoch": 20.694444444444443,
"grad_norm": 0.44957494735717773,
"learning_rate": 0.000289784569812845,
"loss": 3.0058,
"step": 76900
},
{
"epoch": 20.707902670111974,
"grad_norm": 0.447897732257843,
"learning_rate": 0.00028958260401238723,
"loss": 3.0046,
"step": 76950
},
{
"epoch": 20.7213608957795,
"grad_norm": 0.4053957164287567,
"learning_rate": 0.00028938063821192943,
"loss": 3.0066,
"step": 77000
},
{
"epoch": 20.7213608957795,
"eval_accuracy": 0.3959690060942194,
"eval_loss": 3.284266710281372,
"eval_runtime": 84.0458,
"eval_samples_per_second": 214.312,
"eval_steps_per_second": 13.397,
"step": 77000
},
{
"epoch": 20.73481912144703,
"grad_norm": 0.42543306946754456,
"learning_rate": 0.0002891786724114716,
"loss": 3.0144,
"step": 77050
},
{
"epoch": 20.748277347114556,
"grad_norm": 0.43379610776901245,
"learning_rate": 0.0002889767066110138,
"loss": 3.0165,
"step": 77100
},
{
"epoch": 20.761735572782083,
"grad_norm": 0.4360213279724121,
"learning_rate": 0.000288774740810556,
"loss": 3.0053,
"step": 77150
},
{
"epoch": 20.775193798449614,
"grad_norm": 0.4409184455871582,
"learning_rate": 0.00028857277501009827,
"loss": 3.0098,
"step": 77200
},
{
"epoch": 20.78865202411714,
"grad_norm": 0.4014228582382202,
"learning_rate": 0.00028837080920964046,
"loss": 3.0204,
"step": 77250
},
{
"epoch": 20.80211024978467,
"grad_norm": 0.44066837430000305,
"learning_rate": 0.0002881688434091827,
"loss": 3.0154,
"step": 77300
},
{
"epoch": 20.815568475452196,
"grad_norm": 0.4298141598701477,
"learning_rate": 0.0002879668776087249,
"loss": 3.0089,
"step": 77350
},
{
"epoch": 20.829026701119723,
"grad_norm": 0.45948871970176697,
"learning_rate": 0.0002877649118082671,
"loss": 3.0072,
"step": 77400
},
{
"epoch": 20.842484926787254,
"grad_norm": 0.409786194562912,
"learning_rate": 0.0002875629460078093,
"loss": 3.0263,
"step": 77450
},
{
"epoch": 20.85594315245478,
"grad_norm": 0.4201638996601105,
"learning_rate": 0.0002873609802073515,
"loss": 3.0099,
"step": 77500
},
{
"epoch": 20.86940137812231,
"grad_norm": 0.4363999664783478,
"learning_rate": 0.00028715901440689375,
"loss": 3.0135,
"step": 77550
},
{
"epoch": 20.882859603789836,
"grad_norm": 0.43155109882354736,
"learning_rate": 0.00028695704860643594,
"loss": 3.0108,
"step": 77600
},
{
"epoch": 20.896317829457363,
"grad_norm": 0.4437941014766693,
"learning_rate": 0.0002867550828059782,
"loss": 3.0232,
"step": 77650
},
{
"epoch": 20.909776055124894,
"grad_norm": 0.41121140122413635,
"learning_rate": 0.0002865531170055204,
"loss": 3.0258,
"step": 77700
},
{
"epoch": 20.92323428079242,
"grad_norm": 0.42540350556373596,
"learning_rate": 0.0002863511512050626,
"loss": 3.0168,
"step": 77750
},
{
"epoch": 20.93669250645995,
"grad_norm": 0.41012293100357056,
"learning_rate": 0.0002861491854046048,
"loss": 3.0203,
"step": 77800
},
{
"epoch": 20.950150732127476,
"grad_norm": 0.4128982126712799,
"learning_rate": 0.000285947219604147,
"loss": 3.0204,
"step": 77850
},
{
"epoch": 20.963608957795003,
"grad_norm": 0.452108234167099,
"learning_rate": 0.00028574525380368923,
"loss": 3.0198,
"step": 77900
},
{
"epoch": 20.977067183462534,
"grad_norm": 0.4156314432621002,
"learning_rate": 0.0002855432880032314,
"loss": 3.0175,
"step": 77950
},
{
"epoch": 20.99052540913006,
"grad_norm": 0.4258723855018616,
"learning_rate": 0.0002853413222027736,
"loss": 3.0236,
"step": 78000
},
{
"epoch": 20.99052540913006,
"eval_accuracy": 0.39619499155379345,
"eval_loss": 3.275212287902832,
"eval_runtime": 84.0307,
"eval_samples_per_second": 214.35,
"eval_steps_per_second": 13.4,
"step": 78000
},
{
"epoch": 21.00376830318691,
"grad_norm": 0.4432719051837921,
"learning_rate": 0.00028513935640231587,
"loss": 2.9986,
"step": 78050
},
{
"epoch": 21.017226528854437,
"grad_norm": 0.422293484210968,
"learning_rate": 0.00028493739060185807,
"loss": 2.9247,
"step": 78100
},
{
"epoch": 21.030684754521964,
"grad_norm": 0.44783154129981995,
"learning_rate": 0.00028473542480140026,
"loss": 2.9395,
"step": 78150
},
{
"epoch": 21.04414298018949,
"grad_norm": 0.4324184060096741,
"learning_rate": 0.0002845334590009425,
"loss": 2.9322,
"step": 78200
},
{
"epoch": 21.05760120585702,
"grad_norm": 0.4230210781097412,
"learning_rate": 0.0002843314932004847,
"loss": 2.938,
"step": 78250
},
{
"epoch": 21.07105943152455,
"grad_norm": 0.4204503893852234,
"learning_rate": 0.0002841295274000269,
"loss": 2.9474,
"step": 78300
},
{
"epoch": 21.084517657192077,
"grad_norm": 0.413437157869339,
"learning_rate": 0.0002839275615995691,
"loss": 2.9412,
"step": 78350
},
{
"epoch": 21.097975882859604,
"grad_norm": 0.44208431243896484,
"learning_rate": 0.0002837255957991113,
"loss": 2.9442,
"step": 78400
},
{
"epoch": 21.11143410852713,
"grad_norm": 0.4027329087257385,
"learning_rate": 0.00028352362999865355,
"loss": 2.9592,
"step": 78450
},
{
"epoch": 21.12489233419466,
"grad_norm": 0.4688771665096283,
"learning_rate": 0.00028332166419819574,
"loss": 2.9498,
"step": 78500
},
{
"epoch": 21.13835055986219,
"grad_norm": 0.4691401720046997,
"learning_rate": 0.000283119698397738,
"loss": 2.9517,
"step": 78550
},
{
"epoch": 21.151808785529717,
"grad_norm": 0.4198927879333496,
"learning_rate": 0.0002829177325972802,
"loss": 2.9617,
"step": 78600
},
{
"epoch": 21.165267011197244,
"grad_norm": 0.4209950566291809,
"learning_rate": 0.0002827157667968224,
"loss": 2.9547,
"step": 78650
},
{
"epoch": 21.17872523686477,
"grad_norm": 0.44256526231765747,
"learning_rate": 0.0002825138009963646,
"loss": 2.959,
"step": 78700
},
{
"epoch": 21.1921834625323,
"grad_norm": 0.4478328227996826,
"learning_rate": 0.0002823118351959068,
"loss": 2.9712,
"step": 78750
},
{
"epoch": 21.205641688199826,
"grad_norm": 0.4415542483329773,
"learning_rate": 0.00028210986939544903,
"loss": 2.9695,
"step": 78800
},
{
"epoch": 21.219099913867357,
"grad_norm": 0.42408132553100586,
"learning_rate": 0.0002819079035949912,
"loss": 2.9576,
"step": 78850
},
{
"epoch": 21.232558139534884,
"grad_norm": 0.4532260596752167,
"learning_rate": 0.0002817059377945334,
"loss": 2.9654,
"step": 78900
},
{
"epoch": 21.24601636520241,
"grad_norm": 0.4424733519554138,
"learning_rate": 0.00028150397199407567,
"loss": 2.971,
"step": 78950
},
{
"epoch": 21.25947459086994,
"grad_norm": 0.42319878935813904,
"learning_rate": 0.00028130200619361787,
"loss": 2.9793,
"step": 79000
},
{
"epoch": 21.25947459086994,
"eval_accuracy": 0.3956429568806705,
"eval_loss": 3.291125774383545,
"eval_runtime": 84.0955,
"eval_samples_per_second": 214.185,
"eval_steps_per_second": 13.39,
"step": 79000
},
{
"epoch": 21.272932816537466,
"grad_norm": 0.43081074953079224,
"learning_rate": 0.00028110004039316006,
"loss": 2.9673,
"step": 79050
},
{
"epoch": 21.286391042204997,
"grad_norm": 0.4247548282146454,
"learning_rate": 0.00028089807459270226,
"loss": 2.9688,
"step": 79100
},
{
"epoch": 21.299849267872524,
"grad_norm": 0.46288377046585083,
"learning_rate": 0.0002806961087922445,
"loss": 2.9804,
"step": 79150
},
{
"epoch": 21.31330749354005,
"grad_norm": 0.4464956521987915,
"learning_rate": 0.0002804941429917867,
"loss": 2.9755,
"step": 79200
},
{
"epoch": 21.32676571920758,
"grad_norm": 0.42706120014190674,
"learning_rate": 0.0002802921771913289,
"loss": 2.9889,
"step": 79250
},
{
"epoch": 21.340223944875106,
"grad_norm": 0.4685516655445099,
"learning_rate": 0.0002800902113908711,
"loss": 2.9751,
"step": 79300
},
{
"epoch": 21.353682170542637,
"grad_norm": 0.44617944955825806,
"learning_rate": 0.0002798882455904133,
"loss": 2.9713,
"step": 79350
},
{
"epoch": 21.367140396210164,
"grad_norm": 0.4019935429096222,
"learning_rate": 0.00027968627978995555,
"loss": 2.9746,
"step": 79400
},
{
"epoch": 21.38059862187769,
"grad_norm": 0.41742581129074097,
"learning_rate": 0.00027948431398949774,
"loss": 2.9702,
"step": 79450
},
{
"epoch": 21.39405684754522,
"grad_norm": 0.44034504890441895,
"learning_rate": 0.00027928234818904,
"loss": 2.9848,
"step": 79500
},
{
"epoch": 21.407515073212746,
"grad_norm": 0.44808515906333923,
"learning_rate": 0.0002790803823885822,
"loss": 2.9756,
"step": 79550
},
{
"epoch": 21.420973298880277,
"grad_norm": 0.4280387759208679,
"learning_rate": 0.0002788784165881244,
"loss": 2.9905,
"step": 79600
},
{
"epoch": 21.434431524547804,
"grad_norm": 0.42210879921913147,
"learning_rate": 0.0002786764507876666,
"loss": 2.9842,
"step": 79650
},
{
"epoch": 21.44788975021533,
"grad_norm": 0.44455796480178833,
"learning_rate": 0.0002784744849872088,
"loss": 2.9805,
"step": 79700
},
{
"epoch": 21.46134797588286,
"grad_norm": 0.45095279812812805,
"learning_rate": 0.000278272519186751,
"loss": 2.9814,
"step": 79750
},
{
"epoch": 21.474806201550386,
"grad_norm": 0.4286750257015228,
"learning_rate": 0.0002780705533862932,
"loss": 2.9958,
"step": 79800
},
{
"epoch": 21.488264427217917,
"grad_norm": 0.43087276816368103,
"learning_rate": 0.0002778685875858355,
"loss": 2.9874,
"step": 79850
},
{
"epoch": 21.501722652885444,
"grad_norm": 0.4437640309333801,
"learning_rate": 0.00027766662178537767,
"loss": 3.0018,
"step": 79900
},
{
"epoch": 21.51518087855297,
"grad_norm": 0.44071564078330994,
"learning_rate": 0.00027746465598491987,
"loss": 2.9924,
"step": 79950
},
{
"epoch": 21.5286391042205,
"grad_norm": 0.41708236932754517,
"learning_rate": 0.00027726269018446206,
"loss": 2.9953,
"step": 80000
},
{
"epoch": 21.5286391042205,
"eval_accuracy": 0.39576029548468017,
"eval_loss": 3.2859139442443848,
"eval_runtime": 84.2568,
"eval_samples_per_second": 213.775,
"eval_steps_per_second": 13.364,
"step": 80000
},
{
"epoch": 21.542097329888026,
"grad_norm": 0.4567016661167145,
"learning_rate": 0.00027706072438400426,
"loss": 2.982,
"step": 80050
},
{
"epoch": 21.555555555555557,
"grad_norm": 0.43089812994003296,
"learning_rate": 0.0002768587585835465,
"loss": 2.9855,
"step": 80100
},
{
"epoch": 21.569013781223084,
"grad_norm": 0.414765328168869,
"learning_rate": 0.0002766567927830887,
"loss": 2.9885,
"step": 80150
},
{
"epoch": 21.58247200689061,
"grad_norm": 0.429155170917511,
"learning_rate": 0.0002764548269826309,
"loss": 2.9986,
"step": 80200
},
{
"epoch": 21.59593023255814,
"grad_norm": 0.46186089515686035,
"learning_rate": 0.0002762528611821731,
"loss": 2.988,
"step": 80250
},
{
"epoch": 21.609388458225666,
"grad_norm": 0.44444066286087036,
"learning_rate": 0.00027605089538171535,
"loss": 2.9902,
"step": 80300
},
{
"epoch": 21.622846683893197,
"grad_norm": 0.42528036236763,
"learning_rate": 0.00027584892958125754,
"loss": 2.9959,
"step": 80350
},
{
"epoch": 21.636304909560724,
"grad_norm": 0.45731401443481445,
"learning_rate": 0.0002756469637807998,
"loss": 3.0027,
"step": 80400
},
{
"epoch": 21.649763135228252,
"grad_norm": 0.42883455753326416,
"learning_rate": 0.000275444997980342,
"loss": 2.9955,
"step": 80450
},
{
"epoch": 21.66322136089578,
"grad_norm": 0.40275973081588745,
"learning_rate": 0.0002752430321798842,
"loss": 2.9981,
"step": 80500
},
{
"epoch": 21.676679586563306,
"grad_norm": 0.4312342405319214,
"learning_rate": 0.0002750410663794264,
"loss": 3.0058,
"step": 80550
},
{
"epoch": 21.690137812230837,
"grad_norm": 0.43648454546928406,
"learning_rate": 0.0002748391005789686,
"loss": 3.0002,
"step": 80600
},
{
"epoch": 21.703596037898365,
"grad_norm": 0.4063485264778137,
"learning_rate": 0.00027463713477851083,
"loss": 3.0025,
"step": 80650
},
{
"epoch": 21.717054263565892,
"grad_norm": 0.44078052043914795,
"learning_rate": 0.000274435168978053,
"loss": 2.9997,
"step": 80700
},
{
"epoch": 21.73051248923342,
"grad_norm": 0.4490727186203003,
"learning_rate": 0.0002742332031775953,
"loss": 2.9849,
"step": 80750
},
{
"epoch": 21.743970714900946,
"grad_norm": 0.4792262613773346,
"learning_rate": 0.00027403123737713747,
"loss": 3.015,
"step": 80800
},
{
"epoch": 21.757428940568474,
"grad_norm": 0.4362357258796692,
"learning_rate": 0.00027382927157667967,
"loss": 3.0016,
"step": 80850
},
{
"epoch": 21.770887166236005,
"grad_norm": 0.4123806059360504,
"learning_rate": 0.00027362730577622186,
"loss": 3.0035,
"step": 80900
},
{
"epoch": 21.784345391903532,
"grad_norm": 0.41003212332725525,
"learning_rate": 0.00027342533997576406,
"loss": 2.9887,
"step": 80950
},
{
"epoch": 21.79780361757106,
"grad_norm": 0.42923009395599365,
"learning_rate": 0.0002732233741753063,
"loss": 3.0028,
"step": 81000
},
{
"epoch": 21.79780361757106,
"eval_accuracy": 0.39638946942525377,
"eval_loss": 3.2749459743499756,
"eval_runtime": 83.8283,
"eval_samples_per_second": 214.868,
"eval_steps_per_second": 13.432,
"step": 81000
},
{
"epoch": 21.811261843238587,
"grad_norm": 0.44958174228668213,
"learning_rate": 0.0002730214083748485,
"loss": 3.0057,
"step": 81050
},
{
"epoch": 21.824720068906114,
"grad_norm": 0.437704473733902,
"learning_rate": 0.0002728194425743907,
"loss": 3.0011,
"step": 81100
},
{
"epoch": 21.838178294573645,
"grad_norm": 0.4207727313041687,
"learning_rate": 0.0002726174767739329,
"loss": 2.9998,
"step": 81150
},
{
"epoch": 21.851636520241172,
"grad_norm": 0.42127111554145813,
"learning_rate": 0.00027241551097347515,
"loss": 3.0176,
"step": 81200
},
{
"epoch": 21.8650947459087,
"grad_norm": 0.4241941273212433,
"learning_rate": 0.00027221354517301734,
"loss": 2.9997,
"step": 81250
},
{
"epoch": 21.878552971576227,
"grad_norm": 0.4211195111274719,
"learning_rate": 0.00027201157937255954,
"loss": 3.0019,
"step": 81300
},
{
"epoch": 21.892011197243754,
"grad_norm": 0.4051482081413269,
"learning_rate": 0.0002718096135721018,
"loss": 3.0036,
"step": 81350
},
{
"epoch": 21.905469422911285,
"grad_norm": 0.44209906458854675,
"learning_rate": 0.000271607647771644,
"loss": 3.0129,
"step": 81400
},
{
"epoch": 21.918927648578812,
"grad_norm": 0.40235477685928345,
"learning_rate": 0.0002714056819711862,
"loss": 3.017,
"step": 81450
},
{
"epoch": 21.93238587424634,
"grad_norm": 0.42072221636772156,
"learning_rate": 0.0002712037161707284,
"loss": 3.0024,
"step": 81500
},
{
"epoch": 21.945844099913867,
"grad_norm": 0.4275611340999603,
"learning_rate": 0.0002710017503702706,
"loss": 3.0145,
"step": 81550
},
{
"epoch": 21.959302325581394,
"grad_norm": 0.43438443541526794,
"learning_rate": 0.0002707997845698128,
"loss": 3.0087,
"step": 81600
},
{
"epoch": 21.972760551248925,
"grad_norm": 0.44001317024230957,
"learning_rate": 0.000270597818769355,
"loss": 3.0132,
"step": 81650
},
{
"epoch": 21.986218776916452,
"grad_norm": 0.41912901401519775,
"learning_rate": 0.00027039585296889727,
"loss": 3.0129,
"step": 81700
},
{
"epoch": 21.99967700258398,
"grad_norm": 0.42187735438346863,
"learning_rate": 0.00027019388716843947,
"loss": 3.0144,
"step": 81750
},
{
"epoch": 22.012919896640827,
"grad_norm": 0.4827214479446411,
"learning_rate": 0.00026999192136798166,
"loss": 2.9178,
"step": 81800
},
{
"epoch": 22.026378122308355,
"grad_norm": 0.43587005138397217,
"learning_rate": 0.00026978995556752386,
"loss": 2.9323,
"step": 81850
},
{
"epoch": 22.039836347975882,
"grad_norm": 0.44450515508651733,
"learning_rate": 0.00026958798976706606,
"loss": 2.9233,
"step": 81900
},
{
"epoch": 22.05329457364341,
"grad_norm": 0.43394994735717773,
"learning_rate": 0.0002693860239666083,
"loss": 2.9288,
"step": 81950
},
{
"epoch": 22.06675279931094,
"grad_norm": 0.44612663984298706,
"learning_rate": 0.0002691840581661505,
"loss": 2.9377,
"step": 82000
},
{
"epoch": 22.06675279931094,
"eval_accuracy": 0.396058422456349,
"eval_loss": 3.289057493209839,
"eval_runtime": 83.9224,
"eval_samples_per_second": 214.627,
"eval_steps_per_second": 13.417,
"step": 82000
},
{
"epoch": 22.080211024978468,
"grad_norm": 0.4203513562679291,
"learning_rate": 0.0002689820923656927,
"loss": 2.9275,
"step": 82050
},
{
"epoch": 22.093669250645995,
"grad_norm": 0.45312684774398804,
"learning_rate": 0.00026878012656523495,
"loss": 2.9374,
"step": 82100
},
{
"epoch": 22.107127476313522,
"grad_norm": 0.47145548462867737,
"learning_rate": 0.00026857816076477714,
"loss": 2.9412,
"step": 82150
},
{
"epoch": 22.12058570198105,
"grad_norm": 0.43565475940704346,
"learning_rate": 0.00026837619496431934,
"loss": 2.9443,
"step": 82200
},
{
"epoch": 22.13404392764858,
"grad_norm": 0.42642873525619507,
"learning_rate": 0.0002681742291638616,
"loss": 2.9447,
"step": 82250
},
{
"epoch": 22.147502153316108,
"grad_norm": 0.47161009907722473,
"learning_rate": 0.0002679722633634038,
"loss": 2.9448,
"step": 82300
},
{
"epoch": 22.160960378983635,
"grad_norm": 0.47496211528778076,
"learning_rate": 0.000267770297562946,
"loss": 2.9489,
"step": 82350
},
{
"epoch": 22.174418604651162,
"grad_norm": 0.4288054406642914,
"learning_rate": 0.0002675683317624882,
"loss": 2.9493,
"step": 82400
},
{
"epoch": 22.18787683031869,
"grad_norm": 0.4706723093986511,
"learning_rate": 0.0002673663659620304,
"loss": 2.9505,
"step": 82450
},
{
"epoch": 22.20133505598622,
"grad_norm": 0.45602041482925415,
"learning_rate": 0.0002671644001615726,
"loss": 2.9579,
"step": 82500
},
{
"epoch": 22.214793281653748,
"grad_norm": 0.4584351181983948,
"learning_rate": 0.0002669624343611148,
"loss": 2.9577,
"step": 82550
},
{
"epoch": 22.228251507321275,
"grad_norm": 0.42825227975845337,
"learning_rate": 0.00026676046856065707,
"loss": 2.9568,
"step": 82600
},
{
"epoch": 22.241709732988802,
"grad_norm": 0.46178528666496277,
"learning_rate": 0.00026655850276019927,
"loss": 2.9482,
"step": 82650
},
{
"epoch": 22.25516795865633,
"grad_norm": 0.41845497488975525,
"learning_rate": 0.00026635653695974146,
"loss": 2.9635,
"step": 82700
},
{
"epoch": 22.26862618432386,
"grad_norm": 0.4651488661766052,
"learning_rate": 0.00026615457115928366,
"loss": 2.9652,
"step": 82750
},
{
"epoch": 22.282084409991388,
"grad_norm": 0.43473976850509644,
"learning_rate": 0.00026595260535882586,
"loss": 2.9566,
"step": 82800
},
{
"epoch": 22.295542635658915,
"grad_norm": 0.4451102316379547,
"learning_rate": 0.0002657506395583681,
"loss": 2.9629,
"step": 82850
},
{
"epoch": 22.309000861326442,
"grad_norm": 0.4579492509365082,
"learning_rate": 0.0002655486737579103,
"loss": 2.9704,
"step": 82900
},
{
"epoch": 22.32245908699397,
"grad_norm": 0.4861108362674713,
"learning_rate": 0.00026534670795745255,
"loss": 2.9632,
"step": 82950
},
{
"epoch": 22.3359173126615,
"grad_norm": 0.4306095838546753,
"learning_rate": 0.00026514474215699475,
"loss": 2.9726,
"step": 83000
},
{
"epoch": 22.3359173126615,
"eval_accuracy": 0.3960007309760442,
"eval_loss": 3.2885091304779053,
"eval_runtime": 83.9431,
"eval_samples_per_second": 214.574,
"eval_steps_per_second": 13.414,
"step": 83000
},
{
"epoch": 22.349375538329028,
"grad_norm": 0.46351316571235657,
"learning_rate": 0.00026494277635653694,
"loss": 2.9677,
"step": 83050
},
{
"epoch": 22.362833763996555,
"grad_norm": 0.46026599407196045,
"learning_rate": 0.00026474081055607914,
"loss": 2.9621,
"step": 83100
},
{
"epoch": 22.376291989664082,
"grad_norm": 0.4538576900959015,
"learning_rate": 0.00026453884475562134,
"loss": 2.963,
"step": 83150
},
{
"epoch": 22.38975021533161,
"grad_norm": 0.42874839901924133,
"learning_rate": 0.0002643368789551636,
"loss": 2.9723,
"step": 83200
},
{
"epoch": 22.403208440999137,
"grad_norm": 0.4932001829147339,
"learning_rate": 0.0002641349131547058,
"loss": 2.9793,
"step": 83250
},
{
"epoch": 22.416666666666668,
"grad_norm": 0.4467307925224304,
"learning_rate": 0.000263932947354248,
"loss": 2.9627,
"step": 83300
},
{
"epoch": 22.430124892334195,
"grad_norm": 0.4314446747303009,
"learning_rate": 0.0002637309815537902,
"loss": 2.9787,
"step": 83350
},
{
"epoch": 22.443583118001722,
"grad_norm": 0.43153485655784607,
"learning_rate": 0.0002635290157533324,
"loss": 2.9745,
"step": 83400
},
{
"epoch": 22.45704134366925,
"grad_norm": 0.4590122401714325,
"learning_rate": 0.0002633270499528746,
"loss": 2.9702,
"step": 83450
},
{
"epoch": 22.470499569336777,
"grad_norm": 0.43995824456214905,
"learning_rate": 0.0002631250841524168,
"loss": 2.9735,
"step": 83500
},
{
"epoch": 22.483957795004308,
"grad_norm": 0.45518141984939575,
"learning_rate": 0.00026292311835195907,
"loss": 2.9699,
"step": 83550
},
{
"epoch": 22.497416020671835,
"grad_norm": 0.459585577249527,
"learning_rate": 0.00026272115255150126,
"loss": 2.9735,
"step": 83600
},
{
"epoch": 22.510874246339363,
"grad_norm": 0.4053054749965668,
"learning_rate": 0.00026251918675104346,
"loss": 2.9722,
"step": 83650
},
{
"epoch": 22.52433247200689,
"grad_norm": 0.42346060276031494,
"learning_rate": 0.00026231722095058566,
"loss": 2.9775,
"step": 83700
},
{
"epoch": 22.537790697674417,
"grad_norm": 0.4662436246871948,
"learning_rate": 0.00026211525515012785,
"loss": 2.9793,
"step": 83750
},
{
"epoch": 22.551248923341948,
"grad_norm": 0.4733358919620514,
"learning_rate": 0.0002619132893496701,
"loss": 2.9769,
"step": 83800
},
{
"epoch": 22.564707149009475,
"grad_norm": 0.4621383547782898,
"learning_rate": 0.0002617113235492123,
"loss": 2.9854,
"step": 83850
},
{
"epoch": 22.578165374677003,
"grad_norm": 0.4373529851436615,
"learning_rate": 0.00026150935774875455,
"loss": 2.9861,
"step": 83900
},
{
"epoch": 22.59162360034453,
"grad_norm": 0.43990325927734375,
"learning_rate": 0.00026130739194829675,
"loss": 2.9911,
"step": 83950
},
{
"epoch": 22.605081826012057,
"grad_norm": 0.4210509657859802,
"learning_rate": 0.00026110542614783894,
"loss": 2.9868,
"step": 84000
},
{
"epoch": 22.605081826012057,
"eval_accuracy": 0.3961254575662322,
"eval_loss": 3.286008596420288,
"eval_runtime": 83.9197,
"eval_samples_per_second": 214.634,
"eval_steps_per_second": 13.418,
"step": 84000
},
{
"epoch": 22.618540051679588,
"grad_norm": 0.4237697422504425,
"learning_rate": 0.00026090346034738114,
"loss": 2.9899,
"step": 84050
},
{
"epoch": 22.631998277347115,
"grad_norm": 0.4568379819393158,
"learning_rate": 0.0002607014945469234,
"loss": 2.9839,
"step": 84100
},
{
"epoch": 22.645456503014643,
"grad_norm": 0.4833359718322754,
"learning_rate": 0.0002604995287464656,
"loss": 2.9809,
"step": 84150
},
{
"epoch": 22.65891472868217,
"grad_norm": 0.4397493302822113,
"learning_rate": 0.0002602975629460078,
"loss": 2.9883,
"step": 84200
},
{
"epoch": 22.672372954349697,
"grad_norm": 0.4514474868774414,
"learning_rate": 0.00026009559714555,
"loss": 2.9968,
"step": 84250
},
{
"epoch": 22.685831180017228,
"grad_norm": 0.4431244730949402,
"learning_rate": 0.0002598936313450922,
"loss": 2.9908,
"step": 84300
},
{
"epoch": 22.699289405684755,
"grad_norm": 0.4466392695903778,
"learning_rate": 0.0002596916655446344,
"loss": 2.981,
"step": 84350
},
{
"epoch": 22.712747631352283,
"grad_norm": 0.47167009115219116,
"learning_rate": 0.0002594896997441766,
"loss": 2.9881,
"step": 84400
},
{
"epoch": 22.72620585701981,
"grad_norm": 0.4247466027736664,
"learning_rate": 0.00025928773394371887,
"loss": 2.9811,
"step": 84450
},
{
"epoch": 22.739664082687337,
"grad_norm": 0.4402214586734772,
"learning_rate": 0.00025908576814326107,
"loss": 2.9915,
"step": 84500
},
{
"epoch": 22.753122308354868,
"grad_norm": 0.4316474199295044,
"learning_rate": 0.00025888380234280326,
"loss": 2.9842,
"step": 84550
},
{
"epoch": 22.766580534022395,
"grad_norm": 0.4481562674045563,
"learning_rate": 0.00025868183654234546,
"loss": 2.9933,
"step": 84600
},
{
"epoch": 22.780038759689923,
"grad_norm": 0.4497462213039398,
"learning_rate": 0.00025847987074188765,
"loss": 3.0006,
"step": 84650
},
{
"epoch": 22.79349698535745,
"grad_norm": 0.4655749499797821,
"learning_rate": 0.0002582779049414299,
"loss": 2.997,
"step": 84700
},
{
"epoch": 22.806955211024977,
"grad_norm": 0.4840506315231323,
"learning_rate": 0.0002580759391409721,
"loss": 3.0028,
"step": 84750
},
{
"epoch": 22.820413436692505,
"grad_norm": 0.4458298981189728,
"learning_rate": 0.00025787397334051435,
"loss": 2.9878,
"step": 84800
},
{
"epoch": 22.833871662360036,
"grad_norm": 0.4646628797054291,
"learning_rate": 0.00025767200754005655,
"loss": 3.0075,
"step": 84850
},
{
"epoch": 22.847329888027563,
"grad_norm": 0.49665722250938416,
"learning_rate": 0.00025747004173959874,
"loss": 3.0067,
"step": 84900
},
{
"epoch": 22.86078811369509,
"grad_norm": 0.43514347076416016,
"learning_rate": 0.00025726807593914094,
"loss": 2.9904,
"step": 84950
},
{
"epoch": 22.874246339362617,
"grad_norm": 0.4649242162704468,
"learning_rate": 0.00025706611013868313,
"loss": 3.0054,
"step": 85000
},
{
"epoch": 22.874246339362617,
"eval_accuracy": 0.39701668772242726,
"eval_loss": 3.2736573219299316,
"eval_runtime": 83.9771,
"eval_samples_per_second": 214.487,
"eval_steps_per_second": 13.408,
"step": 85000
},
{
"epoch": 22.887704565030145,
"grad_norm": 0.4361061751842499,
"learning_rate": 0.0002568641443382254,
"loss": 2.9948,
"step": 85050
},
{
"epoch": 22.901162790697676,
"grad_norm": 0.41159138083457947,
"learning_rate": 0.0002566621785377676,
"loss": 2.9974,
"step": 85100
},
{
"epoch": 22.914621016365203,
"grad_norm": 0.4136468768119812,
"learning_rate": 0.0002564602127373098,
"loss": 2.9977,
"step": 85150
},
{
"epoch": 22.92807924203273,
"grad_norm": 0.45003828406333923,
"learning_rate": 0.00025625824693685203,
"loss": 2.9851,
"step": 85200
},
{
"epoch": 22.941537467700257,
"grad_norm": 0.42788249254226685,
"learning_rate": 0.0002560562811363942,
"loss": 2.9985,
"step": 85250
},
{
"epoch": 22.954995693367785,
"grad_norm": 0.43006718158721924,
"learning_rate": 0.0002558543153359364,
"loss": 3.0012,
"step": 85300
},
{
"epoch": 22.968453919035316,
"grad_norm": 0.42816489934921265,
"learning_rate": 0.0002556523495354786,
"loss": 2.9948,
"step": 85350
},
{
"epoch": 22.981912144702843,
"grad_norm": 0.42106497287750244,
"learning_rate": 0.00025545038373502087,
"loss": 2.9958,
"step": 85400
},
{
"epoch": 22.99537037037037,
"grad_norm": 0.4418798089027405,
"learning_rate": 0.00025524841793456306,
"loss": 3.0094,
"step": 85450
},
{
"epoch": 23.00861326442722,
"grad_norm": 0.4254322052001953,
"learning_rate": 0.00025504645213410526,
"loss": 2.9523,
"step": 85500
},
{
"epoch": 23.022071490094746,
"grad_norm": 0.4377330541610718,
"learning_rate": 0.00025484448633364745,
"loss": 2.9239,
"step": 85550
},
{
"epoch": 23.035529715762273,
"grad_norm": 0.4505152404308319,
"learning_rate": 0.00025464252053318965,
"loss": 2.9205,
"step": 85600
},
{
"epoch": 23.0489879414298,
"grad_norm": 0.45015689730644226,
"learning_rate": 0.0002544405547327319,
"loss": 2.9157,
"step": 85650
},
{
"epoch": 23.06244616709733,
"grad_norm": 0.4604092240333557,
"learning_rate": 0.0002542385889322741,
"loss": 2.919,
"step": 85700
},
{
"epoch": 23.07590439276486,
"grad_norm": 0.4214881360530853,
"learning_rate": 0.00025403662313181635,
"loss": 2.9218,
"step": 85750
},
{
"epoch": 23.089362618432386,
"grad_norm": 0.46785667538642883,
"learning_rate": 0.00025383465733135854,
"loss": 2.9308,
"step": 85800
},
{
"epoch": 23.102820844099913,
"grad_norm": 0.45559147000312805,
"learning_rate": 0.00025363269153090074,
"loss": 2.9302,
"step": 85850
},
{
"epoch": 23.11627906976744,
"grad_norm": 0.4463777244091034,
"learning_rate": 0.00025343072573044294,
"loss": 2.929,
"step": 85900
},
{
"epoch": 23.12973729543497,
"grad_norm": 0.463544636964798,
"learning_rate": 0.00025322875992998513,
"loss": 2.9306,
"step": 85950
},
{
"epoch": 23.1431955211025,
"grad_norm": 0.4276140630245209,
"learning_rate": 0.0002530267941295274,
"loss": 2.9233,
"step": 86000
},
{
"epoch": 23.1431955211025,
"eval_accuracy": 0.39601811447293456,
"eval_loss": 3.291121482849121,
"eval_runtime": 84.0617,
"eval_samples_per_second": 214.271,
"eval_steps_per_second": 13.395,
"step": 86000
},
{
"epoch": 23.156653746770026,
"grad_norm": 0.4453248083591461,
"learning_rate": 0.0002528248283290696,
"loss": 2.9369,
"step": 86050
},
{
"epoch": 23.170111972437553,
"grad_norm": 0.4330044984817505,
"learning_rate": 0.00025262286252861183,
"loss": 2.9344,
"step": 86100
},
{
"epoch": 23.18357019810508,
"grad_norm": 0.43674394488334656,
"learning_rate": 0.000252420896728154,
"loss": 2.9474,
"step": 86150
},
{
"epoch": 23.19702842377261,
"grad_norm": 0.48593881726264954,
"learning_rate": 0.0002522189309276962,
"loss": 2.9449,
"step": 86200
},
{
"epoch": 23.21048664944014,
"grad_norm": 0.4298238754272461,
"learning_rate": 0.0002520169651272384,
"loss": 2.9502,
"step": 86250
},
{
"epoch": 23.223944875107666,
"grad_norm": 0.5104933381080627,
"learning_rate": 0.00025181499932678067,
"loss": 2.943,
"step": 86300
},
{
"epoch": 23.237403100775193,
"grad_norm": 0.4299788773059845,
"learning_rate": 0.00025161303352632286,
"loss": 2.9565,
"step": 86350
},
{
"epoch": 23.25086132644272,
"grad_norm": 0.4533037841320038,
"learning_rate": 0.00025141106772586506,
"loss": 2.9529,
"step": 86400
},
{
"epoch": 23.26431955211025,
"grad_norm": 0.5035787224769592,
"learning_rate": 0.00025120910192540726,
"loss": 2.9517,
"step": 86450
},
{
"epoch": 23.27777777777778,
"grad_norm": 0.46547257900238037,
"learning_rate": 0.00025100713612494945,
"loss": 2.9443,
"step": 86500
},
{
"epoch": 23.291236003445306,
"grad_norm": 0.42908090353012085,
"learning_rate": 0.0002508051703244917,
"loss": 2.9455,
"step": 86550
},
{
"epoch": 23.304694229112833,
"grad_norm": 0.47686538100242615,
"learning_rate": 0.0002506032045240339,
"loss": 2.9493,
"step": 86600
},
{
"epoch": 23.31815245478036,
"grad_norm": 0.44427627325057983,
"learning_rate": 0.00025040123872357615,
"loss": 2.9707,
"step": 86650
},
{
"epoch": 23.33161068044789,
"grad_norm": 0.4498743712902069,
"learning_rate": 0.00025019927292311834,
"loss": 2.958,
"step": 86700
},
{
"epoch": 23.34506890611542,
"grad_norm": 0.45148012042045593,
"learning_rate": 0.00024999730712266054,
"loss": 2.9652,
"step": 86750
},
{
"epoch": 23.358527131782946,
"grad_norm": 0.4607517421245575,
"learning_rate": 0.00024979534132220274,
"loss": 2.9589,
"step": 86800
},
{
"epoch": 23.371985357450473,
"grad_norm": 0.42376628518104553,
"learning_rate": 0.00024959337552174493,
"loss": 2.9705,
"step": 86850
},
{
"epoch": 23.385443583118,
"grad_norm": 0.47415992617607117,
"learning_rate": 0.0002493914097212872,
"loss": 2.9589,
"step": 86900
},
{
"epoch": 23.39890180878553,
"grad_norm": 0.4472445249557495,
"learning_rate": 0.0002491894439208294,
"loss": 2.9609,
"step": 86950
},
{
"epoch": 23.41236003445306,
"grad_norm": 0.46697506308555603,
"learning_rate": 0.00024898747812037163,
"loss": 2.9528,
"step": 87000
},
{
"epoch": 23.41236003445306,
"eval_accuracy": 0.3964992027493739,
"eval_loss": 3.283453941345215,
"eval_runtime": 83.9935,
"eval_samples_per_second": 214.445,
"eval_steps_per_second": 13.406,
"step": 87000
},
{
"epoch": 23.425818260120586,
"grad_norm": 0.5904593467712402,
"learning_rate": 0.0002487855123199138,
"loss": 2.967,
"step": 87050
},
{
"epoch": 23.439276485788113,
"grad_norm": 0.49468109011650085,
"learning_rate": 0.000248583546519456,
"loss": 2.9732,
"step": 87100
},
{
"epoch": 23.45273471145564,
"grad_norm": 0.45308589935302734,
"learning_rate": 0.0002483815807189982,
"loss": 2.9798,
"step": 87150
},
{
"epoch": 23.466192937123168,
"grad_norm": 0.4565460979938507,
"learning_rate": 0.0002481796149185404,
"loss": 2.9742,
"step": 87200
},
{
"epoch": 23.4796511627907,
"grad_norm": 0.4237934350967407,
"learning_rate": 0.00024797764911808266,
"loss": 2.9548,
"step": 87250
},
{
"epoch": 23.493109388458226,
"grad_norm": 0.426654577255249,
"learning_rate": 0.00024777568331762486,
"loss": 2.963,
"step": 87300
},
{
"epoch": 23.506567614125753,
"grad_norm": 0.447750449180603,
"learning_rate": 0.00024757371751716706,
"loss": 2.9707,
"step": 87350
},
{
"epoch": 23.52002583979328,
"grad_norm": 0.4716593027114868,
"learning_rate": 0.0002473717517167093,
"loss": 2.9643,
"step": 87400
},
{
"epoch": 23.533484065460808,
"grad_norm": 0.43913519382476807,
"learning_rate": 0.0002471697859162515,
"loss": 2.9654,
"step": 87450
},
{
"epoch": 23.54694229112834,
"grad_norm": 0.4803863763809204,
"learning_rate": 0.0002469678201157937,
"loss": 2.9741,
"step": 87500
},
{
"epoch": 23.560400516795866,
"grad_norm": 0.4569952189922333,
"learning_rate": 0.0002467658543153359,
"loss": 2.9743,
"step": 87550
},
{
"epoch": 23.573858742463393,
"grad_norm": 0.4601038694381714,
"learning_rate": 0.00024656388851487815,
"loss": 2.9795,
"step": 87600
},
{
"epoch": 23.58731696813092,
"grad_norm": 0.44096481800079346,
"learning_rate": 0.00024636192271442034,
"loss": 2.9774,
"step": 87650
},
{
"epoch": 23.600775193798448,
"grad_norm": 0.45582062005996704,
"learning_rate": 0.00024615995691396254,
"loss": 2.9792,
"step": 87700
},
{
"epoch": 23.61423341946598,
"grad_norm": 0.4384517967700958,
"learning_rate": 0.00024595799111350473,
"loss": 2.9735,
"step": 87750
},
{
"epoch": 23.627691645133506,
"grad_norm": 0.4709102511405945,
"learning_rate": 0.00024575602531304693,
"loss": 2.9772,
"step": 87800
},
{
"epoch": 23.641149870801033,
"grad_norm": 0.4474182426929474,
"learning_rate": 0.0002455540595125892,
"loss": 2.9737,
"step": 87850
},
{
"epoch": 23.65460809646856,
"grad_norm": 0.47512778639793396,
"learning_rate": 0.00024535209371213143,
"loss": 2.969,
"step": 87900
},
{
"epoch": 23.668066322136088,
"grad_norm": 0.4888509213924408,
"learning_rate": 0.0002451501279116736,
"loss": 2.9792,
"step": 87950
},
{
"epoch": 23.68152454780362,
"grad_norm": 0.47287872433662415,
"learning_rate": 0.0002449481621112158,
"loss": 2.97,
"step": 88000
},
{
"epoch": 23.68152454780362,
"eval_accuracy": 0.39661110901060526,
"eval_loss": 3.2801294326782227,
"eval_runtime": 84.0084,
"eval_samples_per_second": 214.407,
"eval_steps_per_second": 13.403,
"step": 88000
},
{
"epoch": 23.694982773471146,
"grad_norm": 0.47039493918418884,
"learning_rate": 0.000244746196310758,
"loss": 2.9744,
"step": 88050
},
{
"epoch": 23.708440999138674,
"grad_norm": 0.43563172221183777,
"learning_rate": 0.0002445442305103002,
"loss": 2.9885,
"step": 88100
},
{
"epoch": 23.7218992248062,
"grad_norm": 0.42644500732421875,
"learning_rate": 0.00024434226470984246,
"loss": 2.9807,
"step": 88150
},
{
"epoch": 23.735357450473728,
"grad_norm": 0.452154278755188,
"learning_rate": 0.00024414029890938463,
"loss": 2.9846,
"step": 88200
},
{
"epoch": 23.74881567614126,
"grad_norm": 0.47990021109580994,
"learning_rate": 0.00024393833310892686,
"loss": 2.985,
"step": 88250
},
{
"epoch": 23.762273901808786,
"grad_norm": 0.4605526626110077,
"learning_rate": 0.0002437363673084691,
"loss": 2.9819,
"step": 88300
},
{
"epoch": 23.775732127476314,
"grad_norm": 0.43846940994262695,
"learning_rate": 0.0002435344015080113,
"loss": 2.965,
"step": 88350
},
{
"epoch": 23.78919035314384,
"grad_norm": 0.4395309090614319,
"learning_rate": 0.0002433324357075535,
"loss": 2.9785,
"step": 88400
},
{
"epoch": 23.802648578811368,
"grad_norm": 0.4506181478500366,
"learning_rate": 0.00024313046990709572,
"loss": 2.9777,
"step": 88450
},
{
"epoch": 23.8161068044789,
"grad_norm": 0.4906696379184723,
"learning_rate": 0.00024292850410663792,
"loss": 2.9772,
"step": 88500
},
{
"epoch": 23.829565030146426,
"grad_norm": 0.47423872351646423,
"learning_rate": 0.00024272653830618014,
"loss": 2.9695,
"step": 88550
},
{
"epoch": 23.843023255813954,
"grad_norm": 0.4664286971092224,
"learning_rate": 0.00024252457250572234,
"loss": 2.9769,
"step": 88600
},
{
"epoch": 23.85648148148148,
"grad_norm": 0.44155094027519226,
"learning_rate": 0.00024232260670526453,
"loss": 2.9766,
"step": 88650
},
{
"epoch": 23.86993970714901,
"grad_norm": 0.45096275210380554,
"learning_rate": 0.00024212064090480676,
"loss": 2.9964,
"step": 88700
},
{
"epoch": 23.88339793281654,
"grad_norm": 0.43338635563850403,
"learning_rate": 0.00024191867510434898,
"loss": 2.9943,
"step": 88750
},
{
"epoch": 23.896856158484066,
"grad_norm": 0.45911747217178345,
"learning_rate": 0.0002417167093038912,
"loss": 2.9875,
"step": 88800
},
{
"epoch": 23.910314384151594,
"grad_norm": 0.4572448134422302,
"learning_rate": 0.0002415147435034334,
"loss": 2.9924,
"step": 88850
},
{
"epoch": 23.92377260981912,
"grad_norm": 0.4544650614261627,
"learning_rate": 0.00024131277770297562,
"loss": 2.9838,
"step": 88900
},
{
"epoch": 23.93723083548665,
"grad_norm": 0.46896883845329285,
"learning_rate": 0.00024111081190251782,
"loss": 2.9849,
"step": 88950
},
{
"epoch": 23.95068906115418,
"grad_norm": 0.4377521574497223,
"learning_rate": 0.00024090884610206002,
"loss": 2.9728,
"step": 89000
},
{
"epoch": 23.95068906115418,
"eval_accuracy": 0.3970318982822063,
"eval_loss": 3.2736828327178955,
"eval_runtime": 84.1787,
"eval_samples_per_second": 213.973,
"eval_steps_per_second": 13.376,
"step": 89000
},
{
"epoch": 23.964147286821706,
"grad_norm": 0.459255188703537,
"learning_rate": 0.00024070688030160224,
"loss": 2.9961,
"step": 89050
},
{
"epoch": 23.977605512489234,
"grad_norm": 0.45383593440055847,
"learning_rate": 0.00024050491450114443,
"loss": 2.989,
"step": 89100
},
{
"epoch": 23.99106373815676,
"grad_norm": 0.4712170660495758,
"learning_rate": 0.00024030294870068666,
"loss": 2.9929,
"step": 89150
},
{
"epoch": 24.00430663221361,
"grad_norm": 0.47109752893447876,
"learning_rate": 0.00024010098290022888,
"loss": 2.9677,
"step": 89200
},
{
"epoch": 24.017764857881136,
"grad_norm": 0.4482331871986389,
"learning_rate": 0.0002398990170997711,
"loss": 2.9114,
"step": 89250
},
{
"epoch": 24.031223083548664,
"grad_norm": 0.4347875416278839,
"learning_rate": 0.0002396970512993133,
"loss": 2.8935,
"step": 89300
},
{
"epoch": 24.044681309216195,
"grad_norm": 0.46514439582824707,
"learning_rate": 0.00023949508549885552,
"loss": 2.9106,
"step": 89350
},
{
"epoch": 24.058139534883722,
"grad_norm": 0.47808149456977844,
"learning_rate": 0.00023929311969839772,
"loss": 2.917,
"step": 89400
},
{
"epoch": 24.07159776055125,
"grad_norm": 0.49696797132492065,
"learning_rate": 0.00023909115389793992,
"loss": 2.9212,
"step": 89450
},
{
"epoch": 24.085055986218777,
"grad_norm": 0.47918155789375305,
"learning_rate": 0.00023888918809748214,
"loss": 2.9234,
"step": 89500
},
{
"epoch": 24.098514211886304,
"grad_norm": 0.4478085935115814,
"learning_rate": 0.00023868722229702433,
"loss": 2.9265,
"step": 89550
},
{
"epoch": 24.11197243755383,
"grad_norm": 0.45360133051872253,
"learning_rate": 0.00023848525649656656,
"loss": 2.9209,
"step": 89600
},
{
"epoch": 24.125430663221362,
"grad_norm": 0.4664364755153656,
"learning_rate": 0.00023828329069610878,
"loss": 2.9289,
"step": 89650
},
{
"epoch": 24.13888888888889,
"grad_norm": 0.48554185032844543,
"learning_rate": 0.000238081324895651,
"loss": 2.9267,
"step": 89700
},
{
"epoch": 24.152347114556417,
"grad_norm": 0.47263389825820923,
"learning_rate": 0.0002378793590951932,
"loss": 2.933,
"step": 89750
},
{
"epoch": 24.165805340223944,
"grad_norm": 0.48952844738960266,
"learning_rate": 0.0002376773932947354,
"loss": 2.924,
"step": 89800
},
{
"epoch": 24.17926356589147,
"grad_norm": 0.44809359312057495,
"learning_rate": 0.00023747542749427762,
"loss": 2.9298,
"step": 89850
},
{
"epoch": 24.192721791559002,
"grad_norm": 0.4537670314311981,
"learning_rate": 0.00023727346169381982,
"loss": 2.9255,
"step": 89900
},
{
"epoch": 24.20618001722653,
"grad_norm": 0.4854421317577362,
"learning_rate": 0.00023707149589336204,
"loss": 2.9313,
"step": 89950
},
{
"epoch": 24.219638242894057,
"grad_norm": 0.46356984972953796,
"learning_rate": 0.00023686953009290424,
"loss": 2.9435,
"step": 90000
},
{
"epoch": 24.219638242894057,
"eval_accuracy": 0.39640793939069974,
"eval_loss": 3.288700819015503,
"eval_runtime": 84.1466,
"eval_samples_per_second": 214.055,
"eval_steps_per_second": 13.381,
"step": 90000
},
{
"epoch": 24.233096468561584,
"grad_norm": 0.5046513080596924,
"learning_rate": 0.00023666756429244643,
"loss": 2.9484,
"step": 90050
},
{
"epoch": 24.24655469422911,
"grad_norm": 0.4581146538257599,
"learning_rate": 0.00023646559849198868,
"loss": 2.9311,
"step": 90100
},
{
"epoch": 24.260012919896642,
"grad_norm": 0.42044317722320557,
"learning_rate": 0.00023626363269153088,
"loss": 2.9344,
"step": 90150
},
{
"epoch": 24.27347114556417,
"grad_norm": 0.4418257474899292,
"learning_rate": 0.0002360616668910731,
"loss": 2.9471,
"step": 90200
},
{
"epoch": 24.286929371231697,
"grad_norm": 0.4563981294631958,
"learning_rate": 0.0002358597010906153,
"loss": 2.9365,
"step": 90250
},
{
"epoch": 24.300387596899224,
"grad_norm": 0.4627529978752136,
"learning_rate": 0.00023565773529015752,
"loss": 2.9524,
"step": 90300
},
{
"epoch": 24.31384582256675,
"grad_norm": 0.44930917024612427,
"learning_rate": 0.00023545576948969972,
"loss": 2.9427,
"step": 90350
},
{
"epoch": 24.327304048234282,
"grad_norm": 0.4439551830291748,
"learning_rate": 0.00023525380368924194,
"loss": 2.9419,
"step": 90400
},
{
"epoch": 24.34076227390181,
"grad_norm": 0.4861012101173401,
"learning_rate": 0.00023505183788878414,
"loss": 2.9401,
"step": 90450
},
{
"epoch": 24.354220499569337,
"grad_norm": 0.43457016348838806,
"learning_rate": 0.00023484987208832633,
"loss": 2.9452,
"step": 90500
},
{
"epoch": 24.367678725236864,
"grad_norm": 0.4539009928703308,
"learning_rate": 0.00023464790628786858,
"loss": 2.9355,
"step": 90550
},
{
"epoch": 24.38113695090439,
"grad_norm": 0.495311439037323,
"learning_rate": 0.00023444594048741078,
"loss": 2.9517,
"step": 90600
},
{
"epoch": 24.394595176571922,
"grad_norm": 0.4873776137828827,
"learning_rate": 0.000234243974686953,
"loss": 2.9379,
"step": 90650
},
{
"epoch": 24.40805340223945,
"grad_norm": 0.4435269236564636,
"learning_rate": 0.0002340420088864952,
"loss": 2.9579,
"step": 90700
},
{
"epoch": 24.421511627906977,
"grad_norm": 0.4554521143436432,
"learning_rate": 0.00023384004308603742,
"loss": 2.9542,
"step": 90750
},
{
"epoch": 24.434969853574504,
"grad_norm": 0.44386592507362366,
"learning_rate": 0.00023363807728557962,
"loss": 2.96,
"step": 90800
},
{
"epoch": 24.44842807924203,
"grad_norm": 0.436383992433548,
"learning_rate": 0.0002334361114851218,
"loss": 2.9601,
"step": 90850
},
{
"epoch": 24.461886304909562,
"grad_norm": 0.4544837772846222,
"learning_rate": 0.00023323414568466404,
"loss": 2.9581,
"step": 90900
},
{
"epoch": 24.47534453057709,
"grad_norm": 0.44542551040649414,
"learning_rate": 0.00023303217988420623,
"loss": 2.9575,
"step": 90950
},
{
"epoch": 24.488802756244617,
"grad_norm": 0.4572954773902893,
"learning_rate": 0.00023283021408374848,
"loss": 2.9571,
"step": 91000
},
{
"epoch": 24.488802756244617,
"eval_accuracy": 0.39685578172933633,
"eval_loss": 3.2813351154327393,
"eval_runtime": 84.0012,
"eval_samples_per_second": 214.425,
"eval_steps_per_second": 13.405,
"step": 91000
},
{
"epoch": 24.502260981912144,
"grad_norm": 0.4547075033187866,
"learning_rate": 0.00023262824828329068,
"loss": 2.9532,
"step": 91050
},
{
"epoch": 24.51571920757967,
"grad_norm": 0.4635523855686188,
"learning_rate": 0.0002324262824828329,
"loss": 2.9631,
"step": 91100
},
{
"epoch": 24.529177433247202,
"grad_norm": 0.49691566824913025,
"learning_rate": 0.0002322243166823751,
"loss": 2.9491,
"step": 91150
},
{
"epoch": 24.54263565891473,
"grad_norm": 0.4702029526233673,
"learning_rate": 0.0002320223508819173,
"loss": 2.9545,
"step": 91200
},
{
"epoch": 24.556093884582257,
"grad_norm": 0.42786115407943726,
"learning_rate": 0.00023182038508145952,
"loss": 2.967,
"step": 91250
},
{
"epoch": 24.569552110249784,
"grad_norm": 0.4416402578353882,
"learning_rate": 0.0002316184192810017,
"loss": 2.9609,
"step": 91300
},
{
"epoch": 24.58301033591731,
"grad_norm": 0.465877890586853,
"learning_rate": 0.00023141645348054394,
"loss": 2.9648,
"step": 91350
},
{
"epoch": 24.596468561584842,
"grad_norm": 0.44854775071144104,
"learning_rate": 0.00023121448768008613,
"loss": 2.9509,
"step": 91400
},
{
"epoch": 24.60992678725237,
"grad_norm": 0.456569641828537,
"learning_rate": 0.00023101252187962838,
"loss": 2.9626,
"step": 91450
},
{
"epoch": 24.623385012919897,
"grad_norm": 0.4613625109195709,
"learning_rate": 0.00023081055607917058,
"loss": 2.9674,
"step": 91500
},
{
"epoch": 24.636843238587424,
"grad_norm": 0.47509756684303284,
"learning_rate": 0.0002306085902787128,
"loss": 2.9574,
"step": 91550
},
{
"epoch": 24.65030146425495,
"grad_norm": 0.480103999376297,
"learning_rate": 0.000230406624478255,
"loss": 2.9611,
"step": 91600
},
{
"epoch": 24.66375968992248,
"grad_norm": 0.44871267676353455,
"learning_rate": 0.0002302046586777972,
"loss": 2.9734,
"step": 91650
},
{
"epoch": 24.67721791559001,
"grad_norm": 0.4531683921813965,
"learning_rate": 0.00023000269287733942,
"loss": 2.9583,
"step": 91700
},
{
"epoch": 24.690676141257537,
"grad_norm": 0.4490005671977997,
"learning_rate": 0.00022980072707688161,
"loss": 2.9628,
"step": 91750
},
{
"epoch": 24.704134366925064,
"grad_norm": 0.546238362789154,
"learning_rate": 0.00022959876127642384,
"loss": 2.973,
"step": 91800
},
{
"epoch": 24.71759259259259,
"grad_norm": 0.43302252888679504,
"learning_rate": 0.00022939679547596603,
"loss": 2.9686,
"step": 91850
},
{
"epoch": 24.73105081826012,
"grad_norm": 0.47013354301452637,
"learning_rate": 0.00022919482967550828,
"loss": 2.9526,
"step": 91900
},
{
"epoch": 24.74450904392765,
"grad_norm": 0.4680594801902771,
"learning_rate": 0.00022899286387505048,
"loss": 2.9645,
"step": 91950
},
{
"epoch": 24.757967269595177,
"grad_norm": 0.5106493234634399,
"learning_rate": 0.00022879089807459268,
"loss": 2.9761,
"step": 92000
},
{
"epoch": 24.757967269595177,
"eval_accuracy": 0.3969161893810302,
"eval_loss": 3.279923439025879,
"eval_runtime": 83.9591,
"eval_samples_per_second": 214.533,
"eval_steps_per_second": 13.411,
"step": 92000
},
{
"epoch": 24.771425495262704,
"grad_norm": 0.45333370566368103,
"learning_rate": 0.0002285889322741349,
"loss": 2.9641,
"step": 92050
},
{
"epoch": 24.78488372093023,
"grad_norm": 0.48366883397102356,
"learning_rate": 0.0002283869664736771,
"loss": 2.967,
"step": 92100
},
{
"epoch": 24.79834194659776,
"grad_norm": 0.46147432923316956,
"learning_rate": 0.00022818500067321932,
"loss": 2.9815,
"step": 92150
},
{
"epoch": 24.81180017226529,
"grad_norm": 0.4639972150325775,
"learning_rate": 0.00022798303487276151,
"loss": 2.9669,
"step": 92200
},
{
"epoch": 24.825258397932817,
"grad_norm": 0.45889705419540405,
"learning_rate": 0.0002277810690723037,
"loss": 2.9811,
"step": 92250
},
{
"epoch": 24.838716623600344,
"grad_norm": 0.4609624445438385,
"learning_rate": 0.00022757910327184596,
"loss": 2.979,
"step": 92300
},
{
"epoch": 24.852174849267872,
"grad_norm": 0.45697692036628723,
"learning_rate": 0.00022737713747138818,
"loss": 2.9789,
"step": 92350
},
{
"epoch": 24.8656330749354,
"grad_norm": 0.45627814531326294,
"learning_rate": 0.00022717517167093038,
"loss": 2.9801,
"step": 92400
},
{
"epoch": 24.87909130060293,
"grad_norm": 0.4848513901233673,
"learning_rate": 0.00022697320587047258,
"loss": 2.9808,
"step": 92450
},
{
"epoch": 24.892549526270457,
"grad_norm": 0.44247519969940186,
"learning_rate": 0.0002267712400700148,
"loss": 2.9763,
"step": 92500
},
{
"epoch": 24.906007751937985,
"grad_norm": 0.45784133672714233,
"learning_rate": 0.000226569274269557,
"loss": 2.9812,
"step": 92550
},
{
"epoch": 24.919465977605512,
"grad_norm": 0.45247259736061096,
"learning_rate": 0.00022636730846909922,
"loss": 2.9914,
"step": 92600
},
{
"epoch": 24.93292420327304,
"grad_norm": 0.48047441244125366,
"learning_rate": 0.00022616534266864141,
"loss": 2.983,
"step": 92650
},
{
"epoch": 24.94638242894057,
"grad_norm": 0.5039317011833191,
"learning_rate": 0.0002259633768681836,
"loss": 2.9788,
"step": 92700
},
{
"epoch": 24.959840654608097,
"grad_norm": 0.4445413053035736,
"learning_rate": 0.00022576141106772586,
"loss": 2.9792,
"step": 92750
},
{
"epoch": 24.973298880275625,
"grad_norm": 0.4654720425605774,
"learning_rate": 0.00022555944526726806,
"loss": 2.9884,
"step": 92800
},
{
"epoch": 24.986757105943152,
"grad_norm": 0.4949401021003723,
"learning_rate": 0.00022535747946681028,
"loss": 2.9895,
"step": 92850
},
{
"epoch": 25.0,
"grad_norm": 0.9609937071800232,
"learning_rate": 0.00022515551366635248,
"loss": 2.9747,
"step": 92900
},
{
"epoch": 25.013458225667527,
"grad_norm": 0.4609181582927704,
"learning_rate": 0.0002249535478658947,
"loss": 2.8994,
"step": 92950
},
{
"epoch": 25.026916451335055,
"grad_norm": 0.4751029312610626,
"learning_rate": 0.0002247515820654369,
"loss": 2.8999,
"step": 93000
},
{
"epoch": 25.026916451335055,
"eval_accuracy": 0.3965985059753598,
"eval_loss": 3.2856178283691406,
"eval_runtime": 83.9914,
"eval_samples_per_second": 214.45,
"eval_steps_per_second": 13.406,
"step": 93000
},
{
"epoch": 25.040374677002585,
"grad_norm": 0.4698486924171448,
"learning_rate": 0.0002245496162649791,
"loss": 2.8946,
"step": 93050
},
{
"epoch": 25.053832902670113,
"grad_norm": 0.4572475254535675,
"learning_rate": 0.00022434765046452131,
"loss": 2.9131,
"step": 93100
},
{
"epoch": 25.06729112833764,
"grad_norm": 0.4726247191429138,
"learning_rate": 0.0002241456846640635,
"loss": 2.9103,
"step": 93150
},
{
"epoch": 25.080749354005167,
"grad_norm": 0.4614805579185486,
"learning_rate": 0.00022394371886360576,
"loss": 2.9139,
"step": 93200
},
{
"epoch": 25.094207579672695,
"grad_norm": 0.4564429819583893,
"learning_rate": 0.00022374175306314796,
"loss": 2.911,
"step": 93250
},
{
"epoch": 25.107665805340226,
"grad_norm": 0.4605506658554077,
"learning_rate": 0.00022353978726269018,
"loss": 2.916,
"step": 93300
},
{
"epoch": 25.121124031007753,
"grad_norm": 0.4785519540309906,
"learning_rate": 0.00022333782146223238,
"loss": 2.9163,
"step": 93350
},
{
"epoch": 25.13458225667528,
"grad_norm": 0.46773386001586914,
"learning_rate": 0.0002231358556617746,
"loss": 2.9313,
"step": 93400
},
{
"epoch": 25.148040482342807,
"grad_norm": 0.44259631633758545,
"learning_rate": 0.0002229338898613168,
"loss": 2.9122,
"step": 93450
},
{
"epoch": 25.161498708010335,
"grad_norm": 0.4517664909362793,
"learning_rate": 0.000222731924060859,
"loss": 2.923,
"step": 93500
},
{
"epoch": 25.174956933677866,
"grad_norm": 0.46426427364349365,
"learning_rate": 0.00022252995826040122,
"loss": 2.9165,
"step": 93550
},
{
"epoch": 25.188415159345393,
"grad_norm": 0.48092758655548096,
"learning_rate": 0.0002223279924599434,
"loss": 2.91,
"step": 93600
},
{
"epoch": 25.20187338501292,
"grad_norm": 0.4678271412849426,
"learning_rate": 0.00022212602665948566,
"loss": 2.9233,
"step": 93650
},
{
"epoch": 25.215331610680447,
"grad_norm": 0.45270681381225586,
"learning_rate": 0.00022192406085902786,
"loss": 2.9271,
"step": 93700
},
{
"epoch": 25.228789836347975,
"grad_norm": 0.5142789483070374,
"learning_rate": 0.00022172209505857008,
"loss": 2.9075,
"step": 93750
},
{
"epoch": 25.242248062015506,
"grad_norm": 0.5167204141616821,
"learning_rate": 0.00022152012925811228,
"loss": 2.9228,
"step": 93800
},
{
"epoch": 25.255706287683033,
"grad_norm": 0.4759088158607483,
"learning_rate": 0.00022131816345765447,
"loss": 2.9355,
"step": 93850
},
{
"epoch": 25.26916451335056,
"grad_norm": 0.4936668872833252,
"learning_rate": 0.0002211161976571967,
"loss": 2.9284,
"step": 93900
},
{
"epoch": 25.282622739018088,
"grad_norm": 0.48341456055641174,
"learning_rate": 0.0002209142318567389,
"loss": 2.9428,
"step": 93950
},
{
"epoch": 25.296080964685615,
"grad_norm": 0.4980866611003876,
"learning_rate": 0.00022071226605628112,
"loss": 2.9318,
"step": 94000
},
{
"epoch": 25.296080964685615,
"eval_accuracy": 0.3968106932842771,
"eval_loss": 3.2875349521636963,
"eval_runtime": 85.4169,
"eval_samples_per_second": 210.872,
"eval_steps_per_second": 13.182,
"step": 94000
},
{
"epoch": 25.309539190353142,
"grad_norm": 0.47112831473350525,
"learning_rate": 0.0002205103002558233,
"loss": 2.9312,
"step": 94050
},
{
"epoch": 25.322997416020673,
"grad_norm": 0.4677145779132843,
"learning_rate": 0.00022030833445536556,
"loss": 2.925,
"step": 94100
},
{
"epoch": 25.3364556416882,
"grad_norm": 0.47816231846809387,
"learning_rate": 0.00022010636865490776,
"loss": 2.9364,
"step": 94150
},
{
"epoch": 25.349913867355728,
"grad_norm": 0.5191966891288757,
"learning_rate": 0.00021990440285444998,
"loss": 2.9393,
"step": 94200
},
{
"epoch": 25.363372093023255,
"grad_norm": 0.49411892890930176,
"learning_rate": 0.00021970243705399218,
"loss": 2.9502,
"step": 94250
},
{
"epoch": 25.376830318690782,
"grad_norm": 0.4911155700683594,
"learning_rate": 0.00021950047125353437,
"loss": 2.9462,
"step": 94300
},
{
"epoch": 25.390288544358313,
"grad_norm": 0.47045525908470154,
"learning_rate": 0.0002192985054530766,
"loss": 2.9318,
"step": 94350
},
{
"epoch": 25.40374677002584,
"grad_norm": 0.47992491722106934,
"learning_rate": 0.0002190965396526188,
"loss": 2.9352,
"step": 94400
},
{
"epoch": 25.417204995693368,
"grad_norm": 0.45442041754722595,
"learning_rate": 0.00021889457385216102,
"loss": 2.9349,
"step": 94450
},
{
"epoch": 25.430663221360895,
"grad_norm": 0.481173038482666,
"learning_rate": 0.0002186926080517032,
"loss": 2.9338,
"step": 94500
},
{
"epoch": 25.444121447028422,
"grad_norm": 0.49459052085876465,
"learning_rate": 0.00021849064225124546,
"loss": 2.9449,
"step": 94550
},
{
"epoch": 25.457579672695953,
"grad_norm": 0.4651554524898529,
"learning_rate": 0.00021828867645078766,
"loss": 2.9481,
"step": 94600
},
{
"epoch": 25.47103789836348,
"grad_norm": 0.45628347992897034,
"learning_rate": 0.00021808671065032985,
"loss": 2.9565,
"step": 94650
},
{
"epoch": 25.484496124031008,
"grad_norm": 0.45765358209609985,
"learning_rate": 0.00021788474484987208,
"loss": 2.9551,
"step": 94700
},
{
"epoch": 25.497954349698535,
"grad_norm": 0.4496651589870453,
"learning_rate": 0.00021768277904941427,
"loss": 2.9388,
"step": 94750
},
{
"epoch": 25.511412575366062,
"grad_norm": 0.46032917499542236,
"learning_rate": 0.0002174808132489565,
"loss": 2.9494,
"step": 94800
},
{
"epoch": 25.524870801033593,
"grad_norm": 0.4763024151325226,
"learning_rate": 0.0002172788474484987,
"loss": 2.9531,
"step": 94850
},
{
"epoch": 25.53832902670112,
"grad_norm": 0.48358646035194397,
"learning_rate": 0.0002170768816480409,
"loss": 2.9531,
"step": 94900
},
{
"epoch": 25.551787252368648,
"grad_norm": 0.4779926836490631,
"learning_rate": 0.0002168749158475831,
"loss": 2.9441,
"step": 94950
},
{
"epoch": 25.565245478036175,
"grad_norm": 0.4780193567276001,
"learning_rate": 0.00021667295004712534,
"loss": 2.9437,
"step": 95000
},
{
"epoch": 25.565245478036175,
"eval_accuracy": 0.39687327387308224,
"eval_loss": 3.283360242843628,
"eval_runtime": 83.7458,
"eval_samples_per_second": 215.079,
"eval_steps_per_second": 13.445,
"step": 95000
},
{
"epoch": 25.578703703703702,
"grad_norm": 0.488501638174057,
"learning_rate": 0.00021647098424666756,
"loss": 2.9528,
"step": 95050
},
{
"epoch": 25.592161929371233,
"grad_norm": 0.44475841522216797,
"learning_rate": 0.00021626901844620976,
"loss": 2.9496,
"step": 95100
},
{
"epoch": 25.60562015503876,
"grad_norm": 0.480144739151001,
"learning_rate": 0.00021606705264575198,
"loss": 2.9611,
"step": 95150
},
{
"epoch": 25.619078380706288,
"grad_norm": 0.44786661863327026,
"learning_rate": 0.00021586508684529417,
"loss": 2.9509,
"step": 95200
},
{
"epoch": 25.632536606373815,
"grad_norm": 0.45520010590553284,
"learning_rate": 0.0002156631210448364,
"loss": 2.9417,
"step": 95250
},
{
"epoch": 25.645994832041342,
"grad_norm": 0.4738970994949341,
"learning_rate": 0.0002154611552443786,
"loss": 2.9596,
"step": 95300
},
{
"epoch": 25.659453057708873,
"grad_norm": 0.4593546986579895,
"learning_rate": 0.0002152591894439208,
"loss": 2.9519,
"step": 95350
},
{
"epoch": 25.6729112833764,
"grad_norm": 0.46724069118499756,
"learning_rate": 0.000215057223643463,
"loss": 2.9491,
"step": 95400
},
{
"epoch": 25.686369509043928,
"grad_norm": 0.48032504320144653,
"learning_rate": 0.00021485525784300524,
"loss": 2.9718,
"step": 95450
},
{
"epoch": 25.699827734711455,
"grad_norm": 0.46497252583503723,
"learning_rate": 0.00021465329204254746,
"loss": 2.9629,
"step": 95500
},
{
"epoch": 25.713285960378983,
"grad_norm": 0.453347772359848,
"learning_rate": 0.00021445132624208966,
"loss": 2.9635,
"step": 95550
},
{
"epoch": 25.72674418604651,
"grad_norm": 0.46574610471725464,
"learning_rate": 0.00021424936044163188,
"loss": 2.9652,
"step": 95600
},
{
"epoch": 25.74020241171404,
"grad_norm": 0.47337067127227783,
"learning_rate": 0.00021404739464117407,
"loss": 2.9635,
"step": 95650
},
{
"epoch": 25.753660637381568,
"grad_norm": 0.45252326130867004,
"learning_rate": 0.00021384542884071627,
"loss": 2.9683,
"step": 95700
},
{
"epoch": 25.767118863049095,
"grad_norm": 0.4818001985549927,
"learning_rate": 0.0002136434630402585,
"loss": 2.9698,
"step": 95750
},
{
"epoch": 25.780577088716623,
"grad_norm": 0.46499624848365784,
"learning_rate": 0.0002134414972398007,
"loss": 2.9622,
"step": 95800
},
{
"epoch": 25.794035314384153,
"grad_norm": 0.4897556006908417,
"learning_rate": 0.0002132395314393429,
"loss": 2.965,
"step": 95850
},
{
"epoch": 25.80749354005168,
"grad_norm": 0.4888896644115448,
"learning_rate": 0.00021303756563888514,
"loss": 2.9696,
"step": 95900
},
{
"epoch": 25.820951765719208,
"grad_norm": 0.46453386545181274,
"learning_rate": 0.00021283559983842736,
"loss": 2.9519,
"step": 95950
},
{
"epoch": 25.834409991386735,
"grad_norm": 0.4988659918308258,
"learning_rate": 0.00021263363403796956,
"loss": 2.9606,
"step": 96000
},
{
"epoch": 25.834409991386735,
"eval_accuracy": 0.3976640056878802,
"eval_loss": 3.2750282287597656,
"eval_runtime": 83.8338,
"eval_samples_per_second": 214.854,
"eval_steps_per_second": 13.431,
"step": 96000
},
{
"epoch": 25.847868217054263,
"grad_norm": 0.5047129392623901,
"learning_rate": 0.00021243166823751175,
"loss": 2.959,
"step": 96050
},
{
"epoch": 25.86132644272179,
"grad_norm": 0.4605954885482788,
"learning_rate": 0.00021222970243705398,
"loss": 2.9571,
"step": 96100
},
{
"epoch": 25.87478466838932,
"grad_norm": 0.4731179475784302,
"learning_rate": 0.00021202773663659617,
"loss": 2.9732,
"step": 96150
},
{
"epoch": 25.888242894056848,
"grad_norm": 0.49822744727134705,
"learning_rate": 0.0002118257708361384,
"loss": 2.9623,
"step": 96200
},
{
"epoch": 25.901701119724375,
"grad_norm": 0.46090516448020935,
"learning_rate": 0.0002116238050356806,
"loss": 2.9764,
"step": 96250
},
{
"epoch": 25.915159345391903,
"grad_norm": 0.4802016019821167,
"learning_rate": 0.00021142183923522281,
"loss": 2.9673,
"step": 96300
},
{
"epoch": 25.92861757105943,
"grad_norm": 0.477112740278244,
"learning_rate": 0.00021121987343476504,
"loss": 2.9716,
"step": 96350
},
{
"epoch": 25.94207579672696,
"grad_norm": 0.4376496970653534,
"learning_rate": 0.00021101790763430726,
"loss": 2.9672,
"step": 96400
},
{
"epoch": 25.955534022394488,
"grad_norm": 0.45158451795578003,
"learning_rate": 0.00021081594183384946,
"loss": 2.9593,
"step": 96450
},
{
"epoch": 25.968992248062015,
"grad_norm": 0.5054965615272522,
"learning_rate": 0.00021061397603339165,
"loss": 2.968,
"step": 96500
},
{
"epoch": 25.982450473729543,
"grad_norm": 0.46955764293670654,
"learning_rate": 0.00021041201023293388,
"loss": 2.9688,
"step": 96550
},
{
"epoch": 25.99590869939707,
"grad_norm": 0.47011783719062805,
"learning_rate": 0.00021021004443247607,
"loss": 2.9778,
"step": 96600
},
{
"epoch": 26.009151593453918,
"grad_norm": 0.45663025975227356,
"learning_rate": 0.0002100080786320183,
"loss": 2.9147,
"step": 96650
},
{
"epoch": 26.022609819121445,
"grad_norm": 0.4516238272190094,
"learning_rate": 0.0002098061128315605,
"loss": 2.8801,
"step": 96700
},
{
"epoch": 26.036068044788976,
"grad_norm": 0.5051943063735962,
"learning_rate": 0.00020960414703110274,
"loss": 2.8837,
"step": 96750
},
{
"epoch": 26.049526270456504,
"grad_norm": 0.4774962365627289,
"learning_rate": 0.00020940218123064494,
"loss": 2.896,
"step": 96800
},
{
"epoch": 26.06298449612403,
"grad_norm": 0.4575546681880951,
"learning_rate": 0.00020920021543018713,
"loss": 2.9035,
"step": 96850
},
{
"epoch": 26.076442721791558,
"grad_norm": 0.4669003188610077,
"learning_rate": 0.00020899824962972936,
"loss": 2.8937,
"step": 96900
},
{
"epoch": 26.089900947459086,
"grad_norm": 0.47874999046325684,
"learning_rate": 0.00020879628382927155,
"loss": 2.9065,
"step": 96950
},
{
"epoch": 26.103359173126616,
"grad_norm": 0.45106983184814453,
"learning_rate": 0.00020859431802881378,
"loss": 2.9163,
"step": 97000
},
{
"epoch": 26.103359173126616,
"eval_accuracy": 0.39677027665400716,
"eval_loss": 3.2907752990722656,
"eval_runtime": 83.7697,
"eval_samples_per_second": 215.018,
"eval_steps_per_second": 13.442,
"step": 97000
},
{
"epoch": 26.116817398794144,
"grad_norm": 0.46213534474372864,
"learning_rate": 0.00020839235222835597,
"loss": 2.9136,
"step": 97050
},
{
"epoch": 26.13027562446167,
"grad_norm": 0.47658196091651917,
"learning_rate": 0.00020819038642789817,
"loss": 2.908,
"step": 97100
},
{
"epoch": 26.1437338501292,
"grad_norm": 0.4760020971298218,
"learning_rate": 0.0002079884206274404,
"loss": 2.9077,
"step": 97150
},
{
"epoch": 26.157192075796726,
"grad_norm": 0.5179885625839233,
"learning_rate": 0.00020778645482698264,
"loss": 2.9048,
"step": 97200
},
{
"epoch": 26.170650301464256,
"grad_norm": 0.46104031801223755,
"learning_rate": 0.00020758448902652484,
"loss": 2.9132,
"step": 97250
},
{
"epoch": 26.184108527131784,
"grad_norm": 0.48797598481178284,
"learning_rate": 0.00020738252322606703,
"loss": 2.927,
"step": 97300
},
{
"epoch": 26.19756675279931,
"grad_norm": 0.47429800033569336,
"learning_rate": 0.00020718055742560926,
"loss": 2.9094,
"step": 97350
},
{
"epoch": 26.21102497846684,
"grad_norm": 0.4832933843135834,
"learning_rate": 0.00020697859162515145,
"loss": 2.9134,
"step": 97400
},
{
"epoch": 26.224483204134366,
"grad_norm": 0.46741122007369995,
"learning_rate": 0.00020677662582469368,
"loss": 2.9204,
"step": 97450
},
{
"epoch": 26.237941429801896,
"grad_norm": 0.44947007298469543,
"learning_rate": 0.00020657466002423587,
"loss": 2.9185,
"step": 97500
},
{
"epoch": 26.251399655469424,
"grad_norm": 0.5112219452857971,
"learning_rate": 0.00020637269422377807,
"loss": 2.9169,
"step": 97550
},
{
"epoch": 26.26485788113695,
"grad_norm": 0.47085320949554443,
"learning_rate": 0.0002061707284233203,
"loss": 2.9181,
"step": 97600
},
{
"epoch": 26.27831610680448,
"grad_norm": 0.46664902567863464,
"learning_rate": 0.00020596876262286251,
"loss": 2.9291,
"step": 97650
},
{
"epoch": 26.291774332472006,
"grad_norm": 0.46194952726364136,
"learning_rate": 0.00020576679682240474,
"loss": 2.9217,
"step": 97700
},
{
"epoch": 26.305232558139537,
"grad_norm": 0.46810612082481384,
"learning_rate": 0.00020556483102194693,
"loss": 2.9271,
"step": 97750
},
{
"epoch": 26.318690783807064,
"grad_norm": 0.47608909010887146,
"learning_rate": 0.00020536286522148916,
"loss": 2.9223,
"step": 97800
},
{
"epoch": 26.33214900947459,
"grad_norm": 0.487276166677475,
"learning_rate": 0.00020516089942103135,
"loss": 2.9229,
"step": 97850
},
{
"epoch": 26.34560723514212,
"grad_norm": 0.4970484972000122,
"learning_rate": 0.00020495893362057355,
"loss": 2.9228,
"step": 97900
},
{
"epoch": 26.359065460809646,
"grad_norm": 0.47998833656311035,
"learning_rate": 0.00020475696782011577,
"loss": 2.9395,
"step": 97950
},
{
"epoch": 26.372523686477177,
"grad_norm": 0.468665212392807,
"learning_rate": 0.00020455500201965797,
"loss": 2.9334,
"step": 98000
},
{
"epoch": 26.372523686477177,
"eval_accuracy": 0.39712066276320246,
"eval_loss": 3.2837681770324707,
"eval_runtime": 83.6485,
"eval_samples_per_second": 215.33,
"eval_steps_per_second": 13.461,
"step": 98000
},
{
"epoch": 26.385981912144704,
"grad_norm": 0.4812726080417633,
"learning_rate": 0.0002043530362192002,
"loss": 2.9203,
"step": 98050
},
{
"epoch": 26.39944013781223,
"grad_norm": 0.47966867685317993,
"learning_rate": 0.00020415107041874242,
"loss": 2.9386,
"step": 98100
},
{
"epoch": 26.41289836347976,
"grad_norm": 0.4546014666557312,
"learning_rate": 0.00020394910461828464,
"loss": 2.9366,
"step": 98150
},
{
"epoch": 26.426356589147286,
"grad_norm": 0.4557799994945526,
"learning_rate": 0.00020374713881782683,
"loss": 2.9384,
"step": 98200
},
{
"epoch": 26.439814814814813,
"grad_norm": 0.5014848709106445,
"learning_rate": 0.00020354517301736906,
"loss": 2.9375,
"step": 98250
},
{
"epoch": 26.453273040482344,
"grad_norm": 0.49908778071403503,
"learning_rate": 0.00020334320721691125,
"loss": 2.9293,
"step": 98300
},
{
"epoch": 26.46673126614987,
"grad_norm": 0.49170541763305664,
"learning_rate": 0.00020314124141645345,
"loss": 2.9444,
"step": 98350
},
{
"epoch": 26.4801894918174,
"grad_norm": 0.4941914975643158,
"learning_rate": 0.00020293927561599567,
"loss": 2.9318,
"step": 98400
},
{
"epoch": 26.493647717484926,
"grad_norm": 0.4966713488101959,
"learning_rate": 0.00020273730981553787,
"loss": 2.9447,
"step": 98450
},
{
"epoch": 26.507105943152453,
"grad_norm": 0.5141641497612,
"learning_rate": 0.0002025353440150801,
"loss": 2.9441,
"step": 98500
},
{
"epoch": 26.520564168819984,
"grad_norm": 0.4713413715362549,
"learning_rate": 0.00020233337821462232,
"loss": 2.9434,
"step": 98550
},
{
"epoch": 26.53402239448751,
"grad_norm": 0.4796540439128876,
"learning_rate": 0.00020213141241416454,
"loss": 2.9414,
"step": 98600
},
{
"epoch": 26.54748062015504,
"grad_norm": 0.4777824580669403,
"learning_rate": 0.00020192944661370674,
"loss": 2.9441,
"step": 98650
},
{
"epoch": 26.560938845822566,
"grad_norm": 0.48075199127197266,
"learning_rate": 0.00020172748081324893,
"loss": 2.9466,
"step": 98700
},
{
"epoch": 26.574397071490093,
"grad_norm": 0.4639039933681488,
"learning_rate": 0.00020152551501279115,
"loss": 2.9362,
"step": 98750
},
{
"epoch": 26.587855297157624,
"grad_norm": 0.48601824045181274,
"learning_rate": 0.00020132354921233335,
"loss": 2.9426,
"step": 98800
},
{
"epoch": 26.60131352282515,
"grad_norm": 0.4678082764148712,
"learning_rate": 0.00020112158341187557,
"loss": 2.9364,
"step": 98850
},
{
"epoch": 26.61477174849268,
"grad_norm": 0.4817858338356018,
"learning_rate": 0.00020091961761141777,
"loss": 2.94,
"step": 98900
},
{
"epoch": 26.628229974160206,
"grad_norm": 0.46154728531837463,
"learning_rate": 0.00020071765181095997,
"loss": 2.9263,
"step": 98950
},
{
"epoch": 26.641688199827733,
"grad_norm": 0.45043495297431946,
"learning_rate": 0.00020051568601050222,
"loss": 2.9322,
"step": 99000
},
{
"epoch": 26.641688199827733,
"eval_accuracy": 0.3972260502130999,
"eval_loss": 3.278003454208374,
"eval_runtime": 84.0483,
"eval_samples_per_second": 214.305,
"eval_steps_per_second": 13.397,
"step": 99000
},
{
"epoch": 26.655146425495264,
"grad_norm": 0.5009729266166687,
"learning_rate": 0.00020031372021004444,
"loss": 2.9577,
"step": 99050
},
{
"epoch": 26.66860465116279,
"grad_norm": 0.44591593742370605,
"learning_rate": 0.00020011175440958664,
"loss": 2.9448,
"step": 99100
},
{
"epoch": 26.68206287683032,
"grad_norm": 0.4795645773410797,
"learning_rate": 0.00019990978860912883,
"loss": 2.9498,
"step": 99150
},
{
"epoch": 26.695521102497846,
"grad_norm": 0.5004894733428955,
"learning_rate": 0.00019970782280867105,
"loss": 2.9477,
"step": 99200
},
{
"epoch": 26.708979328165373,
"grad_norm": 0.47407266497612,
"learning_rate": 0.00019950585700821325,
"loss": 2.9451,
"step": 99250
},
{
"epoch": 26.722437553832904,
"grad_norm": 0.4809727668762207,
"learning_rate": 0.00019930389120775547,
"loss": 2.9432,
"step": 99300
},
{
"epoch": 26.73589577950043,
"grad_norm": 0.4729549288749695,
"learning_rate": 0.00019910192540729767,
"loss": 2.9483,
"step": 99350
},
{
"epoch": 26.74935400516796,
"grad_norm": 0.489544540643692,
"learning_rate": 0.00019889995960683987,
"loss": 2.9468,
"step": 99400
},
{
"epoch": 26.762812230835486,
"grad_norm": 0.4922120273113251,
"learning_rate": 0.00019869799380638212,
"loss": 2.9615,
"step": 99450
},
{
"epoch": 26.776270456503013,
"grad_norm": 0.47596344351768494,
"learning_rate": 0.0001984960280059243,
"loss": 2.9431,
"step": 99500
},
{
"epoch": 26.789728682170544,
"grad_norm": 0.5055460333824158,
"learning_rate": 0.00019829406220546654,
"loss": 2.9551,
"step": 99550
},
{
"epoch": 26.80318690783807,
"grad_norm": 0.4794882833957672,
"learning_rate": 0.00019809209640500873,
"loss": 2.9575,
"step": 99600
},
{
"epoch": 26.8166451335056,
"grad_norm": 0.4809652864933014,
"learning_rate": 0.00019789013060455096,
"loss": 2.9513,
"step": 99650
},
{
"epoch": 26.830103359173126,
"grad_norm": 0.44904300570487976,
"learning_rate": 0.00019768816480409315,
"loss": 2.9546,
"step": 99700
},
{
"epoch": 26.843561584840653,
"grad_norm": 0.5154889225959778,
"learning_rate": 0.00019748619900363535,
"loss": 2.9653,
"step": 99750
},
{
"epoch": 26.857019810508184,
"grad_norm": 0.528717577457428,
"learning_rate": 0.00019728423320317757,
"loss": 2.9559,
"step": 99800
},
{
"epoch": 26.87047803617571,
"grad_norm": 0.4803217053413391,
"learning_rate": 0.00019708226740271977,
"loss": 2.9515,
"step": 99850
},
{
"epoch": 26.88393626184324,
"grad_norm": 0.4940747320652008,
"learning_rate": 0.00019688030160226202,
"loss": 2.9521,
"step": 99900
},
{
"epoch": 26.897394487510766,
"grad_norm": 0.46168291568756104,
"learning_rate": 0.0001966783358018042,
"loss": 2.9596,
"step": 99950
},
{
"epoch": 26.910852713178294,
"grad_norm": 0.5004278421401978,
"learning_rate": 0.00019647637000134644,
"loss": 2.9657,
"step": 100000
},
{
"epoch": 26.910852713178294,
"eval_accuracy": 0.3976188085959654,
"eval_loss": 3.275502920150757,
"eval_runtime": 84.0403,
"eval_samples_per_second": 214.326,
"eval_steps_per_second": 13.398,
"step": 100000
},
{
"epoch": 26.92431093884582,
"grad_norm": 0.4664307236671448,
"learning_rate": 0.00019627440420088863,
"loss": 2.9633,
"step": 100050
},
{
"epoch": 26.93776916451335,
"grad_norm": 0.492509126663208,
"learning_rate": 0.00019607243840043086,
"loss": 2.9557,
"step": 100100
},
{
"epoch": 26.95122739018088,
"grad_norm": 0.5034955739974976,
"learning_rate": 0.00019587047259997305,
"loss": 2.9548,
"step": 100150
},
{
"epoch": 26.964685615848406,
"grad_norm": 0.48850440979003906,
"learning_rate": 0.00019566850679951525,
"loss": 2.9565,
"step": 100200
},
{
"epoch": 26.978143841515934,
"grad_norm": 0.45618629455566406,
"learning_rate": 0.00019546654099905747,
"loss": 2.9572,
"step": 100250
},
{
"epoch": 26.99160206718346,
"grad_norm": 0.4687550365924835,
"learning_rate": 0.00019526457519859967,
"loss": 2.9539,
"step": 100300
},
{
"epoch": 27.00484496124031,
"grad_norm": 0.5012471675872803,
"learning_rate": 0.00019506260939814192,
"loss": 2.9226,
"step": 100350
},
{
"epoch": 27.01830318690784,
"grad_norm": 0.4801206886768341,
"learning_rate": 0.0001948606435976841,
"loss": 2.8866,
"step": 100400
},
{
"epoch": 27.031761412575367,
"grad_norm": 0.49274516105651855,
"learning_rate": 0.00019465867779722634,
"loss": 2.8789,
"step": 100450
},
{
"epoch": 27.045219638242894,
"grad_norm": 0.46303367614746094,
"learning_rate": 0.00019445671199676853,
"loss": 2.8867,
"step": 100500
},
{
"epoch": 27.05867786391042,
"grad_norm": 0.47992032766342163,
"learning_rate": 0.00019425474619631073,
"loss": 2.8882,
"step": 100550
},
{
"epoch": 27.07213608957795,
"grad_norm": 0.48844286799430847,
"learning_rate": 0.00019405278039585295,
"loss": 2.8891,
"step": 100600
},
{
"epoch": 27.085594315245476,
"grad_norm": 0.47748664021492004,
"learning_rate": 0.00019385081459539515,
"loss": 2.8819,
"step": 100650
},
{
"epoch": 27.099052540913007,
"grad_norm": 0.49191465973854065,
"learning_rate": 0.00019364884879493737,
"loss": 2.8856,
"step": 100700
},
{
"epoch": 27.112510766580534,
"grad_norm": 0.49035146832466125,
"learning_rate": 0.00019344688299447957,
"loss": 2.9045,
"step": 100750
},
{
"epoch": 27.125968992248062,
"grad_norm": 0.492626428604126,
"learning_rate": 0.00019324491719402182,
"loss": 2.907,
"step": 100800
},
{
"epoch": 27.13942721791559,
"grad_norm": 0.4873746931552887,
"learning_rate": 0.00019304295139356401,
"loss": 2.9093,
"step": 100850
},
{
"epoch": 27.152885443583116,
"grad_norm": 0.48530182242393494,
"learning_rate": 0.0001928409855931062,
"loss": 2.9079,
"step": 100900
},
{
"epoch": 27.166343669250647,
"grad_norm": 0.5125190019607544,
"learning_rate": 0.00019263901979264843,
"loss": 2.8998,
"step": 100950
},
{
"epoch": 27.179801894918175,
"grad_norm": 0.46221446990966797,
"learning_rate": 0.00019243705399219063,
"loss": 2.9095,
"step": 101000
},
{
"epoch": 27.179801894918175,
"eval_accuracy": 0.39690793222000725,
"eval_loss": 3.28844952583313,
"eval_runtime": 84.0707,
"eval_samples_per_second": 214.248,
"eval_steps_per_second": 13.393,
"step": 101000
},
{
"epoch": 27.193260120585702,
"grad_norm": 0.49618521332740784,
"learning_rate": 0.00019223508819173285,
"loss": 2.908,
"step": 101050
},
{
"epoch": 27.20671834625323,
"grad_norm": 0.47813424468040466,
"learning_rate": 0.00019203312239127505,
"loss": 2.8946,
"step": 101100
},
{
"epoch": 27.220176571920756,
"grad_norm": 0.4884779453277588,
"learning_rate": 0.00019183115659081727,
"loss": 2.9174,
"step": 101150
},
{
"epoch": 27.233634797588287,
"grad_norm": 0.5239505767822266,
"learning_rate": 0.00019162919079035947,
"loss": 2.9165,
"step": 101200
},
{
"epoch": 27.247093023255815,
"grad_norm": 0.4790191352367401,
"learning_rate": 0.00019142722498990172,
"loss": 2.9176,
"step": 101250
},
{
"epoch": 27.260551248923342,
"grad_norm": 0.5082780122756958,
"learning_rate": 0.00019122525918944391,
"loss": 2.9076,
"step": 101300
},
{
"epoch": 27.27400947459087,
"grad_norm": 0.4764711260795593,
"learning_rate": 0.0001910232933889861,
"loss": 2.9146,
"step": 101350
},
{
"epoch": 27.287467700258397,
"grad_norm": 0.46437740325927734,
"learning_rate": 0.00019082132758852833,
"loss": 2.9212,
"step": 101400
},
{
"epoch": 27.300925925925927,
"grad_norm": 0.4945138394832611,
"learning_rate": 0.00019061936178807053,
"loss": 2.9105,
"step": 101450
},
{
"epoch": 27.314384151593455,
"grad_norm": 0.5206960439682007,
"learning_rate": 0.00019041739598761275,
"loss": 2.9113,
"step": 101500
},
{
"epoch": 27.327842377260982,
"grad_norm": 0.5159957408905029,
"learning_rate": 0.00019021543018715495,
"loss": 2.9172,
"step": 101550
},
{
"epoch": 27.34130060292851,
"grad_norm": Infinity,
"learning_rate": 0.00019001346438669714,
"loss": 2.9163,
"step": 101600
},
{
"epoch": 27.354758828596037,
"grad_norm": 0.48938295245170593,
"learning_rate": 0.0001898114985862394,
"loss": 2.9308,
"step": 101650
},
{
"epoch": 27.368217054263567,
"grad_norm": 0.49014759063720703,
"learning_rate": 0.0001896095327857816,
"loss": 2.9108,
"step": 101700
},
{
"epoch": 27.381675279931095,
"grad_norm": 0.4875050187110901,
"learning_rate": 0.00018940756698532381,
"loss": 2.9145,
"step": 101750
},
{
"epoch": 27.395133505598622,
"grad_norm": 0.4899139106273651,
"learning_rate": 0.000189205601184866,
"loss": 2.9245,
"step": 101800
},
{
"epoch": 27.40859173126615,
"grad_norm": 0.4790636897087097,
"learning_rate": 0.00018900363538440823,
"loss": 2.9169,
"step": 101850
},
{
"epoch": 27.422049956933677,
"grad_norm": 0.4836386740207672,
"learning_rate": 0.00018880166958395043,
"loss": 2.9226,
"step": 101900
},
{
"epoch": 27.435508182601207,
"grad_norm": 0.5054500699043274,
"learning_rate": 0.00018859970378349263,
"loss": 2.9227,
"step": 101950
},
{
"epoch": 27.448966408268735,
"grad_norm": 0.4658251702785492,
"learning_rate": 0.00018839773798303485,
"loss": 2.9236,
"step": 102000
},
{
"epoch": 27.448966408268735,
"eval_accuracy": 0.39734056399886486,
"eval_loss": 3.286876678466797,
"eval_runtime": 83.971,
"eval_samples_per_second": 214.503,
"eval_steps_per_second": 13.409,
"step": 102000
},
{
"epoch": 27.462424633936262,
"grad_norm": 0.4930401146411896,
"learning_rate": 0.00018819577218257705,
"loss": 2.9215,
"step": 102050
},
{
"epoch": 27.47588285960379,
"grad_norm": 0.4859257638454437,
"learning_rate": 0.0001879938063821193,
"loss": 2.9259,
"step": 102100
},
{
"epoch": 27.489341085271317,
"grad_norm": 0.46256932616233826,
"learning_rate": 0.0001877918405816615,
"loss": 2.9334,
"step": 102150
},
{
"epoch": 27.502799310938848,
"grad_norm": 0.48986995220184326,
"learning_rate": 0.00018758987478120371,
"loss": 2.9305,
"step": 102200
},
{
"epoch": 27.516257536606375,
"grad_norm": 0.4924154281616211,
"learning_rate": 0.0001873879089807459,
"loss": 2.9334,
"step": 102250
},
{
"epoch": 27.529715762273902,
"grad_norm": 0.47396549582481384,
"learning_rate": 0.00018718594318028813,
"loss": 2.9282,
"step": 102300
},
{
"epoch": 27.54317398794143,
"grad_norm": 0.4847225844860077,
"learning_rate": 0.00018698397737983033,
"loss": 2.9254,
"step": 102350
},
{
"epoch": 27.556632213608957,
"grad_norm": 0.4604421555995941,
"learning_rate": 0.00018678201157937253,
"loss": 2.9322,
"step": 102400
},
{
"epoch": 27.570090439276484,
"grad_norm": 0.4923710823059082,
"learning_rate": 0.00018658004577891475,
"loss": 2.9295,
"step": 102450
},
{
"epoch": 27.583548664944015,
"grad_norm": 0.5104304552078247,
"learning_rate": 0.00018637807997845695,
"loss": 2.9257,
"step": 102500
},
{
"epoch": 27.597006890611542,
"grad_norm": 0.4732654392719269,
"learning_rate": 0.0001861761141779992,
"loss": 2.9429,
"step": 102550
},
{
"epoch": 27.61046511627907,
"grad_norm": 0.47959786653518677,
"learning_rate": 0.0001859741483775414,
"loss": 2.9296,
"step": 102600
},
{
"epoch": 27.623923341946597,
"grad_norm": 0.5053572058677673,
"learning_rate": 0.00018577218257708362,
"loss": 2.9232,
"step": 102650
},
{
"epoch": 27.637381567614124,
"grad_norm": 0.47885841131210327,
"learning_rate": 0.0001855702167766258,
"loss": 2.9288,
"step": 102700
},
{
"epoch": 27.650839793281655,
"grad_norm": 0.5167924761772156,
"learning_rate": 0.000185368250976168,
"loss": 2.929,
"step": 102750
},
{
"epoch": 27.664298018949182,
"grad_norm": 0.48342621326446533,
"learning_rate": 0.00018516628517571023,
"loss": 2.938,
"step": 102800
},
{
"epoch": 27.67775624461671,
"grad_norm": 0.5054475665092468,
"learning_rate": 0.00018496431937525243,
"loss": 2.9331,
"step": 102850
},
{
"epoch": 27.691214470284237,
"grad_norm": 0.46684083342552185,
"learning_rate": 0.00018476235357479465,
"loss": 2.9428,
"step": 102900
},
{
"epoch": 27.704672695951764,
"grad_norm": 0.5017584562301636,
"learning_rate": 0.00018456038777433685,
"loss": 2.9348,
"step": 102950
},
{
"epoch": 27.718130921619295,
"grad_norm": 0.46952277421951294,
"learning_rate": 0.0001843584219738791,
"loss": 2.9426,
"step": 103000
},
{
"epoch": 27.718130921619295,
"eval_accuracy": 0.39760642285443104,
"eval_loss": 3.2794814109802246,
"eval_runtime": 83.8754,
"eval_samples_per_second": 214.747,
"eval_steps_per_second": 13.425,
"step": 103000
},
{
"epoch": 27.731589147286822,
"grad_norm": 0.491432785987854,
"learning_rate": 0.0001841564561734213,
"loss": 2.948,
"step": 103050
},
{
"epoch": 27.74504737295435,
"grad_norm": 0.4925087094306946,
"learning_rate": 0.00018395449037296352,
"loss": 2.9392,
"step": 103100
},
{
"epoch": 27.758505598621877,
"grad_norm": 0.4647904932498932,
"learning_rate": 0.0001837525245725057,
"loss": 2.953,
"step": 103150
},
{
"epoch": 27.771963824289404,
"grad_norm": 0.5009186863899231,
"learning_rate": 0.0001835505587720479,
"loss": 2.9375,
"step": 103200
},
{
"epoch": 27.785422049956935,
"grad_norm": 0.5034534931182861,
"learning_rate": 0.00018334859297159013,
"loss": 2.9452,
"step": 103250
},
{
"epoch": 27.798880275624462,
"grad_norm": 0.47611668705940247,
"learning_rate": 0.00018314662717113233,
"loss": 2.9399,
"step": 103300
},
{
"epoch": 27.81233850129199,
"grad_norm": 0.506561815738678,
"learning_rate": 0.00018294466137067455,
"loss": 2.951,
"step": 103350
},
{
"epoch": 27.825796726959517,
"grad_norm": 0.4973388612270355,
"learning_rate": 0.00018274269557021675,
"loss": 2.9287,
"step": 103400
},
{
"epoch": 27.839254952627044,
"grad_norm": 0.4861920475959778,
"learning_rate": 0.000182540729769759,
"loss": 2.9454,
"step": 103450
},
{
"epoch": 27.852713178294575,
"grad_norm": 0.5094921588897705,
"learning_rate": 0.0001823387639693012,
"loss": 2.9481,
"step": 103500
},
{
"epoch": 27.866171403962102,
"grad_norm": 0.4648674428462982,
"learning_rate": 0.0001821367981688434,
"loss": 2.9432,
"step": 103550
},
{
"epoch": 27.87962962962963,
"grad_norm": 0.5445858240127563,
"learning_rate": 0.0001819348323683856,
"loss": 2.948,
"step": 103600
},
{
"epoch": 27.893087855297157,
"grad_norm": 0.48193544149398804,
"learning_rate": 0.0001817328665679278,
"loss": 2.942,
"step": 103650
},
{
"epoch": 27.906546080964684,
"grad_norm": 0.5002299547195435,
"learning_rate": 0.00018153090076747003,
"loss": 2.9535,
"step": 103700
},
{
"epoch": 27.920004306632215,
"grad_norm": 0.5245981216430664,
"learning_rate": 0.00018132893496701223,
"loss": 2.9465,
"step": 103750
},
{
"epoch": 27.933462532299743,
"grad_norm": 0.5068734288215637,
"learning_rate": 0.00018112696916655442,
"loss": 2.9405,
"step": 103800
},
{
"epoch": 27.94692075796727,
"grad_norm": 0.4961998462677002,
"learning_rate": 0.00018092500336609665,
"loss": 2.9508,
"step": 103850
},
{
"epoch": 27.960378983634797,
"grad_norm": 0.46381625533103943,
"learning_rate": 0.0001807230375656389,
"loss": 2.94,
"step": 103900
},
{
"epoch": 27.973837209302324,
"grad_norm": 0.4833906292915344,
"learning_rate": 0.0001805210717651811,
"loss": 2.95,
"step": 103950
},
{
"epoch": 27.987295434969855,
"grad_norm": 0.4873865246772766,
"learning_rate": 0.0001803191059647233,
"loss": 2.963,
"step": 104000
},
{
"epoch": 27.987295434969855,
"eval_accuracy": 0.39772006746535143,
"eval_loss": 3.2727417945861816,
"eval_runtime": 83.8715,
"eval_samples_per_second": 214.757,
"eval_steps_per_second": 13.425,
"step": 104000
},
{
"epoch": 28.0005383290267,
"grad_norm": 0.4941236972808838,
"learning_rate": 0.0001801171401642655,
"loss": 2.9422,
"step": 104050
},
{
"epoch": 28.01399655469423,
"grad_norm": 0.49093982577323914,
"learning_rate": 0.0001799151743638077,
"loss": 2.8689,
"step": 104100
},
{
"epoch": 28.027454780361758,
"grad_norm": 0.5120478272438049,
"learning_rate": 0.00017971320856334993,
"loss": 2.8857,
"step": 104150
},
{
"epoch": 28.040913006029285,
"grad_norm": 0.4717957377433777,
"learning_rate": 0.00017951124276289213,
"loss": 2.8768,
"step": 104200
},
{
"epoch": 28.054371231696813,
"grad_norm": 0.4936118423938751,
"learning_rate": 0.00017930927696243432,
"loss": 2.8843,
"step": 104250
},
{
"epoch": 28.06782945736434,
"grad_norm": 0.4972836971282959,
"learning_rate": 0.00017910731116197655,
"loss": 2.8788,
"step": 104300
},
{
"epoch": 28.08128768303187,
"grad_norm": 0.4746985137462616,
"learning_rate": 0.00017890534536151877,
"loss": 2.8813,
"step": 104350
},
{
"epoch": 28.094745908699398,
"grad_norm": 0.5059629082679749,
"learning_rate": 0.000178703379561061,
"loss": 2.8828,
"step": 104400
},
{
"epoch": 28.108204134366925,
"grad_norm": 0.4806491434574127,
"learning_rate": 0.0001785014137606032,
"loss": 2.89,
"step": 104450
},
{
"epoch": 28.121662360034453,
"grad_norm": 0.5036650896072388,
"learning_rate": 0.0001782994479601454,
"loss": 2.8802,
"step": 104500
},
{
"epoch": 28.13512058570198,
"grad_norm": 0.4994170367717743,
"learning_rate": 0.0001780974821596876,
"loss": 2.8938,
"step": 104550
},
{
"epoch": 28.14857881136951,
"grad_norm": 0.5043787956237793,
"learning_rate": 0.0001778955163592298,
"loss": 2.8922,
"step": 104600
},
{
"epoch": 28.162037037037038,
"grad_norm": 0.47212567925453186,
"learning_rate": 0.00017769355055877203,
"loss": 2.8962,
"step": 104650
},
{
"epoch": 28.175495262704565,
"grad_norm": 0.4917432367801666,
"learning_rate": 0.00017749158475831422,
"loss": 2.8879,
"step": 104700
},
{
"epoch": 28.188953488372093,
"grad_norm": 0.48865607380867004,
"learning_rate": 0.00017728961895785645,
"loss": 2.898,
"step": 104750
},
{
"epoch": 28.20241171403962,
"grad_norm": 0.505818784236908,
"learning_rate": 0.00017708765315739867,
"loss": 2.8974,
"step": 104800
},
{
"epoch": 28.215869939707147,
"grad_norm": 0.5183230638504028,
"learning_rate": 0.0001768856873569409,
"loss": 2.8967,
"step": 104850
},
{
"epoch": 28.229328165374678,
"grad_norm": 0.48350223898887634,
"learning_rate": 0.0001766837215564831,
"loss": 2.9063,
"step": 104900
},
{
"epoch": 28.242786391042205,
"grad_norm": 0.4778302311897278,
"learning_rate": 0.0001764817557560253,
"loss": 2.8915,
"step": 104950
},
{
"epoch": 28.256244616709733,
"grad_norm": 0.524963915348053,
"learning_rate": 0.0001762797899555675,
"loss": 2.9024,
"step": 105000
},
{
"epoch": 28.256244616709733,
"eval_accuracy": 0.3972076888945095,
"eval_loss": 3.289083242416382,
"eval_runtime": 84.2414,
"eval_samples_per_second": 213.814,
"eval_steps_per_second": 13.366,
"step": 105000
},
{
"epoch": 28.26970284237726,
"grad_norm": 0.49806711077690125,
"learning_rate": 0.0001760778241551097,
"loss": 2.9048,
"step": 105050
},
{
"epoch": 28.283161068044787,
"grad_norm": 0.4823664724826813,
"learning_rate": 0.00017587585835465193,
"loss": 2.91,
"step": 105100
},
{
"epoch": 28.296619293712318,
"grad_norm": 0.4940940737724304,
"learning_rate": 0.00017567389255419412,
"loss": 2.9071,
"step": 105150
},
{
"epoch": 28.310077519379846,
"grad_norm": 0.4807303845882416,
"learning_rate": 0.00017547192675373635,
"loss": 2.9051,
"step": 105200
},
{
"epoch": 28.323535745047373,
"grad_norm": 0.47984981536865234,
"learning_rate": 0.00017526996095327857,
"loss": 2.9115,
"step": 105250
},
{
"epoch": 28.3369939707149,
"grad_norm": 0.48672279715538025,
"learning_rate": 0.0001750679951528208,
"loss": 2.9045,
"step": 105300
},
{
"epoch": 28.350452196382427,
"grad_norm": 0.49630510807037354,
"learning_rate": 0.000174866029352363,
"loss": 2.9019,
"step": 105350
},
{
"epoch": 28.36391042204996,
"grad_norm": 0.4995165467262268,
"learning_rate": 0.0001746640635519052,
"loss": 2.9166,
"step": 105400
},
{
"epoch": 28.377368647717486,
"grad_norm": 0.4939239025115967,
"learning_rate": 0.0001744620977514474,
"loss": 2.8999,
"step": 105450
},
{
"epoch": 28.390826873385013,
"grad_norm": 0.5187941193580627,
"learning_rate": 0.0001742601319509896,
"loss": 2.9106,
"step": 105500
},
{
"epoch": 28.40428509905254,
"grad_norm": 0.5010408759117126,
"learning_rate": 0.00017405816615053183,
"loss": 2.9143,
"step": 105550
},
{
"epoch": 28.417743324720067,
"grad_norm": 0.4850703775882721,
"learning_rate": 0.00017385620035007403,
"loss": 2.9171,
"step": 105600
},
{
"epoch": 28.4312015503876,
"grad_norm": 0.49331873655319214,
"learning_rate": 0.00017365423454961622,
"loss": 2.9133,
"step": 105650
},
{
"epoch": 28.444659776055126,
"grad_norm": 0.48752540349960327,
"learning_rate": 0.00017345226874915847,
"loss": 2.9162,
"step": 105700
},
{
"epoch": 28.458118001722653,
"grad_norm": 0.48763734102249146,
"learning_rate": 0.00017325030294870067,
"loss": 2.9166,
"step": 105750
},
{
"epoch": 28.47157622739018,
"grad_norm": 0.5082513689994812,
"learning_rate": 0.0001730483371482429,
"loss": 2.911,
"step": 105800
},
{
"epoch": 28.485034453057708,
"grad_norm": 0.511461615562439,
"learning_rate": 0.0001728463713477851,
"loss": 2.9104,
"step": 105850
},
{
"epoch": 28.49849267872524,
"grad_norm": 0.4683901071548462,
"learning_rate": 0.0001726444055473273,
"loss": 2.9134,
"step": 105900
},
{
"epoch": 28.511950904392766,
"grad_norm": 0.46261781454086304,
"learning_rate": 0.0001724424397468695,
"loss": 2.9188,
"step": 105950
},
{
"epoch": 28.525409130060293,
"grad_norm": 0.5006155371665955,
"learning_rate": 0.00017224047394641173,
"loss": 2.9245,
"step": 106000
},
{
"epoch": 28.525409130060293,
"eval_accuracy": 0.3973929317832469,
"eval_loss": 3.2823522090911865,
"eval_runtime": 84.2348,
"eval_samples_per_second": 213.831,
"eval_steps_per_second": 13.367,
"step": 106000
},
{
"epoch": 28.53886735572782,
"grad_norm": 0.49079206585884094,
"learning_rate": 0.00017203850814595393,
"loss": 2.9187,
"step": 106050
},
{
"epoch": 28.552325581395348,
"grad_norm": 0.4927675127983093,
"learning_rate": 0.00017183654234549612,
"loss": 2.928,
"step": 106100
},
{
"epoch": 28.56578380706288,
"grad_norm": 0.4958413541316986,
"learning_rate": 0.00017163457654503837,
"loss": 2.9321,
"step": 106150
},
{
"epoch": 28.579242032730406,
"grad_norm": 0.5253115296363831,
"learning_rate": 0.00017143261074458057,
"loss": 2.9212,
"step": 106200
},
{
"epoch": 28.592700258397933,
"grad_norm": 0.4913957417011261,
"learning_rate": 0.0001712306449441228,
"loss": 2.9217,
"step": 106250
},
{
"epoch": 28.60615848406546,
"grad_norm": 0.48931920528411865,
"learning_rate": 0.000171028679143665,
"loss": 2.9303,
"step": 106300
},
{
"epoch": 28.619616709732988,
"grad_norm": 0.49885618686676025,
"learning_rate": 0.0001708267133432072,
"loss": 2.9186,
"step": 106350
},
{
"epoch": 28.63307493540052,
"grad_norm": 0.49400222301483154,
"learning_rate": 0.0001706247475427494,
"loss": 2.9239,
"step": 106400
},
{
"epoch": 28.646533161068046,
"grad_norm": 0.4754203259944916,
"learning_rate": 0.0001704227817422916,
"loss": 2.9284,
"step": 106450
},
{
"epoch": 28.659991386735573,
"grad_norm": 0.515365719795227,
"learning_rate": 0.00017022081594183383,
"loss": 2.9208,
"step": 106500
},
{
"epoch": 28.6734496124031,
"grad_norm": 0.4889947772026062,
"learning_rate": 0.00017001885014137605,
"loss": 2.9298,
"step": 106550
},
{
"epoch": 28.686907838070628,
"grad_norm": 0.507577121257782,
"learning_rate": 0.00016981688434091827,
"loss": 2.9336,
"step": 106600
},
{
"epoch": 28.70036606373816,
"grad_norm": 0.4832504391670227,
"learning_rate": 0.00016961491854046047,
"loss": 2.9211,
"step": 106650
},
{
"epoch": 28.713824289405686,
"grad_norm": 0.48080530762672424,
"learning_rate": 0.0001694129527400027,
"loss": 2.9281,
"step": 106700
},
{
"epoch": 28.727282515073213,
"grad_norm": 0.48052066564559937,
"learning_rate": 0.0001692109869395449,
"loss": 2.9356,
"step": 106750
},
{
"epoch": 28.74074074074074,
"grad_norm": 0.4884222149848938,
"learning_rate": 0.00016900902113908708,
"loss": 2.9386,
"step": 106800
},
{
"epoch": 28.754198966408268,
"grad_norm": 0.4646649658679962,
"learning_rate": 0.0001688070553386293,
"loss": 2.929,
"step": 106850
},
{
"epoch": 28.767657192075795,
"grad_norm": 0.49423521757125854,
"learning_rate": 0.0001686050895381715,
"loss": 2.9336,
"step": 106900
},
{
"epoch": 28.781115417743326,
"grad_norm": 0.47869858145713806,
"learning_rate": 0.00016840312373771373,
"loss": 2.9254,
"step": 106950
},
{
"epoch": 28.794573643410853,
"grad_norm": 0.48208820819854736,
"learning_rate": 0.00016820115793725595,
"loss": 2.9363,
"step": 107000
},
{
"epoch": 28.794573643410853,
"eval_accuracy": 0.3979914673105514,
"eval_loss": 3.278270721435547,
"eval_runtime": 84.5483,
"eval_samples_per_second": 213.038,
"eval_steps_per_second": 13.318,
"step": 107000
},
{
"epoch": 28.80803186907838,
"grad_norm": 0.5072889924049377,
"learning_rate": 0.00016799919213679817,
"loss": 2.9372,
"step": 107050
},
{
"epoch": 28.821490094745908,
"grad_norm": 0.4976736307144165,
"learning_rate": 0.00016779722633634037,
"loss": 2.9245,
"step": 107100
},
{
"epoch": 28.834948320413435,
"grad_norm": 0.46875888109207153,
"learning_rate": 0.0001675952605358826,
"loss": 2.9375,
"step": 107150
},
{
"epoch": 28.848406546080966,
"grad_norm": 0.5036534667015076,
"learning_rate": 0.0001673932947354248,
"loss": 2.9346,
"step": 107200
},
{
"epoch": 28.861864771748493,
"grad_norm": 0.5005677938461304,
"learning_rate": 0.00016719132893496698,
"loss": 2.9357,
"step": 107250
},
{
"epoch": 28.87532299741602,
"grad_norm": 0.47183048725128174,
"learning_rate": 0.0001669893631345092,
"loss": 2.9305,
"step": 107300
},
{
"epoch": 28.888781223083548,
"grad_norm": 0.47492411732673645,
"learning_rate": 0.0001667873973340514,
"loss": 2.9351,
"step": 107350
},
{
"epoch": 28.902239448751075,
"grad_norm": 0.5148754715919495,
"learning_rate": 0.00016658543153359363,
"loss": 2.9298,
"step": 107400
},
{
"epoch": 28.915697674418606,
"grad_norm": 0.48464518785476685,
"learning_rate": 0.00016638346573313585,
"loss": 2.9515,
"step": 107450
},
{
"epoch": 28.929155900086133,
"grad_norm": 0.5281347036361694,
"learning_rate": 0.00016618149993267807,
"loss": 2.9466,
"step": 107500
},
{
"epoch": 28.94261412575366,
"grad_norm": 0.5068103075027466,
"learning_rate": 0.00016597953413222027,
"loss": 2.9318,
"step": 107550
},
{
"epoch": 28.956072351421188,
"grad_norm": 0.4937821626663208,
"learning_rate": 0.00016577756833176247,
"loss": 2.9267,
"step": 107600
},
{
"epoch": 28.969530577088715,
"grad_norm": 0.47072193026542664,
"learning_rate": 0.0001655756025313047,
"loss": 2.939,
"step": 107650
},
{
"epoch": 28.982988802756246,
"grad_norm": 0.482469767332077,
"learning_rate": 0.00016537363673084688,
"loss": 2.9345,
"step": 107700
},
{
"epoch": 28.996447028423773,
"grad_norm": 0.4808470606803894,
"learning_rate": 0.0001651716709303891,
"loss": 2.9333,
"step": 107750
},
{
"epoch": 29.00968992248062,
"grad_norm": 0.5030806660652161,
"learning_rate": 0.0001649697051299313,
"loss": 2.89,
"step": 107800
},
{
"epoch": 29.02314814814815,
"grad_norm": 0.5034016966819763,
"learning_rate": 0.0001647677393294735,
"loss": 2.8709,
"step": 107850
},
{
"epoch": 29.036606373815676,
"grad_norm": 0.4961067736148834,
"learning_rate": 0.00016456577352901575,
"loss": 2.8661,
"step": 107900
},
{
"epoch": 29.050064599483203,
"grad_norm": 0.5241340398788452,
"learning_rate": 0.00016436380772855797,
"loss": 2.8781,
"step": 107950
},
{
"epoch": 29.06352282515073,
"grad_norm": 0.5237793922424316,
"learning_rate": 0.00016416184192810017,
"loss": 2.8756,
"step": 108000
},
{
"epoch": 29.06352282515073,
"eval_accuracy": 0.3974687672884309,
"eval_loss": 3.2890923023223877,
"eval_runtime": 84.0275,
"eval_samples_per_second": 214.358,
"eval_steps_per_second": 13.4,
"step": 108000
},
{
"epoch": 29.07698105081826,
"grad_norm": 0.4945203363895416,
"learning_rate": 0.00016395987612764237,
"loss": 2.8869,
"step": 108050
},
{
"epoch": 29.09043927648579,
"grad_norm": 0.5141903758049011,
"learning_rate": 0.0001637579103271846,
"loss": 2.8779,
"step": 108100
},
{
"epoch": 29.103897502153316,
"grad_norm": 0.5113682746887207,
"learning_rate": 0.00016355594452672679,
"loss": 2.8839,
"step": 108150
},
{
"epoch": 29.117355727820843,
"grad_norm": 0.4902428388595581,
"learning_rate": 0.000163353978726269,
"loss": 2.8777,
"step": 108200
},
{
"epoch": 29.13081395348837,
"grad_norm": 0.5147128105163574,
"learning_rate": 0.0001631520129258112,
"loss": 2.8808,
"step": 108250
},
{
"epoch": 29.1442721791559,
"grad_norm": 0.5210860967636108,
"learning_rate": 0.0001629500471253534,
"loss": 2.8876,
"step": 108300
},
{
"epoch": 29.15773040482343,
"grad_norm": 0.4885682463645935,
"learning_rate": 0.00016274808132489565,
"loss": 2.8836,
"step": 108350
},
{
"epoch": 29.171188630490956,
"grad_norm": 0.46739473938941956,
"learning_rate": 0.00016254611552443785,
"loss": 2.8854,
"step": 108400
},
{
"epoch": 29.184646856158484,
"grad_norm": 0.5281350016593933,
"learning_rate": 0.00016234414972398007,
"loss": 2.8739,
"step": 108450
},
{
"epoch": 29.19810508182601,
"grad_norm": 0.516180694103241,
"learning_rate": 0.00016214218392352227,
"loss": 2.8952,
"step": 108500
},
{
"epoch": 29.21156330749354,
"grad_norm": 0.5106911063194275,
"learning_rate": 0.0001619402181230645,
"loss": 2.886,
"step": 108550
},
{
"epoch": 29.22502153316107,
"grad_norm": 0.5003798604011536,
"learning_rate": 0.00016173825232260669,
"loss": 2.8872,
"step": 108600
},
{
"epoch": 29.238479758828596,
"grad_norm": 0.49094250798225403,
"learning_rate": 0.00016153628652214888,
"loss": 2.8918,
"step": 108650
},
{
"epoch": 29.251937984496124,
"grad_norm": 0.48865559697151184,
"learning_rate": 0.0001613343207216911,
"loss": 2.895,
"step": 108700
},
{
"epoch": 29.26539621016365,
"grad_norm": 0.46626487374305725,
"learning_rate": 0.0001611323549212333,
"loss": 2.9045,
"step": 108750
},
{
"epoch": 29.27885443583118,
"grad_norm": 0.457952082157135,
"learning_rate": 0.00016093038912077555,
"loss": 2.9005,
"step": 108800
},
{
"epoch": 29.29231266149871,
"grad_norm": 0.49217286705970764,
"learning_rate": 0.00016072842332031775,
"loss": 2.9044,
"step": 108850
},
{
"epoch": 29.305770887166236,
"grad_norm": 0.4805237054824829,
"learning_rate": 0.00016052645751985997,
"loss": 2.8909,
"step": 108900
},
{
"epoch": 29.319229112833764,
"grad_norm": 0.4833526909351349,
"learning_rate": 0.00016032449171940217,
"loss": 2.8915,
"step": 108950
},
{
"epoch": 29.33268733850129,
"grad_norm": 0.48899632692337036,
"learning_rate": 0.0001601225259189444,
"loss": 2.8986,
"step": 109000
},
{
"epoch": 29.33268733850129,
"eval_accuracy": 0.39724647582194605,
"eval_loss": 3.2852847576141357,
"eval_runtime": 84.1715,
"eval_samples_per_second": 213.992,
"eval_steps_per_second": 13.377,
"step": 109000
},
{
"epoch": 29.34614556416882,
"grad_norm": 0.4914514124393463,
"learning_rate": 0.00015992056011848659,
"loss": 2.8909,
"step": 109050
},
{
"epoch": 29.35960378983635,
"grad_norm": 0.519985020160675,
"learning_rate": 0.00015971859431802878,
"loss": 2.9079,
"step": 109100
},
{
"epoch": 29.373062015503876,
"grad_norm": 0.5080176591873169,
"learning_rate": 0.000159516628517571,
"loss": 2.8982,
"step": 109150
},
{
"epoch": 29.386520241171404,
"grad_norm": 0.49518075585365295,
"learning_rate": 0.0001593146627171132,
"loss": 2.9058,
"step": 109200
},
{
"epoch": 29.39997846683893,
"grad_norm": 0.5173744559288025,
"learning_rate": 0.00015911269691665545,
"loss": 2.9055,
"step": 109250
},
{
"epoch": 29.41343669250646,
"grad_norm": 0.5047296285629272,
"learning_rate": 0.00015891073111619765,
"loss": 2.9116,
"step": 109300
},
{
"epoch": 29.42689491817399,
"grad_norm": 0.5084612369537354,
"learning_rate": 0.00015870876531573987,
"loss": 2.9139,
"step": 109350
},
{
"epoch": 29.440353143841516,
"grad_norm": 0.4801039397716522,
"learning_rate": 0.00015850679951528207,
"loss": 2.905,
"step": 109400
},
{
"epoch": 29.453811369509044,
"grad_norm": 0.5018198490142822,
"learning_rate": 0.00015830483371482426,
"loss": 2.9154,
"step": 109450
},
{
"epoch": 29.46726959517657,
"grad_norm": 0.4790503680706024,
"learning_rate": 0.00015810286791436649,
"loss": 2.9117,
"step": 109500
},
{
"epoch": 29.4807278208441,
"grad_norm": 0.5301246047019958,
"learning_rate": 0.00015790090211390868,
"loss": 2.9013,
"step": 109550
},
{
"epoch": 29.49418604651163,
"grad_norm": 0.5275840163230896,
"learning_rate": 0.0001576989363134509,
"loss": 2.9033,
"step": 109600
},
{
"epoch": 29.507644272179157,
"grad_norm": 0.49783873558044434,
"learning_rate": 0.0001574969705129931,
"loss": 2.9009,
"step": 109650
},
{
"epoch": 29.521102497846684,
"grad_norm": 0.4843440055847168,
"learning_rate": 0.00015729500471253535,
"loss": 2.9055,
"step": 109700
},
{
"epoch": 29.53456072351421,
"grad_norm": 0.5092455148696899,
"learning_rate": 0.00015709303891207755,
"loss": 2.923,
"step": 109750
},
{
"epoch": 29.54801894918174,
"grad_norm": 0.4970625638961792,
"learning_rate": 0.00015689107311161977,
"loss": 2.9098,
"step": 109800
},
{
"epoch": 29.56147717484927,
"grad_norm": 0.5609508156776428,
"learning_rate": 0.00015668910731116197,
"loss": 2.9101,
"step": 109850
},
{
"epoch": 29.574935400516797,
"grad_norm": 0.49250948429107666,
"learning_rate": 0.00015648714151070416,
"loss": 2.918,
"step": 109900
},
{
"epoch": 29.588393626184324,
"grad_norm": 0.4990371763706207,
"learning_rate": 0.0001562851757102464,
"loss": 2.9178,
"step": 109950
},
{
"epoch": 29.60185185185185,
"grad_norm": 0.5044280886650085,
"learning_rate": 0.00015608320990978858,
"loss": 2.9028,
"step": 110000
},
{
"epoch": 29.60185185185185,
"eval_accuracy": 0.39783610230709426,
"eval_loss": 3.280951976776123,
"eval_runtime": 84.2722,
"eval_samples_per_second": 213.736,
"eval_steps_per_second": 13.361,
"step": 110000
},
{
"epoch": 29.61531007751938,
"grad_norm": 0.5069197416305542,
"learning_rate": 0.0001558812441093308,
"loss": 2.9157,
"step": 110050
},
{
"epoch": 29.62876830318691,
"grad_norm": 0.5038443207740784,
"learning_rate": 0.000155679278308873,
"loss": 2.9223,
"step": 110100
},
{
"epoch": 29.642226528854437,
"grad_norm": 0.48724475502967834,
"learning_rate": 0.00015547731250841525,
"loss": 2.9161,
"step": 110150
},
{
"epoch": 29.655684754521964,
"grad_norm": 0.5094838738441467,
"learning_rate": 0.00015527534670795745,
"loss": 2.9074,
"step": 110200
},
{
"epoch": 29.66914298018949,
"grad_norm": 0.47840720415115356,
"learning_rate": 0.00015507338090749964,
"loss": 2.9163,
"step": 110250
},
{
"epoch": 29.68260120585702,
"grad_norm": 0.4904327392578125,
"learning_rate": 0.00015487141510704187,
"loss": 2.9138,
"step": 110300
},
{
"epoch": 29.69605943152455,
"grad_norm": 0.47974318265914917,
"learning_rate": 0.00015466944930658406,
"loss": 2.9062,
"step": 110350
},
{
"epoch": 29.709517657192077,
"grad_norm": 0.49263542890548706,
"learning_rate": 0.0001544674835061263,
"loss": 2.9131,
"step": 110400
},
{
"epoch": 29.722975882859604,
"grad_norm": 0.5041084289550781,
"learning_rate": 0.00015426551770566848,
"loss": 2.9162,
"step": 110450
},
{
"epoch": 29.73643410852713,
"grad_norm": 0.4936414957046509,
"learning_rate": 0.00015406355190521068,
"loss": 2.9207,
"step": 110500
},
{
"epoch": 29.74989233419466,
"grad_norm": 0.49592894315719604,
"learning_rate": 0.0001538615861047529,
"loss": 2.9263,
"step": 110550
},
{
"epoch": 29.76335055986219,
"grad_norm": 0.5023185610771179,
"learning_rate": 0.00015365962030429513,
"loss": 2.9115,
"step": 110600
},
{
"epoch": 29.776808785529717,
"grad_norm": 0.4932989776134491,
"learning_rate": 0.00015345765450383735,
"loss": 2.9321,
"step": 110650
},
{
"epoch": 29.790267011197244,
"grad_norm": 0.5014770030975342,
"learning_rate": 0.00015325568870337955,
"loss": 2.9144,
"step": 110700
},
{
"epoch": 29.80372523686477,
"grad_norm": 0.5211611390113831,
"learning_rate": 0.00015305372290292177,
"loss": 2.9225,
"step": 110750
},
{
"epoch": 29.8171834625323,
"grad_norm": 0.4848087728023529,
"learning_rate": 0.00015285175710246396,
"loss": 2.9207,
"step": 110800
},
{
"epoch": 29.830641688199826,
"grad_norm": 0.5076679587364197,
"learning_rate": 0.0001526497913020062,
"loss": 2.9128,
"step": 110850
},
{
"epoch": 29.844099913867357,
"grad_norm": 0.4708443582057953,
"learning_rate": 0.00015244782550154838,
"loss": 2.9207,
"step": 110900
},
{
"epoch": 29.857558139534884,
"grad_norm": 0.4948192834854126,
"learning_rate": 0.00015224585970109058,
"loss": 2.9277,
"step": 110950
},
{
"epoch": 29.87101636520241,
"grad_norm": 0.5008178353309631,
"learning_rate": 0.00015204389390063283,
"loss": 2.9246,
"step": 111000
},
{
"epoch": 29.87101636520241,
"eval_accuracy": 0.3981930072276234,
"eval_loss": 3.2754876613616943,
"eval_runtime": 84.2371,
"eval_samples_per_second": 213.825,
"eval_steps_per_second": 13.367,
"step": 111000
},
{
"epoch": 29.88447459086994,
"grad_norm": 0.5242527723312378,
"learning_rate": 0.00015184192810017503,
"loss": 2.9257,
"step": 111050
},
{
"epoch": 29.897932816537466,
"grad_norm": Infinity,
"learning_rate": 0.00015163996229971725,
"loss": 2.9271,
"step": 111100
},
{
"epoch": 29.911391042204997,
"grad_norm": 0.5184350609779358,
"learning_rate": 0.00015143799649925945,
"loss": 2.9267,
"step": 111150
},
{
"epoch": 29.924849267872524,
"grad_norm": 0.540956437587738,
"learning_rate": 0.00015123603069880167,
"loss": 2.9265,
"step": 111200
},
{
"epoch": 29.93830749354005,
"grad_norm": 0.5077881217002869,
"learning_rate": 0.00015103406489834386,
"loss": 2.9124,
"step": 111250
},
{
"epoch": 29.95176571920758,
"grad_norm": 0.4702873229980469,
"learning_rate": 0.00015083209909788606,
"loss": 2.9345,
"step": 111300
},
{
"epoch": 29.965223944875106,
"grad_norm": 0.48382219672203064,
"learning_rate": 0.00015063013329742828,
"loss": 2.9227,
"step": 111350
},
{
"epoch": 29.978682170542637,
"grad_norm": 0.4831846058368683,
"learning_rate": 0.00015042816749697048,
"loss": 2.9184,
"step": 111400
},
{
"epoch": 29.992140396210164,
"grad_norm": 0.4870881736278534,
"learning_rate": 0.00015022620169651273,
"loss": 2.9305,
"step": 111450
},
{
"epoch": 30.005383290267012,
"grad_norm": 0.5071820020675659,
"learning_rate": 0.00015002423589605493,
"loss": 2.8904,
"step": 111500
},
{
"epoch": 30.01884151593454,
"grad_norm": 0.5172536373138428,
"learning_rate": 0.00014982227009559712,
"loss": 2.8505,
"step": 111550
},
{
"epoch": 30.032299741602067,
"grad_norm": 0.5182663202285767,
"learning_rate": 0.00014962030429513935,
"loss": 2.8645,
"step": 111600
},
{
"epoch": 30.045757967269594,
"grad_norm": 0.526090681552887,
"learning_rate": 0.00014941833849468154,
"loss": 2.8609,
"step": 111650
},
{
"epoch": 30.05921619293712,
"grad_norm": 0.485584557056427,
"learning_rate": 0.00014921637269422377,
"loss": 2.8625,
"step": 111700
},
{
"epoch": 30.072674418604652,
"grad_norm": 0.5050146579742432,
"learning_rate": 0.00014901440689376596,
"loss": 2.8641,
"step": 111750
},
{
"epoch": 30.08613264427218,
"grad_norm": 0.4776037335395813,
"learning_rate": 0.00014881244109330818,
"loss": 2.8723,
"step": 111800
},
{
"epoch": 30.099590869939707,
"grad_norm": 0.5164368152618408,
"learning_rate": 0.0001486104752928504,
"loss": 2.8731,
"step": 111850
},
{
"epoch": 30.113049095607234,
"grad_norm": 0.5111197829246521,
"learning_rate": 0.0001484085094923926,
"loss": 2.8712,
"step": 111900
},
{
"epoch": 30.12650732127476,
"grad_norm": 0.5170813798904419,
"learning_rate": 0.00014820654369193483,
"loss": 2.8597,
"step": 111950
},
{
"epoch": 30.139965546942292,
"grad_norm": 0.5497874021530151,
"learning_rate": 0.00014800457789147702,
"loss": 2.8615,
"step": 112000
},
{
"epoch": 30.139965546942292,
"eval_accuracy": 0.3975050553381894,
"eval_loss": 3.2868740558624268,
"eval_runtime": 84.2131,
"eval_samples_per_second": 213.886,
"eval_steps_per_second": 13.371,
"step": 112000
},
{
"epoch": 30.15342377260982,
"grad_norm": 0.47900107502937317,
"learning_rate": 0.00014780261209101925,
"loss": 2.8826,
"step": 112050
},
{
"epoch": 30.166881998277347,
"grad_norm": 0.5066820979118347,
"learning_rate": 0.00014760064629056144,
"loss": 2.8738,
"step": 112100
},
{
"epoch": 30.180340223944874,
"grad_norm": 0.5189939737319946,
"learning_rate": 0.00014739868049010367,
"loss": 2.8772,
"step": 112150
},
{
"epoch": 30.1937984496124,
"grad_norm": 0.5010428428649902,
"learning_rate": 0.00014719671468964586,
"loss": 2.8769,
"step": 112200
},
{
"epoch": 30.207256675279933,
"grad_norm": 0.5018505454063416,
"learning_rate": 0.00014699474888918808,
"loss": 2.8781,
"step": 112250
},
{
"epoch": 30.22071490094746,
"grad_norm": 0.5155407190322876,
"learning_rate": 0.0001467927830887303,
"loss": 2.871,
"step": 112300
},
{
"epoch": 30.234173126614987,
"grad_norm": 0.49259355664253235,
"learning_rate": 0.0001465908172882725,
"loss": 2.8838,
"step": 112350
},
{
"epoch": 30.247631352282514,
"grad_norm": 0.4987989664077759,
"learning_rate": 0.0001463888514878147,
"loss": 2.8695,
"step": 112400
},
{
"epoch": 30.26108957795004,
"grad_norm": 0.4886040985584259,
"learning_rate": 0.00014618688568735692,
"loss": 2.895,
"step": 112450
},
{
"epoch": 30.274547803617573,
"grad_norm": 0.5024857521057129,
"learning_rate": 0.00014598491988689915,
"loss": 2.878,
"step": 112500
},
{
"epoch": 30.2880060292851,
"grad_norm": 0.5005546808242798,
"learning_rate": 0.00014578295408644134,
"loss": 2.8905,
"step": 112550
},
{
"epoch": 30.301464254952627,
"grad_norm": 0.49909210205078125,
"learning_rate": 0.00014558098828598357,
"loss": 2.888,
"step": 112600
},
{
"epoch": 30.314922480620154,
"grad_norm": 0.5091086626052856,
"learning_rate": 0.00014537902248552576,
"loss": 2.8973,
"step": 112650
},
{
"epoch": 30.328380706287682,
"grad_norm": 0.5131893754005432,
"learning_rate": 0.00014517705668506799,
"loss": 2.894,
"step": 112700
},
{
"epoch": 30.341838931955213,
"grad_norm": 0.4898597300052643,
"learning_rate": 0.0001449750908846102,
"loss": 2.8916,
"step": 112750
},
{
"epoch": 30.35529715762274,
"grad_norm": 0.4944542646408081,
"learning_rate": 0.0001447731250841524,
"loss": 2.8834,
"step": 112800
},
{
"epoch": 30.368755383290267,
"grad_norm": 0.5082573890686035,
"learning_rate": 0.0001445711592836946,
"loss": 2.8951,
"step": 112850
},
{
"epoch": 30.382213608957795,
"grad_norm": 0.49803459644317627,
"learning_rate": 0.00014436919348323682,
"loss": 2.9,
"step": 112900
},
{
"epoch": 30.395671834625322,
"grad_norm": 0.4845969080924988,
"learning_rate": 0.00014416722768277905,
"loss": 2.8976,
"step": 112950
},
{
"epoch": 30.409130060292853,
"grad_norm": 0.4821034073829651,
"learning_rate": 0.00014396526188232124,
"loss": 2.8963,
"step": 113000
},
{
"epoch": 30.409130060292853,
"eval_accuracy": 0.3978802129304534,
"eval_loss": 3.284929037094116,
"eval_runtime": 84.2378,
"eval_samples_per_second": 213.823,
"eval_steps_per_second": 13.367,
"step": 113000
},
{
"epoch": 30.42258828596038,
"grad_norm": 0.5150105953216553,
"learning_rate": 0.00014376329608186347,
"loss": 2.8899,
"step": 113050
},
{
"epoch": 30.436046511627907,
"grad_norm": 0.5295490026473999,
"learning_rate": 0.00014356133028140566,
"loss": 2.8918,
"step": 113100
},
{
"epoch": 30.449504737295435,
"grad_norm": 0.5084060430526733,
"learning_rate": 0.00014335936448094789,
"loss": 2.8944,
"step": 113150
},
{
"epoch": 30.462962962962962,
"grad_norm": 0.4947968125343323,
"learning_rate": 0.00014315739868049008,
"loss": 2.9049,
"step": 113200
},
{
"epoch": 30.47642118863049,
"grad_norm": 0.5141030550003052,
"learning_rate": 0.0001429554328800323,
"loss": 2.8873,
"step": 113250
},
{
"epoch": 30.48987941429802,
"grad_norm": 0.5051911473274231,
"learning_rate": 0.0001427534670795745,
"loss": 2.8991,
"step": 113300
},
{
"epoch": 30.503337639965547,
"grad_norm": 0.5075148940086365,
"learning_rate": 0.00014255150127911672,
"loss": 2.8984,
"step": 113350
},
{
"epoch": 30.516795865633075,
"grad_norm": 0.5077381134033203,
"learning_rate": 0.00014234953547865895,
"loss": 2.9003,
"step": 113400
},
{
"epoch": 30.530254091300602,
"grad_norm": 0.501668393611908,
"learning_rate": 0.00014214756967820114,
"loss": 2.8974,
"step": 113450
},
{
"epoch": 30.54371231696813,
"grad_norm": 0.509915292263031,
"learning_rate": 0.00014194560387774334,
"loss": 2.9069,
"step": 113500
},
{
"epoch": 30.55717054263566,
"grad_norm": 0.5007650852203369,
"learning_rate": 0.00014174363807728556,
"loss": 2.9017,
"step": 113550
},
{
"epoch": 30.570628768303187,
"grad_norm": 0.5104435682296753,
"learning_rate": 0.00014154167227682779,
"loss": 2.8916,
"step": 113600
},
{
"epoch": 30.584086993970715,
"grad_norm": 0.5037990212440491,
"learning_rate": 0.00014133970647636998,
"loss": 2.9098,
"step": 113650
},
{
"epoch": 30.597545219638242,
"grad_norm": 0.5024341940879822,
"learning_rate": 0.0001411377406759122,
"loss": 2.9025,
"step": 113700
},
{
"epoch": 30.61100344530577,
"grad_norm": 0.5257053375244141,
"learning_rate": 0.0001409357748754544,
"loss": 2.8987,
"step": 113750
},
{
"epoch": 30.6244616709733,
"grad_norm": 0.46181410551071167,
"learning_rate": 0.00014073380907499662,
"loss": 2.9109,
"step": 113800
},
{
"epoch": 30.637919896640827,
"grad_norm": 0.5127111673355103,
"learning_rate": 0.00014053184327453885,
"loss": 2.9069,
"step": 113850
},
{
"epoch": 30.651378122308355,
"grad_norm": 0.4897187352180481,
"learning_rate": 0.00014032987747408104,
"loss": 2.9048,
"step": 113900
},
{
"epoch": 30.664836347975882,
"grad_norm": 0.5005426406860352,
"learning_rate": 0.00014012791167362324,
"loss": 2.9008,
"step": 113950
},
{
"epoch": 30.67829457364341,
"grad_norm": 0.5037729740142822,
"learning_rate": 0.00013992594587316546,
"loss": 2.918,
"step": 114000
},
{
"epoch": 30.67829457364341,
"eval_accuracy": 0.3980335136436548,
"eval_loss": 3.278815984725952,
"eval_runtime": 84.3786,
"eval_samples_per_second": 213.466,
"eval_steps_per_second": 13.345,
"step": 114000
},
{
"epoch": 30.69175279931094,
"grad_norm": 0.523506224155426,
"learning_rate": 0.0001397239800727077,
"loss": 2.9053,
"step": 114050
},
{
"epoch": 30.705211024978468,
"grad_norm": 0.4961335062980652,
"learning_rate": 0.00013952201427224988,
"loss": 2.905,
"step": 114100
},
{
"epoch": 30.718669250645995,
"grad_norm": 0.5052204728126526,
"learning_rate": 0.0001393200484717921,
"loss": 2.9071,
"step": 114150
},
{
"epoch": 30.732127476313522,
"grad_norm": 0.5077371001243591,
"learning_rate": 0.0001391180826713343,
"loss": 2.9145,
"step": 114200
},
{
"epoch": 30.74558570198105,
"grad_norm": 0.5354120135307312,
"learning_rate": 0.00013891611687087653,
"loss": 2.9087,
"step": 114250
},
{
"epoch": 30.75904392764858,
"grad_norm": 0.5553905367851257,
"learning_rate": 0.00013871415107041872,
"loss": 2.9169,
"step": 114300
},
{
"epoch": 30.772502153316108,
"grad_norm": 0.5040802359580994,
"learning_rate": 0.00013851218526996094,
"loss": 2.9133,
"step": 114350
},
{
"epoch": 30.785960378983635,
"grad_norm": 0.49840259552001953,
"learning_rate": 0.00013831021946950314,
"loss": 2.9128,
"step": 114400
},
{
"epoch": 30.799418604651162,
"grad_norm": 0.5111094117164612,
"learning_rate": 0.00013810825366904536,
"loss": 2.9148,
"step": 114450
},
{
"epoch": 30.81287683031869,
"grad_norm": 0.5349999070167542,
"learning_rate": 0.0001379062878685876,
"loss": 2.9149,
"step": 114500
},
{
"epoch": 30.82633505598622,
"grad_norm": 0.4961295425891876,
"learning_rate": 0.00013770432206812978,
"loss": 2.9145,
"step": 114550
},
{
"epoch": 30.839793281653748,
"grad_norm": 0.5046606659889221,
"learning_rate": 0.00013750235626767198,
"loss": 2.9172,
"step": 114600
},
{
"epoch": 30.853251507321275,
"grad_norm": 0.5328389406204224,
"learning_rate": 0.0001373003904672142,
"loss": 2.9154,
"step": 114650
},
{
"epoch": 30.866709732988802,
"grad_norm": 0.4973182678222656,
"learning_rate": 0.00013709842466675643,
"loss": 2.9158,
"step": 114700
},
{
"epoch": 30.88016795865633,
"grad_norm": 0.5038853287696838,
"learning_rate": 0.00013689645886629862,
"loss": 2.9113,
"step": 114750
},
{
"epoch": 30.89362618432386,
"grad_norm": 0.5285792946815491,
"learning_rate": 0.00013669449306584084,
"loss": 2.91,
"step": 114800
},
{
"epoch": 30.907084409991388,
"grad_norm": 0.502298891544342,
"learning_rate": 0.00013649252726538304,
"loss": 2.9152,
"step": 114850
},
{
"epoch": 30.920542635658915,
"grad_norm": 0.5127186179161072,
"learning_rate": 0.00013629056146492526,
"loss": 2.9186,
"step": 114900
},
{
"epoch": 30.934000861326442,
"grad_norm": 0.5167381763458252,
"learning_rate": 0.0001360885956644675,
"loss": 2.9176,
"step": 114950
},
{
"epoch": 30.94745908699397,
"grad_norm": 0.5052900910377502,
"learning_rate": 0.00013588662986400968,
"loss": 2.9152,
"step": 115000
},
{
"epoch": 30.94745908699397,
"eval_accuracy": 0.39822255917233695,
"eval_loss": 3.2754769325256348,
"eval_runtime": 84.3993,
"eval_samples_per_second": 213.414,
"eval_steps_per_second": 13.341,
"step": 115000
},
{
"epoch": 30.9609173126615,
"grad_norm": 0.501179039478302,
"learning_rate": 0.00013568466406355188,
"loss": 2.9281,
"step": 115050
},
{
"epoch": 30.974375538329028,
"grad_norm": 0.48699599504470825,
"learning_rate": 0.0001354826982630941,
"loss": 2.9207,
"step": 115100
},
{
"epoch": 30.987833763996555,
"grad_norm": 0.5128811597824097,
"learning_rate": 0.00013528073246263633,
"loss": 2.9114,
"step": 115150
},
{
"epoch": 31.001076658053403,
"grad_norm": 0.5467365384101868,
"learning_rate": 0.00013507876666217852,
"loss": 2.9059,
"step": 115200
},
{
"epoch": 31.01453488372093,
"grad_norm": 0.5502208471298218,
"learning_rate": 0.00013487680086172075,
"loss": 2.849,
"step": 115250
},
{
"epoch": 31.027993109388458,
"grad_norm": 0.5167126655578613,
"learning_rate": 0.00013467483506126294,
"loss": 2.8426,
"step": 115300
},
{
"epoch": 31.041451335055985,
"grad_norm": 0.5184963941574097,
"learning_rate": 0.00013447286926080516,
"loss": 2.8467,
"step": 115350
},
{
"epoch": 31.054909560723516,
"grad_norm": 0.5206332206726074,
"learning_rate": 0.00013427090346034736,
"loss": 2.8499,
"step": 115400
},
{
"epoch": 31.068367786391043,
"grad_norm": 0.5013174414634705,
"learning_rate": 0.00013406893765988958,
"loss": 2.8558,
"step": 115450
},
{
"epoch": 31.08182601205857,
"grad_norm": 0.5105743408203125,
"learning_rate": 0.00013386697185943178,
"loss": 2.86,
"step": 115500
},
{
"epoch": 31.095284237726098,
"grad_norm": 0.511475145816803,
"learning_rate": 0.000133665006058974,
"loss": 2.8515,
"step": 115550
},
{
"epoch": 31.108742463393625,
"grad_norm": 0.5447846055030823,
"learning_rate": 0.00013346304025851623,
"loss": 2.8609,
"step": 115600
},
{
"epoch": 31.122200689061152,
"grad_norm": 0.5339901447296143,
"learning_rate": 0.00013326107445805842,
"loss": 2.8659,
"step": 115650
},
{
"epoch": 31.135658914728683,
"grad_norm": 0.49852365255355835,
"learning_rate": 0.00013305910865760065,
"loss": 2.8626,
"step": 115700
},
{
"epoch": 31.14911714039621,
"grad_norm": 0.5074359774589539,
"learning_rate": 0.00013285714285714284,
"loss": 2.8682,
"step": 115750
},
{
"epoch": 31.162575366063738,
"grad_norm": 0.4895899295806885,
"learning_rate": 0.00013265517705668506,
"loss": 2.8582,
"step": 115800
},
{
"epoch": 31.176033591731265,
"grad_norm": 0.5205885171890259,
"learning_rate": 0.00013245321125622726,
"loss": 2.8695,
"step": 115850
},
{
"epoch": 31.189491817398793,
"grad_norm": 0.47917288541793823,
"learning_rate": 0.00013225124545576948,
"loss": 2.8703,
"step": 115900
},
{
"epoch": 31.202950043066323,
"grad_norm": 0.5210146307945251,
"learning_rate": 0.00013204927965531168,
"loss": 2.8712,
"step": 115950
},
{
"epoch": 31.21640826873385,
"grad_norm": 0.5375768542289734,
"learning_rate": 0.0001318473138548539,
"loss": 2.8768,
"step": 116000
},
{
"epoch": 31.21640826873385,
"eval_accuracy": 0.39771593888483997,
"eval_loss": 3.28580641746521,
"eval_runtime": 84.3527,
"eval_samples_per_second": 213.532,
"eval_steps_per_second": 13.349,
"step": 116000
},
{
"epoch": 31.229866494401378,
"grad_norm": 0.5357069969177246,
"learning_rate": 0.00013164534805439613,
"loss": 2.885,
"step": 116050
},
{
"epoch": 31.243324720068905,
"grad_norm": 0.5172284245491028,
"learning_rate": 0.00013144338225393832,
"loss": 2.8737,
"step": 116100
},
{
"epoch": 31.256782945736433,
"grad_norm": 0.5309697985649109,
"learning_rate": 0.00013124141645348052,
"loss": 2.876,
"step": 116150
},
{
"epoch": 31.270241171403963,
"grad_norm": 0.5113084316253662,
"learning_rate": 0.00013103945065302274,
"loss": 2.8656,
"step": 116200
},
{
"epoch": 31.28369939707149,
"grad_norm": 0.5179013013839722,
"learning_rate": 0.00013083748485256497,
"loss": 2.8736,
"step": 116250
},
{
"epoch": 31.297157622739018,
"grad_norm": 0.516130268573761,
"learning_rate": 0.00013063551905210716,
"loss": 2.8765,
"step": 116300
},
{
"epoch": 31.310615848406545,
"grad_norm": 0.5475269556045532,
"learning_rate": 0.00013043355325164938,
"loss": 2.8789,
"step": 116350
},
{
"epoch": 31.324074074074073,
"grad_norm": 0.5318151712417603,
"learning_rate": 0.00013023158745119158,
"loss": 2.8932,
"step": 116400
},
{
"epoch": 31.337532299741603,
"grad_norm": 0.5311176776885986,
"learning_rate": 0.00013002962165073378,
"loss": 2.8654,
"step": 116450
},
{
"epoch": 31.35099052540913,
"grad_norm": 0.5081773400306702,
"learning_rate": 0.000129827655850276,
"loss": 2.8748,
"step": 116500
},
{
"epoch": 31.364448751076658,
"grad_norm": 0.5100054740905762,
"learning_rate": 0.00012962569004981822,
"loss": 2.8837,
"step": 116550
},
{
"epoch": 31.377906976744185,
"grad_norm": 0.5171526670455933,
"learning_rate": 0.00012942372424936042,
"loss": 2.88,
"step": 116600
},
{
"epoch": 31.391365202411713,
"grad_norm": 0.5279509425163269,
"learning_rate": 0.00012922175844890264,
"loss": 2.892,
"step": 116650
},
{
"epoch": 31.404823428079244,
"grad_norm": 0.5070922374725342,
"learning_rate": 0.00012901979264844487,
"loss": 2.88,
"step": 116700
},
{
"epoch": 31.41828165374677,
"grad_norm": 0.47861653566360474,
"learning_rate": 0.00012881782684798706,
"loss": 2.8934,
"step": 116750
},
{
"epoch": 31.431739879414298,
"grad_norm": 0.490649938583374,
"learning_rate": 0.00012861586104752928,
"loss": 2.888,
"step": 116800
},
{
"epoch": 31.445198105081825,
"grad_norm": 0.5430272817611694,
"learning_rate": 0.00012841389524707148,
"loss": 2.8829,
"step": 116850
},
{
"epoch": 31.458656330749353,
"grad_norm": 0.5027969479560852,
"learning_rate": 0.0001282119294466137,
"loss": 2.8944,
"step": 116900
},
{
"epoch": 31.472114556416884,
"grad_norm": 0.5254911780357361,
"learning_rate": 0.0001280099636461559,
"loss": 2.8932,
"step": 116950
},
{
"epoch": 31.48557278208441,
"grad_norm": 0.4988119900226593,
"learning_rate": 0.00012780799784569812,
"loss": 2.8868,
"step": 117000
},
{
"epoch": 31.48557278208441,
"eval_accuracy": 0.39804426968235573,
"eval_loss": 3.2833847999572754,
"eval_runtime": 84.2746,
"eval_samples_per_second": 213.73,
"eval_steps_per_second": 13.361,
"step": 117000
},
{
"epoch": 31.499031007751938,
"grad_norm": 0.5362693071365356,
"learning_rate": 0.00012760603204524032,
"loss": 2.8915,
"step": 117050
},
{
"epoch": 31.512489233419466,
"grad_norm": 0.5097172260284424,
"learning_rate": 0.00012740406624478254,
"loss": 2.8897,
"step": 117100
},
{
"epoch": 31.525947459086993,
"grad_norm": 0.502714216709137,
"learning_rate": 0.00012720210044432477,
"loss": 2.9004,
"step": 117150
},
{
"epoch": 31.539405684754524,
"grad_norm": 0.5185081362724304,
"learning_rate": 0.00012700013464386696,
"loss": 2.8897,
"step": 117200
},
{
"epoch": 31.55286391042205,
"grad_norm": 0.5322837233543396,
"learning_rate": 0.00012679816884340916,
"loss": 2.8887,
"step": 117250
},
{
"epoch": 31.56632213608958,
"grad_norm": 0.482925146818161,
"learning_rate": 0.00012659620304295138,
"loss": 2.8929,
"step": 117300
},
{
"epoch": 31.579780361757106,
"grad_norm": 0.4954151213169098,
"learning_rate": 0.0001263942372424936,
"loss": 2.8746,
"step": 117350
},
{
"epoch": 31.593238587424633,
"grad_norm": 0.49335721135139465,
"learning_rate": 0.0001261922714420358,
"loss": 2.8896,
"step": 117400
},
{
"epoch": 31.606696813092164,
"grad_norm": 0.5141397714614868,
"learning_rate": 0.00012599030564157802,
"loss": 2.8872,
"step": 117450
},
{
"epoch": 31.62015503875969,
"grad_norm": 0.5417701601982117,
"learning_rate": 0.00012578833984112022,
"loss": 2.8953,
"step": 117500
},
{
"epoch": 31.63361326442722,
"grad_norm": 0.529873251914978,
"learning_rate": 0.00012558637404066244,
"loss": 2.9028,
"step": 117550
},
{
"epoch": 31.647071490094746,
"grad_norm": 0.510009765625,
"learning_rate": 0.00012538440824020467,
"loss": 2.894,
"step": 117600
},
{
"epoch": 31.660529715762273,
"grad_norm": 0.544947624206543,
"learning_rate": 0.00012518244243974686,
"loss": 2.9001,
"step": 117650
},
{
"epoch": 31.6739879414298,
"grad_norm": 0.527076244354248,
"learning_rate": 0.00012498047663928906,
"loss": 2.8974,
"step": 117700
},
{
"epoch": 31.68744616709733,
"grad_norm": 0.5229929089546204,
"learning_rate": 0.00012477851083883128,
"loss": 2.8987,
"step": 117750
},
{
"epoch": 31.70090439276486,
"grad_norm": 0.5023860931396484,
"learning_rate": 0.0001245765450383735,
"loss": 2.9021,
"step": 117800
},
{
"epoch": 31.714362618432386,
"grad_norm": 0.5246515274047852,
"learning_rate": 0.0001243745792379157,
"loss": 2.8945,
"step": 117850
},
{
"epoch": 31.727820844099913,
"grad_norm": 0.5159626007080078,
"learning_rate": 0.00012417261343745792,
"loss": 2.8991,
"step": 117900
},
{
"epoch": 31.74127906976744,
"grad_norm": 0.5257189273834229,
"learning_rate": 0.00012397064763700012,
"loss": 2.9054,
"step": 117950
},
{
"epoch": 31.75473729543497,
"grad_norm": 0.5196221470832825,
"learning_rate": 0.00012376868183654232,
"loss": 2.8962,
"step": 118000
},
{
"epoch": 31.75473729543497,
"eval_accuracy": 0.39826341039002916,
"eval_loss": 3.279503583908081,
"eval_runtime": 84.0907,
"eval_samples_per_second": 214.197,
"eval_steps_per_second": 13.39,
"step": 118000
},
{
"epoch": 31.7681955211025,
"grad_norm": 0.5018131732940674,
"learning_rate": 0.00012356671603608454,
"loss": 2.8978,
"step": 118050
},
{
"epoch": 31.781653746770026,
"grad_norm": 0.47433024644851685,
"learning_rate": 0.00012336475023562676,
"loss": 2.8956,
"step": 118100
},
{
"epoch": 31.795111972437553,
"grad_norm": 0.5425954461097717,
"learning_rate": 0.00012316278443516896,
"loss": 2.9015,
"step": 118150
},
{
"epoch": 31.80857019810508,
"grad_norm": 0.4845956265926361,
"learning_rate": 0.00012296081863471118,
"loss": 2.9057,
"step": 118200
},
{
"epoch": 31.82202842377261,
"grad_norm": 0.5126825571060181,
"learning_rate": 0.0001227588528342534,
"loss": 2.8968,
"step": 118250
},
{
"epoch": 31.83548664944014,
"grad_norm": 0.5188189148902893,
"learning_rate": 0.0001225568870337956,
"loss": 2.9073,
"step": 118300
},
{
"epoch": 31.848944875107666,
"grad_norm": 0.5191381573677063,
"learning_rate": 0.0001223549212333378,
"loss": 2.9082,
"step": 118350
},
{
"epoch": 31.862403100775193,
"grad_norm": 0.4905800223350525,
"learning_rate": 0.00012215295543288002,
"loss": 2.9009,
"step": 118400
},
{
"epoch": 31.87586132644272,
"grad_norm": 0.49753281474113464,
"learning_rate": 0.00012195098963242223,
"loss": 2.9032,
"step": 118450
},
{
"epoch": 31.88931955211025,
"grad_norm": 0.5176593661308289,
"learning_rate": 0.00012174902383196445,
"loss": 2.9051,
"step": 118500
},
{
"epoch": 31.90277777777778,
"grad_norm": 0.49819475412368774,
"learning_rate": 0.00012154705803150665,
"loss": 2.8974,
"step": 118550
},
{
"epoch": 31.916236003445306,
"grad_norm": 0.5007299184799194,
"learning_rate": 0.00012134509223104886,
"loss": 2.9181,
"step": 118600
},
{
"epoch": 31.929694229112833,
"grad_norm": 0.5035253167152405,
"learning_rate": 0.00012114312643059107,
"loss": 2.9012,
"step": 118650
},
{
"epoch": 31.94315245478036,
"grad_norm": 0.6211641430854797,
"learning_rate": 0.00012094116063013329,
"loss": 2.9118,
"step": 118700
},
{
"epoch": 31.95661068044789,
"grad_norm": 0.5061435103416443,
"learning_rate": 0.0001207391948296755,
"loss": 2.9048,
"step": 118750
},
{
"epoch": 31.97006890611542,
"grad_norm": 0.4980998635292053,
"learning_rate": 0.00012053722902921771,
"loss": 2.9065,
"step": 118800
},
{
"epoch": 31.983527131782946,
"grad_norm": 0.5427234768867493,
"learning_rate": 0.00012033526322875992,
"loss": 2.9085,
"step": 118850
},
{
"epoch": 31.996985357450473,
"grad_norm": 0.4822652339935303,
"learning_rate": 0.00012013329742830212,
"loss": 2.9102,
"step": 118900
},
{
"epoch": 32.01022825150732,
"grad_norm": 0.4894477128982544,
"learning_rate": 0.00011993133162784434,
"loss": 2.8644,
"step": 118950
},
{
"epoch": 32.02368647717485,
"grad_norm": 0.526383638381958,
"learning_rate": 0.00011972936582738655,
"loss": 2.847,
"step": 119000
},
{
"epoch": 32.02368647717485,
"eval_accuracy": 0.39809392129534865,
"eval_loss": 3.2846009731292725,
"eval_runtime": 84.1418,
"eval_samples_per_second": 214.067,
"eval_steps_per_second": 13.382,
"step": 119000
},
{
"epoch": 32.037144702842376,
"grad_norm": 0.5376343131065369,
"learning_rate": 0.00011952740002692876,
"loss": 2.8419,
"step": 119050
},
{
"epoch": 32.0506029285099,
"grad_norm": 0.5181412100791931,
"learning_rate": 0.00011932543422647097,
"loss": 2.8521,
"step": 119100
},
{
"epoch": 32.06406115417743,
"grad_norm": 0.48993155360221863,
"learning_rate": 0.00011912346842601319,
"loss": 2.8479,
"step": 119150
},
{
"epoch": 32.07751937984496,
"grad_norm": 0.5353118777275085,
"learning_rate": 0.0001189215026255554,
"loss": 2.8574,
"step": 119200
},
{
"epoch": 32.09097760551249,
"grad_norm": 0.5269080400466919,
"learning_rate": 0.00011871953682509761,
"loss": 2.8595,
"step": 119250
},
{
"epoch": 32.10443583118002,
"grad_norm": 0.5100624561309814,
"learning_rate": 0.00011851757102463981,
"loss": 2.8484,
"step": 119300
},
{
"epoch": 32.11789405684755,
"grad_norm": 0.5078878998756409,
"learning_rate": 0.00011831560522418203,
"loss": 2.8521,
"step": 119350
},
{
"epoch": 32.131352282515074,
"grad_norm": 0.5233724117279053,
"learning_rate": 0.00011811363942372424,
"loss": 2.8533,
"step": 119400
},
{
"epoch": 32.1448105081826,
"grad_norm": 0.4920700192451477,
"learning_rate": 0.00011791167362326645,
"loss": 2.8464,
"step": 119450
},
{
"epoch": 32.15826873385013,
"grad_norm": 0.5175674557685852,
"learning_rate": 0.00011770970782280866,
"loss": 2.8637,
"step": 119500
},
{
"epoch": 32.171726959517656,
"grad_norm": 0.5312711596488953,
"learning_rate": 0.00011750774202235087,
"loss": 2.8582,
"step": 119550
},
{
"epoch": 32.18518518518518,
"grad_norm": 0.4959535002708435,
"learning_rate": 0.00011730577622189309,
"loss": 2.8523,
"step": 119600
},
{
"epoch": 32.19864341085271,
"grad_norm": 0.5239533185958862,
"learning_rate": 0.00011710381042143529,
"loss": 2.8761,
"step": 119650
},
{
"epoch": 32.21210163652024,
"grad_norm": 0.5062153339385986,
"learning_rate": 0.0001169018446209775,
"loss": 2.8616,
"step": 119700
},
{
"epoch": 32.22555986218777,
"grad_norm": 0.530886709690094,
"learning_rate": 0.00011669987882051971,
"loss": 2.8666,
"step": 119750
},
{
"epoch": 32.2390180878553,
"grad_norm": 0.5261471271514893,
"learning_rate": 0.00011649791302006193,
"loss": 2.8676,
"step": 119800
},
{
"epoch": 32.25247631352283,
"grad_norm": 0.5268979668617249,
"learning_rate": 0.00011629594721960414,
"loss": 2.8731,
"step": 119850
},
{
"epoch": 32.265934539190354,
"grad_norm": 0.49872177839279175,
"learning_rate": 0.00011609398141914635,
"loss": 2.8635,
"step": 119900
},
{
"epoch": 32.27939276485788,
"grad_norm": 0.5125359892845154,
"learning_rate": 0.00011589201561868856,
"loss": 2.8658,
"step": 119950
},
{
"epoch": 32.29285099052541,
"grad_norm": 0.510800302028656,
"learning_rate": 0.00011569004981823076,
"loss": 2.872,
"step": 120000
},
{
"epoch": 32.29285099052541,
"eval_accuracy": 0.3980136312690865,
"eval_loss": 3.2843637466430664,
"eval_runtime": 83.3793,
"eval_samples_per_second": 216.025,
"eval_steps_per_second": 13.505,
"step": 120000
},
{
"epoch": 32.306309216192936,
"grad_norm": 0.5112922787666321,
"learning_rate": 0.00011548808401777298,
"loss": 2.8678,
"step": 120050
},
{
"epoch": 32.31976744186046,
"grad_norm": 0.4980347454547882,
"learning_rate": 0.00011528611821731519,
"loss": 2.8694,
"step": 120100
},
{
"epoch": 32.33322566752799,
"grad_norm": 0.5203003287315369,
"learning_rate": 0.0001150841524168574,
"loss": 2.8707,
"step": 120150
},
{
"epoch": 32.34668389319552,
"grad_norm": 0.4873710572719574,
"learning_rate": 0.00011488218661639961,
"loss": 2.8717,
"step": 120200
},
{
"epoch": 32.36014211886305,
"grad_norm": 0.5280223488807678,
"learning_rate": 0.00011468022081594183,
"loss": 2.8711,
"step": 120250
},
{
"epoch": 32.37360034453058,
"grad_norm": 0.4919604957103729,
"learning_rate": 0.00011447825501548404,
"loss": 2.8794,
"step": 120300
},
{
"epoch": 32.38705857019811,
"grad_norm": 0.502147376537323,
"learning_rate": 0.00011427628921502625,
"loss": 2.8699,
"step": 120350
},
{
"epoch": 32.400516795865634,
"grad_norm": 0.5419111251831055,
"learning_rate": 0.00011407432341456845,
"loss": 2.877,
"step": 120400
},
{
"epoch": 32.41397502153316,
"grad_norm": 0.5109226703643799,
"learning_rate": 0.00011387235761411066,
"loss": 2.8717,
"step": 120450
},
{
"epoch": 32.42743324720069,
"grad_norm": 0.5274528861045837,
"learning_rate": 0.00011367039181365288,
"loss": 2.8865,
"step": 120500
},
{
"epoch": 32.440891472868216,
"grad_norm": 0.5329537391662598,
"learning_rate": 0.00011346842601319509,
"loss": 2.8804,
"step": 120550
},
{
"epoch": 32.454349698535744,
"grad_norm": 0.5048822164535522,
"learning_rate": 0.0001132664602127373,
"loss": 2.8793,
"step": 120600
},
{
"epoch": 32.46780792420327,
"grad_norm": 0.49886929988861084,
"learning_rate": 0.00011306449441227951,
"loss": 2.8818,
"step": 120650
},
{
"epoch": 32.4812661498708,
"grad_norm": 0.5073612928390503,
"learning_rate": 0.00011286252861182173,
"loss": 2.8853,
"step": 120700
},
{
"epoch": 32.494724375538325,
"grad_norm": 0.5252017378807068,
"learning_rate": 0.00011266056281136394,
"loss": 2.8712,
"step": 120750
},
{
"epoch": 32.50818260120586,
"grad_norm": 0.49827513098716736,
"learning_rate": 0.00011245859701090614,
"loss": 2.879,
"step": 120800
},
{
"epoch": 32.52164082687339,
"grad_norm": 0.5000596046447754,
"learning_rate": 0.00011225663121044835,
"loss": 2.8784,
"step": 120850
},
{
"epoch": 32.535099052540914,
"grad_norm": 0.5500352382659912,
"learning_rate": 0.00011205466540999056,
"loss": 2.8802,
"step": 120900
},
{
"epoch": 32.54855727820844,
"grad_norm": 0.5427979826927185,
"learning_rate": 0.00011185269960953278,
"loss": 2.8815,
"step": 120950
},
{
"epoch": 32.56201550387597,
"grad_norm": 0.5111308097839355,
"learning_rate": 0.00011165073380907499,
"loss": 2.8846,
"step": 121000
},
{
"epoch": 32.56201550387597,
"eval_accuracy": 0.3983339221992905,
"eval_loss": 3.2795493602752686,
"eval_runtime": 84.28,
"eval_samples_per_second": 213.716,
"eval_steps_per_second": 13.36,
"step": 121000
},
{
"epoch": 32.575473729543496,
"grad_norm": 0.5127388834953308,
"learning_rate": 0.0001114487680086172,
"loss": 2.8879,
"step": 121050
},
{
"epoch": 32.588931955211024,
"grad_norm": 0.5021010637283325,
"learning_rate": 0.0001112468022081594,
"loss": 2.8825,
"step": 121100
},
{
"epoch": 32.60239018087855,
"grad_norm": 0.5289945602416992,
"learning_rate": 0.00011104483640770163,
"loss": 2.8859,
"step": 121150
},
{
"epoch": 32.61584840654608,
"grad_norm": 0.5273314118385315,
"learning_rate": 0.00011084287060724383,
"loss": 2.8894,
"step": 121200
},
{
"epoch": 32.629306632213606,
"grad_norm": 0.5341081023216248,
"learning_rate": 0.00011064090480678604,
"loss": 2.8809,
"step": 121250
},
{
"epoch": 32.64276485788114,
"grad_norm": 0.507257878780365,
"learning_rate": 0.00011043893900632825,
"loss": 2.8815,
"step": 121300
},
{
"epoch": 32.65622308354867,
"grad_norm": 0.5302804708480835,
"learning_rate": 0.00011023697320587046,
"loss": 2.8826,
"step": 121350
},
{
"epoch": 32.669681309216195,
"grad_norm": 0.5019773840904236,
"learning_rate": 0.00011003500740541268,
"loss": 2.8825,
"step": 121400
},
{
"epoch": 32.68313953488372,
"grad_norm": 0.5342096090316772,
"learning_rate": 0.00010983304160495489,
"loss": 2.8835,
"step": 121450
},
{
"epoch": 32.69659776055125,
"grad_norm": 0.5053223371505737,
"learning_rate": 0.00010963107580449709,
"loss": 2.8795,
"step": 121500
},
{
"epoch": 32.71005598621878,
"grad_norm": 0.5185446739196777,
"learning_rate": 0.0001094291100040393,
"loss": 2.8884,
"step": 121550
},
{
"epoch": 32.723514211886304,
"grad_norm": 0.553935170173645,
"learning_rate": 0.00010922714420358152,
"loss": 2.8833,
"step": 121600
},
{
"epoch": 32.73697243755383,
"grad_norm": 0.535712480545044,
"learning_rate": 0.00010902517840312373,
"loss": 2.8855,
"step": 121650
},
{
"epoch": 32.75043066322136,
"grad_norm": 0.5135592222213745,
"learning_rate": 0.00010882321260266594,
"loss": 2.8896,
"step": 121700
},
{
"epoch": 32.763888888888886,
"grad_norm": 0.4964793920516968,
"learning_rate": 0.00010862124680220815,
"loss": 2.8966,
"step": 121750
},
{
"epoch": 32.77734711455642,
"grad_norm": 0.5169598460197449,
"learning_rate": 0.00010841928100175037,
"loss": 2.9057,
"step": 121800
},
{
"epoch": 32.79080534022395,
"grad_norm": 0.5452906489372253,
"learning_rate": 0.00010821731520129258,
"loss": 2.8853,
"step": 121850
},
{
"epoch": 32.804263565891475,
"grad_norm": 0.5410115718841553,
"learning_rate": 0.00010801534940083478,
"loss": 2.8926,
"step": 121900
},
{
"epoch": 32.817721791559,
"grad_norm": 0.5537077784538269,
"learning_rate": 0.00010781338360037699,
"loss": 2.8905,
"step": 121950
},
{
"epoch": 32.83118001722653,
"grad_norm": 0.5235624313354492,
"learning_rate": 0.0001076114177999192,
"loss": 2.8901,
"step": 122000
},
{
"epoch": 32.83118001722653,
"eval_accuracy": 0.3983279466222345,
"eval_loss": 3.277186393737793,
"eval_runtime": 84.0356,
"eval_samples_per_second": 214.338,
"eval_steps_per_second": 13.399,
"step": 122000
},
{
"epoch": 32.84463824289406,
"grad_norm": 0.5131085515022278,
"learning_rate": 0.00010740945199946142,
"loss": 2.8901,
"step": 122050
},
{
"epoch": 32.858096468561584,
"grad_norm": 0.5139198303222656,
"learning_rate": 0.00010720748619900363,
"loss": 2.8771,
"step": 122100
},
{
"epoch": 32.87155469422911,
"grad_norm": 0.5392455458641052,
"learning_rate": 0.00010700552039854584,
"loss": 2.8877,
"step": 122150
},
{
"epoch": 32.88501291989664,
"grad_norm": 0.49859973788261414,
"learning_rate": 0.00010680355459808805,
"loss": 2.9026,
"step": 122200
},
{
"epoch": 32.898471145564166,
"grad_norm": 0.5468138456344604,
"learning_rate": 0.00010660158879763027,
"loss": 2.8863,
"step": 122250
},
{
"epoch": 32.9119293712317,
"grad_norm": 0.5251834988594055,
"learning_rate": 0.00010639962299717247,
"loss": 2.8912,
"step": 122300
},
{
"epoch": 32.92538759689923,
"grad_norm": 0.5111126899719238,
"learning_rate": 0.00010619765719671468,
"loss": 2.8838,
"step": 122350
},
{
"epoch": 32.938845822566755,
"grad_norm": 0.5119268298149109,
"learning_rate": 0.00010599569139625689,
"loss": 2.8871,
"step": 122400
},
{
"epoch": 32.95230404823428,
"grad_norm": 0.5077241659164429,
"learning_rate": 0.0001057937255957991,
"loss": 2.8932,
"step": 122450
},
{
"epoch": 32.96576227390181,
"grad_norm": 0.5331259369850159,
"learning_rate": 0.00010559175979534132,
"loss": 2.8953,
"step": 122500
},
{
"epoch": 32.97922049956934,
"grad_norm": 0.4993937611579895,
"learning_rate": 0.00010538979399488353,
"loss": 2.8972,
"step": 122550
},
{
"epoch": 32.992678725236864,
"grad_norm": 0.5301192402839661,
"learning_rate": 0.00010518782819442573,
"loss": 2.894,
"step": 122600
},
{
"epoch": 33.005921619293716,
"grad_norm": 0.5341933369636536,
"learning_rate": 0.00010498586239396794,
"loss": 2.8617,
"step": 122650
},
{
"epoch": 33.01937984496124,
"grad_norm": 0.5181511044502258,
"learning_rate": 0.00010478389659351016,
"loss": 2.8456,
"step": 122700
},
{
"epoch": 33.03283807062877,
"grad_norm": 0.5161030888557434,
"learning_rate": 0.00010458193079305237,
"loss": 2.8368,
"step": 122750
},
{
"epoch": 33.0462962962963,
"grad_norm": 0.5211310386657715,
"learning_rate": 0.00010437996499259458,
"loss": 2.834,
"step": 122800
},
{
"epoch": 33.059754521963825,
"grad_norm": 0.5173688530921936,
"learning_rate": 0.00010417799919213679,
"loss": 2.8438,
"step": 122850
},
{
"epoch": 33.07321274763135,
"grad_norm": 0.5314739346504211,
"learning_rate": 0.000103976033391679,
"loss": 2.8433,
"step": 122900
},
{
"epoch": 33.08667097329888,
"grad_norm": 0.5037255883216858,
"learning_rate": 0.00010377406759122122,
"loss": 2.8509,
"step": 122950
},
{
"epoch": 33.10012919896641,
"grad_norm": 0.5201254487037659,
"learning_rate": 0.00010357210179076342,
"loss": 2.8568,
"step": 123000
},
{
"epoch": 33.10012919896641,
"eval_accuracy": 0.39814400749576384,
"eval_loss": 3.285991907119751,
"eval_runtime": 84.4772,
"eval_samples_per_second": 213.217,
"eval_steps_per_second": 13.329,
"step": 123000
},
{
"epoch": 33.113587424633934,
"grad_norm": 0.5263310074806213,
"learning_rate": 0.00010337013599030563,
"loss": 2.8527,
"step": 123050
},
{
"epoch": 33.12704565030146,
"grad_norm": 0.5239158868789673,
"learning_rate": 0.00010316817018984784,
"loss": 2.8515,
"step": 123100
},
{
"epoch": 33.14050387596899,
"grad_norm": 0.5073361396789551,
"learning_rate": 0.00010296620438939006,
"loss": 2.8486,
"step": 123150
},
{
"epoch": 33.15396210163652,
"grad_norm": 0.5051706433296204,
"learning_rate": 0.00010276423858893227,
"loss": 2.8489,
"step": 123200
},
{
"epoch": 33.16742032730405,
"grad_norm": 0.49514684081077576,
"learning_rate": 0.00010256227278847448,
"loss": 2.859,
"step": 123250
},
{
"epoch": 33.18087855297158,
"grad_norm": 0.5307257175445557,
"learning_rate": 0.00010236030698801669,
"loss": 2.8379,
"step": 123300
},
{
"epoch": 33.194336778639105,
"grad_norm": 0.5356868505477905,
"learning_rate": 0.00010215834118755888,
"loss": 2.8595,
"step": 123350
},
{
"epoch": 33.20779500430663,
"grad_norm": 0.5151222944259644,
"learning_rate": 0.00010195637538710111,
"loss": 2.8509,
"step": 123400
},
{
"epoch": 33.22125322997416,
"grad_norm": 0.547874391078949,
"learning_rate": 0.00010175440958664332,
"loss": 2.843,
"step": 123450
},
{
"epoch": 33.23471145564169,
"grad_norm": 0.5329242944717407,
"learning_rate": 0.00010155244378618553,
"loss": 2.86,
"step": 123500
},
{
"epoch": 33.248169681309214,
"grad_norm": 0.5350729823112488,
"learning_rate": 0.00010135047798572774,
"loss": 2.8592,
"step": 123550
},
{
"epoch": 33.26162790697674,
"grad_norm": 0.5196654200553894,
"learning_rate": 0.00010114851218526996,
"loss": 2.8569,
"step": 123600
},
{
"epoch": 33.27508613264427,
"grad_norm": 0.5498469471931458,
"learning_rate": 0.00010094654638481217,
"loss": 2.847,
"step": 123650
},
{
"epoch": 33.2885443583118,
"grad_norm": 0.5427135229110718,
"learning_rate": 0.00010074458058435438,
"loss": 2.8574,
"step": 123700
},
{
"epoch": 33.30200258397933,
"grad_norm": 0.5531679391860962,
"learning_rate": 0.00010054261478389658,
"loss": 2.8702,
"step": 123750
},
{
"epoch": 33.31546080964686,
"grad_norm": 0.4978818893432617,
"learning_rate": 0.0001003406489834388,
"loss": 2.8651,
"step": 123800
},
{
"epoch": 33.328919035314385,
"grad_norm": 0.5036234855651855,
"learning_rate": 0.00010013868318298101,
"loss": 2.8549,
"step": 123850
},
{
"epoch": 33.34237726098191,
"grad_norm": 0.5336086750030518,
"learning_rate": 9.993671738252322e-05,
"loss": 2.8537,
"step": 123900
},
{
"epoch": 33.35583548664944,
"grad_norm": 0.5408747792243958,
"learning_rate": 9.973475158206543e-05,
"loss": 2.861,
"step": 123950
},
{
"epoch": 33.36929371231697,
"grad_norm": 0.4914780855178833,
"learning_rate": 9.953278578160764e-05,
"loss": 2.8786,
"step": 124000
},
{
"epoch": 33.36929371231697,
"eval_accuracy": 0.398215062539303,
"eval_loss": 3.284298896789551,
"eval_runtime": 84.0215,
"eval_samples_per_second": 214.374,
"eval_steps_per_second": 13.401,
"step": 124000
},
{
"epoch": 33.36929371231697,
"step": 124000,
"total_flos": 2.591465108668416e+18,
"train_loss": 3.122740319282778,
"train_runtime": 84389.1333,
"train_samples_per_second": 140.879,
"train_steps_per_second": 1.761
}
],
"logging_steps": 50,
"max_steps": 148640,
"num_input_tokens_seen": 0,
"num_train_epochs": 40,
"save_steps": 10000,
"stateful_callbacks": {
"EarlyStoppingCallback": {
"args": {
"early_stopping_patience": 20,
"early_stopping_threshold": 0.0
},
"attributes": {
"early_stopping_patience_counter": 16
}
},
"TrainerControl": {
"args": {
"should_epoch_stop": false,
"should_evaluate": false,
"should_log": false,
"should_save": true,
"should_training_stop": false
},
"attributes": {}
}
},
"total_flos": 2.591465108668416e+18,
"train_batch_size": 16,
"trial_name": null,
"trial_params": null
}
{
"best_global_step": 104000,
"best_metric": 3.2727417945861816,
"best_model_checkpoint": "/scratch/cl5625/exceptions/models/high_2000_1208/checkpoint-20000",
"epoch": 33.36929371231697,
"eval_steps": 1000,
"global_step": 124000,
"is_hyper_param_search": false,
"is_local_process_zero": true,
"is_world_process_zero": true,
"log_history": [
{
"epoch": 0.013458225667527994,
"grad_norm": 0.8966911435127258,
"learning_rate": 0.000294,
"loss": 8.6065,
"step": 50
},
{
"epoch": 0.026916451335055987,
"grad_norm": 1.2048732042312622,
"learning_rate": 0.0005939999999999999,
"loss": 6.8585,
"step": 100
},
{
"epoch": 0.04037467700258398,
"grad_norm": 0.9153667688369751,
"learning_rate": 0.0005998020735155513,
"loss": 6.3263,
"step": 150
},
{
"epoch": 0.053832902670111975,
"grad_norm": 1.4322563409805298,
"learning_rate": 0.0005996001077150935,
"loss": 6.0257,
"step": 200
},
{
"epoch": 0.06729112833763996,
"grad_norm": 1.2138304710388184,
"learning_rate": 0.0005993981419146358,
"loss": 5.865,
"step": 250
},
{
"epoch": 0.08074935400516796,
"grad_norm": 1.4077575206756592,
"learning_rate": 0.0005991961761141779,
"loss": 5.7384,
"step": 300
},
{
"epoch": 0.09420757967269595,
"grad_norm": 1.1934620141983032,
"learning_rate": 0.0005989942103137202,
"loss": 5.6093,
"step": 350
},
{
"epoch": 0.10766580534022395,
"grad_norm": 1.1857770681381226,
"learning_rate": 0.0005987922445132624,
"loss": 5.4578,
"step": 400
},
{
"epoch": 0.12112403100775193,
"grad_norm": 1.8576644659042358,
"learning_rate": 0.0005985902787128047,
"loss": 5.3535,
"step": 450
},
{
"epoch": 0.13458225667527993,
"grad_norm": 1.6441043615341187,
"learning_rate": 0.0005983883129123468,
"loss": 5.2506,
"step": 500
},
{
"epoch": 0.14804048234280792,
"grad_norm": 1.3366986513137817,
"learning_rate": 0.000598186347111889,
"loss": 5.14,
"step": 550
},
{
"epoch": 0.16149870801033592,
"grad_norm": 1.1094809770584106,
"learning_rate": 0.0005979843813114312,
"loss": 5.0689,
"step": 600
},
{
"epoch": 0.1749569336778639,
"grad_norm": 1.1344627141952515,
"learning_rate": 0.0005977824155109734,
"loss": 4.9848,