Instructions to use AutoSurferForCopilot/RedditQwen3.5ToolFt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AutoSurferForCopilot/RedditQwen3.5ToolFt with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="AutoSurferForCopilot/RedditQwen3.5ToolFt")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("AutoSurferForCopilot/RedditQwen3.5ToolFt")
model = AutoModelForMultimodalLM.from_pretrained("AutoSurferForCopilot/RedditQwen3.5ToolFt", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AutoSurferForCopilot/RedditQwen3.5ToolFt with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AutoSurferForCopilot/RedditQwen3.5ToolFt"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AutoSurferForCopilot/RedditQwen3.5ToolFt",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/AutoSurferForCopilot/RedditQwen3.5ToolFt

SGLang

How to use AutoSurferForCopilot/RedditQwen3.5ToolFt with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AutoSurferForCopilot/RedditQwen3.5ToolFt" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AutoSurferForCopilot/RedditQwen3.5ToolFt",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AutoSurferForCopilot/RedditQwen3.5ToolFt" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AutoSurferForCopilot/RedditQwen3.5ToolFt",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use AutoSurferForCopilot/RedditQwen3.5ToolFt with Docker Model Runner:
```
docker model run hf.co/AutoSurferForCopilot/RedditQwen3.5ToolFt
```

RedditQwen3.5ToolFt / trainer_state.json

faisalcse98

Upload folder using huggingface_hub

5cd344a verified about 2 months ago

Raw History Blame Contribute Delete

95.2 kB

	{
	"best_global_step": null,
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 3.0,
	"eval_steps": 500,
	"global_step": 5004,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"epoch": 0.005997001499250375,
	"grad_norm": 33.177966541111424,
	"learning_rate": 1.7964071856287425e-07,
	"loss": 1.153841209411621,
	"step": 10
	},
	{
	"epoch": 0.01199400299850075,
	"grad_norm": 20.328089115017978,
	"learning_rate": 3.792415169660679e-07,
	"loss": 1.0805124282836913,
	"step": 20
	},
	{
	"epoch": 0.017991004497751123,
	"grad_norm": 10.218005071755792,
	"learning_rate": 5.788423153692615e-07,
	"loss": 0.8860118865966797,
	"step": 30
	},
	{
	"epoch": 0.0239880059970015,
	"grad_norm": 7.938230439311131,
	"learning_rate": 7.784431137724552e-07,
	"loss": 0.7478219985961914,
	"step": 40
	},
	{
	"epoch": 0.029985007496251874,
	"grad_norm": 7.18637715739778,
	"learning_rate": 9.780439121756488e-07,
	"loss": 0.6388802528381348,
	"step": 50
	},
	{
	"epoch": 0.035982008995502246,
	"grad_norm": 9.847931360180496,
	"learning_rate": 1.1776447105788423e-06,
	"loss": 0.5508933067321777,
	"step": 60
	},
	{
	"epoch": 0.041979010494752625,
	"grad_norm": 6.140194695978427,
	"learning_rate": 1.377245508982036e-06,
	"loss": 0.5095039367675781,
	"step": 70
	},
	{
	"epoch": 0.047976011994003,
	"grad_norm": 4.2875493005127785,
	"learning_rate": 1.5768463073852298e-06,
	"loss": 0.46800861358642576,
	"step": 80
	},
	{
	"epoch": 0.053973013493253376,
	"grad_norm": 5.100614923580563,
	"learning_rate": 1.7764471057884233e-06,
	"loss": 0.45592150688171384,
	"step": 90
	},
	{
	"epoch": 0.05997001499250375,
	"grad_norm": 5.127997375054051,
	"learning_rate": 1.976047904191617e-06,
	"loss": 0.4315938949584961,
	"step": 100
	},
	{
	"epoch": 0.06596701649175413,
	"grad_norm": 5.903760016475037,
	"learning_rate": 2.1756487025948105e-06,
	"loss": 0.4375575065612793,
	"step": 110
	},
	{
	"epoch": 0.07196401799100449,
	"grad_norm": 5.236597145761759,
	"learning_rate": 2.3752495009980044e-06,
	"loss": 0.404527473449707,
	"step": 120
	},
	{
	"epoch": 0.07796101949025487,
	"grad_norm": 6.02025799268674,
	"learning_rate": 2.5748502994011975e-06,
	"loss": 0.4035046100616455,
	"step": 130
	},
	{
	"epoch": 0.08395802098950525,
	"grad_norm": 4.818873465598998,
	"learning_rate": 2.7744510978043914e-06,
	"loss": 0.4054649829864502,
	"step": 140
	},
	{
	"epoch": 0.08995502248875563,
	"grad_norm": 5.833094616177031,
	"learning_rate": 2.974051896207585e-06,
	"loss": 0.3761675596237183,
	"step": 150
	},
	{
	"epoch": 0.095952023988006,
	"grad_norm": 3.466602113358112,
	"learning_rate": 3.173652694610779e-06,
	"loss": 0.37643632888793943,
	"step": 160
	},
	{
	"epoch": 0.10194902548725637,
	"grad_norm": 4.599646273464669,
	"learning_rate": 3.373253493013972e-06,
	"loss": 0.37375531196594236,
	"step": 170
	},
	{
	"epoch": 0.10794602698650675,
	"grad_norm": 4.186786740506348,
	"learning_rate": 3.572854291417166e-06,
	"loss": 0.3703818082809448,
	"step": 180
	},
	{
	"epoch": 0.11394302848575712,
	"grad_norm": 3.4106803557661642,
	"learning_rate": 3.7724550898203594e-06,
	"loss": 0.36371560096740724,
	"step": 190
	},
	{
	"epoch": 0.1199400299850075,
	"grad_norm": 4.13031284430083,
	"learning_rate": 3.972055888223553e-06,
	"loss": 0.3487649917602539,
	"step": 200
	},
	{
	"epoch": 0.12593703148425786,
	"grad_norm": 4.544792101101583,
	"learning_rate": 4.171656686626747e-06,
	"loss": 0.3448129177093506,
	"step": 210
	},
	{
	"epoch": 0.13193403298350825,
	"grad_norm": 3.6705225567853903,
	"learning_rate": 4.371257485029941e-06,
	"loss": 0.3515609264373779,
	"step": 220
	},
	{
	"epoch": 0.13793103448275862,
	"grad_norm": 3.740380731181647,
	"learning_rate": 4.570858283433134e-06,
	"loss": 0.33996570110321045,
	"step": 230
	},
	{
	"epoch": 0.14392803598200898,
	"grad_norm": 4.12597725308824,
	"learning_rate": 4.770459081836328e-06,
	"loss": 0.3355365991592407,
	"step": 240
	},
	{
	"epoch": 0.14992503748125938,
	"grad_norm": 5.111580927209964,
	"learning_rate": 4.970059880239521e-06,
	"loss": 0.3289555311203003,
	"step": 250
	},
	{
	"epoch": 0.15592203898050974,
	"grad_norm": 3.2316447676635636,
	"learning_rate": 5.169660678642715e-06,
	"loss": 0.3073996067047119,
	"step": 260
	},
	{
	"epoch": 0.1619190404797601,
	"grad_norm": 3.2213363385484164,
	"learning_rate": 5.369261477045909e-06,
	"loss": 0.33562283515930175,
	"step": 270
	},
	{
	"epoch": 0.1679160419790105,
	"grad_norm": 3.248477894758676,
	"learning_rate": 5.568862275449102e-06,
	"loss": 0.3224648475646973,
	"step": 280
	},
	{
	"epoch": 0.17391304347826086,
	"grad_norm": 4.475016970090594,
	"learning_rate": 5.7684630738522965e-06,
	"loss": 0.3164596796035767,
	"step": 290
	},
	{
	"epoch": 0.17991004497751126,
	"grad_norm": 3.8353887369410598,
	"learning_rate": 5.96806387225549e-06,
	"loss": 0.315293550491333,
	"step": 300
	},
	{
	"epoch": 0.18590704647676162,
	"grad_norm": 3.8982388379090023,
	"learning_rate": 6.167664670658683e-06,
	"loss": 0.3085637092590332,
	"step": 310
	},
	{
	"epoch": 0.191904047976012,
	"grad_norm": 3.128884727056146,
	"learning_rate": 6.367265469061877e-06,
	"loss": 0.3139899253845215,
	"step": 320
	},
	{
	"epoch": 0.19790104947526238,
	"grad_norm": 3.1670593299951375,
	"learning_rate": 6.5668662674650705e-06,
	"loss": 0.2892496585845947,
	"step": 330
	},
	{
	"epoch": 0.20389805097451275,
	"grad_norm": 2.715817297324109,
	"learning_rate": 6.7664670658682645e-06,
	"loss": 0.2972956895828247,
	"step": 340
	},
	{
	"epoch": 0.2098950524737631,
	"grad_norm": 3.0758427429703343,
	"learning_rate": 6.9660678642714575e-06,
	"loss": 0.28812122344970703,
	"step": 350
	},
	{
	"epoch": 0.2158920539730135,
	"grad_norm": 2.7229608872037896,
	"learning_rate": 7.165668662674651e-06,
	"loss": 0.28461265563964844,
	"step": 360
	},
	{
	"epoch": 0.22188905547226387,
	"grad_norm": 2.9910149568769664,
	"learning_rate": 7.365269461077845e-06,
	"loss": 0.307127046585083,
	"step": 370
	},
	{
	"epoch": 0.22788605697151423,
	"grad_norm": 3.99391283462534,
	"learning_rate": 7.5648702594810385e-06,
	"loss": 0.2821415901184082,
	"step": 380
	},
	{
	"epoch": 0.23388305847076463,
	"grad_norm": 3.2299949922964064,
	"learning_rate": 7.764471057884232e-06,
	"loss": 0.30055310726165774,
	"step": 390
	},
	{
	"epoch": 0.239880059970015,
	"grad_norm": 2.42324047102494,
	"learning_rate": 7.964071856287425e-06,
	"loss": 0.26854233741760253,
	"step": 400
	},
	{
	"epoch": 0.24587706146926536,
	"grad_norm": 2.8083365616370775,
	"learning_rate": 8.16367265469062e-06,
	"loss": 0.26332242488861085,
	"step": 410
	},
	{
	"epoch": 0.2518740629685157,
	"grad_norm": 2.582528249302077,
	"learning_rate": 8.363273453093813e-06,
	"loss": 0.28701660633087156,
	"step": 420
	},
	{
	"epoch": 0.25787106446776614,
	"grad_norm": 4.248385285155867,
	"learning_rate": 8.562874251497007e-06,
	"loss": 0.26176862716674804,
	"step": 430
	},
	{
	"epoch": 0.2638680659670165,
	"grad_norm": 2.7518510032156542,
	"learning_rate": 8.7624750499002e-06,
	"loss": 0.262543249130249,
	"step": 440
	},
	{
	"epoch": 0.2698650674662669,
	"grad_norm": 2.619448001692531,
	"learning_rate": 8.962075848303395e-06,
	"loss": 0.2730741500854492,
	"step": 450
	},
	{
	"epoch": 0.27586206896551724,
	"grad_norm": 2.495415683566701,
	"learning_rate": 9.161676646706587e-06,
	"loss": 0.24910275936126708,
	"step": 460
	},
	{
	"epoch": 0.2818590704647676,
	"grad_norm": 2.663412861844891,
	"learning_rate": 9.361277445109781e-06,
	"loss": 0.23290033340454103,
	"step": 470
	},
	{
	"epoch": 0.28785607196401797,
	"grad_norm": 2.750468380692841,
	"learning_rate": 9.560878243512974e-06,
	"loss": 0.22923564910888672,
	"step": 480
	},
	{
	"epoch": 0.2938530734632684,
	"grad_norm": 2.8552706150088203,
	"learning_rate": 9.760479041916169e-06,
	"loss": 0.22435421943664552,
	"step": 490
	},
	{
	"epoch": 0.29985007496251875,
	"grad_norm": 2.506778242866157,
	"learning_rate": 9.960079840319361e-06,
	"loss": 0.23597168922424316,
	"step": 500
	},
	{
	"epoch": 0.3058470764617691,
	"grad_norm": 2.72898073861673,
	"learning_rate": 9.999922122014899e-06,
	"loss": 0.24130680561065673,
	"step": 510
	},
	{
	"epoch": 0.3118440779610195,
	"grad_norm": 2.3776420425944056,
	"learning_rate": 9.99960574685824e-06,
	"loss": 0.2126417875289917,
	"step": 520
	},
	{
	"epoch": 0.31784107946026985,
	"grad_norm": 2.6173666721487603,
	"learning_rate": 9.999046022543255e-06,
	"loss": 0.21457178592681886,
	"step": 530
	},
	{
	"epoch": 0.3238380809595202,
	"grad_norm": 2.0818807157318346,
	"learning_rate": 9.998242976313777e-06,
	"loss": 0.20246849060058594,
	"step": 540
	},
	{
	"epoch": 0.32983508245877063,
	"grad_norm": 2.3552710317352283,
	"learning_rate": 9.997196647257012e-06,
	"loss": 0.2250359535217285,
	"step": 550
	},
	{
	"epoch": 0.335832083958021,
	"grad_norm": 4.118901168898432,
	"learning_rate": 9.995907086301639e-06,
	"loss": 0.22331862449645995,
	"step": 560
	},
	{
	"epoch": 0.34182908545727136,
	"grad_norm": 3.107722987815673,
	"learning_rate": 9.994374356215318e-06,
	"loss": 0.21423397064208985,
	"step": 570
	},
	{
	"epoch": 0.34782608695652173,
	"grad_norm": 1.989727964063391,
	"learning_rate": 9.992598531601646e-06,
	"loss": 0.21912760734558107,
	"step": 580
	},
	{
	"epoch": 0.3538230884557721,
	"grad_norm": 2.6704308959689036,
	"learning_rate": 9.990579698896527e-06,
	"loss": 0.2012500286102295,
	"step": 590
	},
	{
	"epoch": 0.3598200899550225,
	"grad_norm": 2.2648389459056677,
	"learning_rate": 9.988317956363956e-06,
	"loss": 0.18341922760009766,
	"step": 600
	},
	{
	"epoch": 0.3658170914542729,
	"grad_norm": 1.8127958595539686,
	"learning_rate": 9.98581341409124e-06,
	"loss": 0.1958906888961792,
	"step": 610
	},
	{
	"epoch": 0.37181409295352325,
	"grad_norm": 2.9664023703432147,
	"learning_rate": 9.983066193983646e-06,
	"loss": 0.1815837502479553,
	"step": 620
	},
	{
	"epoch": 0.3778110944527736,
	"grad_norm": 1.862681161226939,
	"learning_rate": 9.980076429758455e-06,
	"loss": 0.18129918575286866,
	"step": 630
	},
	{
	"epoch": 0.383808095952024,
	"grad_norm": 2.026298189078191,
	"learning_rate": 9.976844266938462e-06,
	"loss": 0.19830081462860108,
	"step": 640
	},
	{
	"epoch": 0.38980509745127434,
	"grad_norm": 2.188624481141618,
	"learning_rate": 9.973369862844894e-06,
	"loss": 0.19266587495803833,
	"step": 650
	},
	{
	"epoch": 0.39580209895052476,
	"grad_norm": 2.036147902644092,
	"learning_rate": 9.969653386589749e-06,
	"loss": 0.1910475969314575,
	"step": 660
	},
	{
	"epoch": 0.4017991004497751,
	"grad_norm": 2.2008635646952968,
	"learning_rate": 9.96569501906756e-06,
	"loss": 0.1740795135498047,
	"step": 670
	},
	{
	"epoch": 0.4077961019490255,
	"grad_norm": 2.03051911897634,
	"learning_rate": 9.961494952946607e-06,
	"loss": 0.15912220478057862,
	"step": 680
	},
	{
	"epoch": 0.41379310344827586,
	"grad_norm": 2.258955637268267,
	"learning_rate": 9.957053392659515e-06,
	"loss": 0.15781991481781005,
	"step": 690
	},
	{
	"epoch": 0.4197901049475262,
	"grad_norm": 2.085338222614203,
	"learning_rate": 9.952370554393324e-06,
	"loss": 0.14586501121520995,
	"step": 700
	},
	{
	"epoch": 0.4257871064467766,
	"grad_norm": 2.6423928119518685,
	"learning_rate": 9.947446666078956e-06,
	"loss": 0.17084224224090577,
	"step": 710
	},
	{
	"epoch": 0.431784107946027,
	"grad_norm": 1.742402104451892,
	"learning_rate": 9.942281967380125e-06,
	"loss": 0.15098991394042968,
	"step": 720
	},
	{
	"epoch": 0.43778110944527737,
	"grad_norm": 1.8829882904051813,
	"learning_rate": 9.936876709681668e-06,
	"loss": 0.15784955024719238,
	"step": 730
	},
	{
	"epoch": 0.44377811094452774,
	"grad_norm": 2.423652655249898,
	"learning_rate": 9.931231156077314e-06,
	"loss": 0.13576219081878663,
	"step": 740
	},
	{
	"epoch": 0.4497751124437781,
	"grad_norm": 2.565748236024762,
	"learning_rate": 9.925345581356872e-06,
	"loss": 0.13479468822479249,
	"step": 750
	},
	{
	"epoch": 0.45577211394302847,
	"grad_norm": 2.1612993803364295,
	"learning_rate": 9.919220271992862e-06,
	"loss": 0.14143543243408202,
	"step": 760
	},
	{
	"epoch": 0.4617691154422789,
	"grad_norm": 2.090996730591019,
	"learning_rate": 9.91285552612657e-06,
	"loss": 0.14490773677825927,
	"step": 770
	},
	{
	"epoch": 0.46776611694152925,
	"grad_norm": 2.460836892186411,
	"learning_rate": 9.906251653553532e-06,
	"loss": 0.11662017107009888,
	"step": 780
	},
	{
	"epoch": 0.4737631184407796,
	"grad_norm": 1.8532341798318503,
	"learning_rate": 9.899408975708457e-06,
	"loss": 0.11657997369766235,
	"step": 790
	},
	{
	"epoch": 0.47976011994003,
	"grad_norm": 2.1616051031010417,
	"learning_rate": 9.892327825649593e-06,
	"loss": 0.13115763664245605,
	"step": 800
	},
	{
	"epoch": 0.48575712143928035,
	"grad_norm": 1.7234672722123352,
	"learning_rate": 9.885008548042492e-06,
	"loss": 0.13340814113616944,
	"step": 810
	},
	{
	"epoch": 0.4917541229385307,
	"grad_norm": 2.2508735089960874,
	"learning_rate": 9.877451499143256e-06,
	"loss": 0.11559282541275025,
	"step": 820
	},
	{
	"epoch": 0.49775112443778113,
	"grad_norm": 2.2639799270500354,
	"learning_rate": 9.869657046781186e-06,
	"loss": 0.124329674243927,
	"step": 830
	},
	{
	"epoch": 0.5037481259370314,
	"grad_norm": 2.119138803956051,
	"learning_rate": 9.861625570340884e-06,
	"loss": 0.12352757453918457,
	"step": 840
	},
	{
	"epoch": 0.5097451274362819,
	"grad_norm": 2.5409964556798377,
	"learning_rate": 9.853357460743777e-06,
	"loss": 0.10006859302520751,
	"step": 850
	},
	{
	"epoch": 0.5157421289355323,
	"grad_norm": 1.967499854259429,
	"learning_rate": 9.8448531204291e-06,
	"loss": 0.12050327062606811,
	"step": 860
	},
	{
	"epoch": 0.5217391304347826,
	"grad_norm": 1.987469417641311,
	"learning_rate": 9.836112963334308e-06,
	"loss": 0.1027034044265747,
	"step": 870
	},
	{
	"epoch": 0.527736131934033,
	"grad_norm": 2.453990944810916,
	"learning_rate": 9.827137414874914e-06,
	"loss": 0.09475643038749695,
	"step": 880
	},
	{
	"epoch": 0.5337331334332833,
	"grad_norm": 1.4711218004236777,
	"learning_rate": 9.817926911923803e-06,
	"loss": 0.08681859970092773,
	"step": 890
	},
	{
	"epoch": 0.5397301349325337,
	"grad_norm": 2.346221977828751,
	"learning_rate": 9.80848190278995e-06,
	"loss": 0.12321888208389283,
	"step": 900
	},
	{
	"epoch": 0.545727136431784,
	"grad_norm": 2.2200494897477348,
	"learning_rate": 9.798802847196612e-06,
	"loss": 0.08713321089744568,
	"step": 910
	},
	{
	"epoch": 0.5517241379310345,
	"grad_norm": 1.8287745985819173,
	"learning_rate": 9.78889021625894e-06,
	"loss": 0.10032504796981812,
	"step": 920
	},
	{
	"epoch": 0.5577211394302849,
	"grad_norm": 2.3670542580247322,
	"learning_rate": 9.778744492461056e-06,
	"loss": 0.08474494218826294,
	"step": 930
	},
	{
	"epoch": 0.5637181409295352,
	"grad_norm": 1.900658078691811,
	"learning_rate": 9.768366169632573e-06,
	"loss": 0.07685534358024597,
	"step": 940
	},
	{
	"epoch": 0.5697151424287856,
	"grad_norm": 1.7382481142038348,
	"learning_rate": 9.757755752924544e-06,
	"loss": 0.07947651743888855,
	"step": 950
	},
	{
	"epoch": 0.5757121439280359,
	"grad_norm": 2.358077726874596,
	"learning_rate": 9.746913758784888e-06,
	"loss": 0.08233010172843933,
	"step": 960
	},
	{
	"epoch": 0.5817091454272864,
	"grad_norm": 1.9390751285478798,
	"learning_rate": 9.735840714933245e-06,
	"loss": 0.08379462957382203,
	"step": 970
	},
	{
	"epoch": 0.5877061469265368,
	"grad_norm": 2.312898693528998,
	"learning_rate": 9.724537160335293e-06,
	"loss": 0.07934452295303344,
	"step": 980
	},
	{
	"epoch": 0.5937031484257871,
	"grad_norm": 1.7558978095237687,
	"learning_rate": 9.713003645176514e-06,
	"loss": 0.06791763305664063,
	"step": 990
	},
	{
	"epoch": 0.5997001499250375,
	"grad_norm": 1.4691764102832905,
	"learning_rate": 9.701240730835415e-06,
	"loss": 0.0725862979888916,
	"step": 1000
	},
	{
	"epoch": 0.6056971514242878,
	"grad_norm": 1.607319864119255,
	"learning_rate": 9.689248989856202e-06,
	"loss": 0.06207773685455322,
	"step": 1010
	},
	{
	"epoch": 0.6116941529235382,
	"grad_norm": 1.5867408510983712,
	"learning_rate": 9.677029005920913e-06,
	"loss": 0.08114245533943176,
	"step": 1020
	},
	{
	"epoch": 0.6176911544227887,
	"grad_norm": 1.4316957183601544,
	"learning_rate": 9.664581373821009e-06,
	"loss": 0.07747445702552795,
	"step": 1030
	},
	{
	"epoch": 0.623688155922039,
	"grad_norm": 1.2489210362045546,
	"learning_rate": 9.651906699428421e-06,
	"loss": 0.07207693457603455,
	"step": 1040
	},
	{
	"epoch": 0.6296851574212894,
	"grad_norm": 1.5333099011546873,
	"learning_rate": 9.639005599666066e-06,
	"loss": 0.06964211463928223,
	"step": 1050
	},
	{
	"epoch": 0.6356821589205397,
	"grad_norm": 1.4517928211398854,
	"learning_rate": 9.625878702477808e-06,
	"loss": 0.06729897856712341,
	"step": 1060
	},
	{
	"epoch": 0.6416791604197901,
	"grad_norm": 2.335473191776381,
	"learning_rate": 9.612526646797909e-06,
	"loss": 0.0634132981300354,
	"step": 1070
	},
	{
	"epoch": 0.6476761619190404,
	"grad_norm": 1.6920817630592229,
	"learning_rate": 9.59895008251991e-06,
	"loss": 0.05858844518661499,
	"step": 1080
	},
	{
	"epoch": 0.6536731634182908,
	"grad_norm": 1.7135181425646213,
	"learning_rate": 9.585149670465018e-06,
	"loss": 0.08531377315521241,
	"step": 1090
	},
	{
	"epoch": 0.6596701649175413,
	"grad_norm": 1.7879448453989304,
	"learning_rate": 9.57112608234993e-06,
	"loss": 0.06577427387237549,
	"step": 1100
	},
	{
	"epoch": 0.6656671664167916,
	"grad_norm": 1.8030036958753215,
	"learning_rate": 9.55688000075414e-06,
	"loss": 0.06042234897613526,
	"step": 1110
	},
	{
	"epoch": 0.671664167916042,
	"grad_norm": 1.4042571452498804,
	"learning_rate": 9.542412119086717e-06,
	"loss": 0.06413464546203614,
	"step": 1120
	},
	{
	"epoch": 0.6776611694152923,
	"grad_norm": 2.51407097901955,
	"learning_rate": 9.527723141552555e-06,
	"loss": 0.0614790678024292,
	"step": 1130
	},
	{
	"epoch": 0.6836581709145427,
	"grad_norm": 2.4615159687048194,
	"learning_rate": 9.512813783118091e-06,
	"loss": 0.04664933979511261,
	"step": 1140
	},
	{
	"epoch": 0.6896551724137931,
	"grad_norm": 2.249941319353258,
	"learning_rate": 9.497684769476512e-06,
	"loss": 0.05516514778137207,
	"step": 1150
	},
	{
	"epoch": 0.6956521739130435,
	"grad_norm": 2.0047882037503477,
	"learning_rate": 9.48233683701243e-06,
	"loss": 0.06430045962333679,
	"step": 1160
	},
	{
	"epoch": 0.7016491754122939,
	"grad_norm": 1.773918235664191,
	"learning_rate": 9.46677073276604e-06,
	"loss": 0.056384491920471194,
	"step": 1170
	},
	{
	"epoch": 0.7076461769115442,
	"grad_norm": 1.7462939879037267,
	"learning_rate": 9.450987214396756e-06,
	"loss": 0.05259143114089966,
	"step": 1180
	},
	{
	"epoch": 0.7136431784107946,
	"grad_norm": 1.37405397516532,
	"learning_rate": 9.434987050146337e-06,
	"loss": 0.058822852373123166,
	"step": 1190
	},
	{
	"epoch": 0.719640179910045,
	"grad_norm": 2.2880200526201104,
	"learning_rate": 9.418771018801493e-06,
	"loss": 0.059318751096725464,
	"step": 1200
	},
	{
	"epoch": 0.7256371814092953,
	"grad_norm": 2.0962825216087015,
	"learning_rate": 9.402339909655973e-06,
	"loss": 0.04961778521537781,
	"step": 1210
	},
	{
	"epoch": 0.7316341829085458,
	"grad_norm": 1.7218940686629112,
	"learning_rate": 9.385694522472157e-06,
	"loss": 0.04554992020130157,
	"step": 1220
	},
	{
	"epoch": 0.7376311844077961,
	"grad_norm": 1.1342373864950697,
	"learning_rate": 9.368835667442121e-06,
	"loss": 0.04457632303237915,
	"step": 1230
	},
	{
	"epoch": 0.7436281859070465,
	"grad_norm": 2.1390110207961217,
	"learning_rate": 9.351764165148207e-06,
	"loss": 0.05258376598358154,
	"step": 1240
	},
	{
	"epoch": 0.7496251874062968,
	"grad_norm": 1.372689552387907,
	"learning_rate": 9.334480846523073e-06,
	"loss": 0.05245867967605591,
	"step": 1250
	},
	{
	"epoch": 0.7556221889055472,
	"grad_norm": 1.882308552978412,
	"learning_rate": 9.316986552809262e-06,
	"loss": 0.046319738030433655,
	"step": 1260
	},
	{
	"epoch": 0.7616191904047976,
	"grad_norm": 1.3041834408477797,
	"learning_rate": 9.299282135518246e-06,
	"loss": 0.04240759611129761,
	"step": 1270
	},
	{
	"epoch": 0.767616191904048,
	"grad_norm": 1.0505082276383715,
	"learning_rate": 9.281368456388983e-06,
	"loss": 0.04255787432193756,
	"step": 1280
	},
	{
	"epoch": 0.7736131934032984,
	"grad_norm": 1.3355656203132729,
	"learning_rate": 9.263246387345975e-06,
	"loss": 0.035089099407196046,
	"step": 1290
	},
	{
	"epoch": 0.7796101949025487,
	"grad_norm": 1.404332843900438,
	"learning_rate": 9.244916810456822e-06,
	"loss": 0.043932384252548216,
	"step": 1300
	},
	{
	"epoch": 0.7856071964017991,
	"grad_norm": 1.4968078819353412,
	"learning_rate": 9.226380617889289e-06,
	"loss": 0.044729065895080564,
	"step": 1310
	},
	{
	"epoch": 0.7916041979010495,
	"grad_norm": 2.6067015691134827,
	"learning_rate": 9.207638711867894e-06,
	"loss": 0.038001728057861325,
	"step": 1320
	},
	{
	"epoch": 0.7976011994002998,
	"grad_norm": 1.5374594433192899,
	"learning_rate": 9.188692004629979e-06,
	"loss": 0.04036279916763306,
	"step": 1330
	},
	{
	"epoch": 0.8035982008995503,
	"grad_norm": 1.3549451816597577,
	"learning_rate": 9.169541418381309e-06,
	"loss": 0.03851274251937866,
	"step": 1340
	},
	{
	"epoch": 0.8095952023988006,
	"grad_norm": 1.1087489652602784,
	"learning_rate": 9.150187885251194e-06,
	"loss": 0.03992776274681091,
	"step": 1350
	},
	{
	"epoch": 0.815592203898051,
	"grad_norm": 1.4886357528414471,
	"learning_rate": 9.130632347247113e-06,
	"loss": 0.040539324283599854,
	"step": 1360
	},
	{
	"epoch": 0.8215892053973014,
	"grad_norm": 1.9023871437983941,
	"learning_rate": 9.11087575620886e-06,
	"loss": 0.03769837617874146,
	"step": 1370
	},
	{
	"epoch": 0.8275862068965517,
	"grad_norm": 1.257782068004155,
	"learning_rate": 9.09091907376222e-06,
	"loss": 0.041015252470970154,
	"step": 1380
	},
	{
	"epoch": 0.8335832083958021,
	"grad_norm": 1.3029772002616744,
	"learning_rate": 9.07076327127216e-06,
	"loss": 0.04067923426628113,
	"step": 1390
	},
	{
	"epoch": 0.8395802098950524,
	"grad_norm": 0.9024496277323651,
	"learning_rate": 9.050409329795553e-06,
	"loss": 0.034747010469436644,
	"step": 1400
	},
	{
	"epoch": 0.8455772113943029,
	"grad_norm": 1.1856806126975943,
	"learning_rate": 9.02985824003342e-06,
	"loss": 0.033799558877944946,
	"step": 1410
	},
	{
	"epoch": 0.8515742128935532,
	"grad_norm": 1.3765504507924757,
	"learning_rate": 9.009111002282718e-06,
	"loss": 0.037110593914985654,
	"step": 1420
	},
	{
	"epoch": 0.8575712143928036,
	"grad_norm": 2.3110145956068835,
	"learning_rate": 8.98816862638764e-06,
	"loss": 0.03612796664237976,
	"step": 1430
	},
	{
	"epoch": 0.863568215892054,
	"grad_norm": 1.4718894760529166,
	"learning_rate": 8.96703213169047e-06,
	"loss": 0.030382347106933594,
	"step": 1440
	},
	{
	"epoch": 0.8695652173913043,
	"grad_norm": 1.4814190462409382,
	"learning_rate": 8.94570254698197e-06,
	"loss": 0.029476076364517212,
	"step": 1450
	},
	{
	"epoch": 0.8755622188905547,
	"grad_norm": 1.0010659006363771,
	"learning_rate": 8.924180910451297e-06,
	"loss": 0.037569868564605716,
	"step": 1460
	},
	{
	"epoch": 0.881559220389805,
	"grad_norm": 1.1847678223897569,
	"learning_rate": 8.90246826963548e-06,
	"loss": 0.03867210745811463,
	"step": 1470
	},
	{
	"epoch": 0.8875562218890555,
	"grad_norm": 3.033954232508573,
	"learning_rate": 8.880565681368423e-06,
	"loss": 0.035211512446403505,
	"step": 1480
	},
	{
	"epoch": 0.8935532233883059,
	"grad_norm": 1.7658300186993254,
	"learning_rate": 8.85847421172947e-06,
	"loss": 0.0327178955078125,
	"step": 1490
	},
	{
	"epoch": 0.8995502248875562,
	"grad_norm": 1.0891133523228218,
	"learning_rate": 8.836194935991517e-06,
	"loss": 0.0287791907787323,
	"step": 1500
	},
	{
	"epoch": 0.9055472263868066,
	"grad_norm": 1.2995796391964896,
	"learning_rate": 8.81372893856867e-06,
	"loss": 0.023604214191436768,
	"step": 1510
	},
	{
	"epoch": 0.9115442278860569,
	"grad_norm": 1.1040502257750737,
	"learning_rate": 8.791077312963465e-06,
	"loss": 0.026106435060501098,
	"step": 1520
	},
	{
	"epoch": 0.9175412293853074,
	"grad_norm": 1.6141831640012578,
	"learning_rate": 8.768241161713644e-06,
	"loss": 0.03596283197402954,
	"step": 1530
	},
	{
	"epoch": 0.9235382308845578,
	"grad_norm": 0.8349977559967721,
	"learning_rate": 8.745221596338486e-06,
	"loss": 0.028143364191055297,
	"step": 1540
	},
	{
	"epoch": 0.9295352323838081,
	"grad_norm": 1.1149619690588555,
	"learning_rate": 8.722019737284715e-06,
	"loss": 0.025614866614341737,
	"step": 1550
	},
	{
	"epoch": 0.9355322338830585,
	"grad_norm": 0.8941277596682687,
	"learning_rate": 8.698636713871948e-06,
	"loss": 0.02644273042678833,
	"step": 1560
	},
	{
	"epoch": 0.9415292353823088,
	"grad_norm": 1.4818648032407833,
	"learning_rate": 8.675073664237744e-06,
	"loss": 0.030157488584518433,
	"step": 1570
	},
	{
	"epoch": 0.9475262368815592,
	"grad_norm": 1.0283152894445524,
	"learning_rate": 8.651331735282194e-06,
	"loss": 0.026942259073257445,
	"step": 1580
	},
	{
	"epoch": 0.9535232383808095,
	"grad_norm": 1.0436195468699556,
	"learning_rate": 8.627412082612105e-06,
	"loss": 0.020822218060493468,
	"step": 1590
	},
	{
	"epoch": 0.95952023988006,
	"grad_norm": 1.625712990583743,
	"learning_rate": 8.603315870484752e-06,
	"loss": 0.02752835154533386,
	"step": 1600
	},
	{
	"epoch": 0.9655172413793104,
	"grad_norm": 1.3657178238805856,
	"learning_rate": 8.579044271751198e-06,
	"loss": 0.03032808005809784,
	"step": 1610
	},
	{
	"epoch": 0.9715142428785607,
	"grad_norm": 1.3371695715282004,
	"learning_rate": 8.554598467799223e-06,
	"loss": 0.02459574192762375,
	"step": 1620
	},
	{
	"epoch": 0.9775112443778111,
	"grad_norm": 1.1078316928380114,
	"learning_rate": 8.529979648495811e-06,
	"loss": 0.023658710718154907,
	"step": 1630
	},
	{
	"epoch": 0.9835082458770614,
	"grad_norm": 1.0188106173142704,
	"learning_rate": 8.50518901212924e-06,
	"loss": 0.02596780061721802,
	"step": 1640
	},
	{
	"epoch": 0.9895052473763118,
	"grad_norm": 1.9057144841790705,
	"learning_rate": 8.480227765350752e-06,
	"loss": 0.02274143695831299,
	"step": 1650
	},
	{
	"epoch": 0.9955022488755623,
	"grad_norm": 0.9293497249739378,
	"learning_rate": 8.45509712311583e-06,
	"loss": 0.021545737981796265,
	"step": 1660
	},
	{
	"epoch": 1.00119940029985,
	"grad_norm": 1.2211162313772652,
	"learning_rate": 8.42979830862505e-06,
	"loss": 0.021124204993247984,
	"step": 1670
	},
	{
	"epoch": 1.0071964017991004,
	"grad_norm": 0.8090076074720539,
	"learning_rate": 8.404332553264548e-06,
	"loss": 0.021261566877365114,
	"step": 1680
	},
	{
	"epoch": 1.0131934032983507,
	"grad_norm": 1.1450259085799877,
	"learning_rate": 8.378701096546087e-06,
	"loss": 0.02077616602182388,
	"step": 1690
	},
	{
	"epoch": 1.0191904047976013,
	"grad_norm": 1.3279769540615485,
	"learning_rate": 8.352905186046724e-06,
	"loss": 0.02154460847377777,
	"step": 1700
	},
	{
	"epoch": 1.0251874062968516,
	"grad_norm": 0.9630340999481978,
	"learning_rate": 8.326946077348087e-06,
	"loss": 0.021153876185417177,
	"step": 1710
	},
	{
	"epoch": 1.0311844077961019,
	"grad_norm": 1.3443920424049434,
	"learning_rate": 8.300825033975252e-06,
	"loss": 0.02120317369699478,
	"step": 1720
	},
	{
	"epoch": 1.0371814092953524,
	"grad_norm": 1.0255536783772927,
	"learning_rate": 8.274543327335257e-06,
	"loss": 0.01988975703716278,
	"step": 1730
	},
	{
	"epoch": 1.0431784107946027,
	"grad_norm": 1.3399217891813955,
	"learning_rate": 8.248102236655204e-06,
	"loss": 0.016486433148384095,
	"step": 1740
	},
	{
	"epoch": 1.049175412293853,
	"grad_norm": 1.128250702568414,
	"learning_rate": 8.221503048920009e-06,
	"loss": 0.01581169068813324,
	"step": 1750
	},
	{
	"epoch": 1.0551724137931036,
	"grad_norm": 1.4569536370248686,
	"learning_rate": 8.194747058809746e-06,
	"loss": 0.01653239130973816,
	"step": 1760
	},
	{
	"epoch": 1.0611694152923539,
	"grad_norm": 0.9893045188976791,
	"learning_rate": 8.167835568636639e-06,
	"loss": 0.014852160215377807,
	"step": 1770
	},
	{
	"epoch": 1.0671664167916042,
	"grad_norm": 1.0715746794458452,
	"learning_rate": 8.14076988828167e-06,
	"loss": 0.01751582622528076,
	"step": 1780
	},
	{
	"epoch": 1.0731634182908545,
	"grad_norm": 0.8752182792263151,
	"learning_rate": 8.11355133513082e-06,
	"loss": 0.01709965318441391,
	"step": 1790
	},
	{
	"epoch": 1.079160419790105,
	"grad_norm": 1.1573812050552144,
	"learning_rate": 8.086181234010955e-06,
	"loss": 0.013780906796455383,
	"step": 1800
	},
	{
	"epoch": 1.0851574212893553,
	"grad_norm": 1.5186639244309021,
	"learning_rate": 8.058660917125332e-06,
	"loss": 0.018228721618652344,
	"step": 1810
	},
	{
	"epoch": 1.0911544227886056,
	"grad_norm": 1.2989495845745542,
	"learning_rate": 8.030991723988764e-06,
	"loss": 0.018793556094169616,
	"step": 1820
	},
	{
	"epoch": 1.0971514242878562,
	"grad_norm": 0.8707487500425773,
	"learning_rate": 8.003175001362417e-06,
	"loss": 0.016458287835121155,
	"step": 1830
	},
	{
	"epoch": 1.1031484257871065,
	"grad_norm": 1.0573657388149416,
	"learning_rate": 7.975212103188256e-06,
	"loss": 0.01714187115430832,
	"step": 1840
	},
	{
	"epoch": 1.1091454272863568,
	"grad_norm": 1.008196061353201,
	"learning_rate": 7.947104390523151e-06,
	"loss": 0.013983532786369324,
	"step": 1850
	},
	{
	"epoch": 1.115142428785607,
	"grad_norm": 1.0761172458741406,
	"learning_rate": 7.918853231472618e-06,
	"loss": 0.015317174792289733,
	"step": 1860
	},
	{
	"epoch": 1.1211394302848576,
	"grad_norm": 1.5359048892043108,
	"learning_rate": 7.890460001124242e-06,
	"loss": 0.01641986668109894,
	"step": 1870
	},
	{
	"epoch": 1.127136431784108,
	"grad_norm": 1.0044243479013089,
	"learning_rate": 7.861926081480735e-06,
	"loss": 0.015841373801231386,
	"step": 1880
	},
	{
	"epoch": 1.1331334332833582,
	"grad_norm": 1.1240125735115383,
	"learning_rate": 7.833252861392677e-06,
	"loss": 0.01708248108625412,
	"step": 1890
	},
	{
	"epoch": 1.1391304347826088,
	"grad_norm": 1.199365626943103,
	"learning_rate": 7.804441736490906e-06,
	"loss": 0.015147665143013,
	"step": 1900
	},
	{
	"epoch": 1.145127436281859,
	"grad_norm": 0.9704067409394836,
	"learning_rate": 7.775494109118598e-06,
	"loss": 0.013679862022399902,
	"step": 1910
	},
	{
	"epoch": 1.1511244377811094,
	"grad_norm": 0.824521767963791,
	"learning_rate": 7.746411388263002e-06,
	"loss": 0.014118371903896332,
	"step": 1920
	},
	{
	"epoch": 1.1571214392803597,
	"grad_norm": 0.8351102882511994,
	"learning_rate": 7.717194989486867e-06,
	"loss": 0.016360051929950714,
	"step": 1930
	},
	{
	"epoch": 1.1631184407796102,
	"grad_norm": 0.9207165986699755,
	"learning_rate": 7.687846334859536e-06,
	"loss": 0.01681569218635559,
	"step": 1940
	},
	{
	"epoch": 1.1691154422788606,
	"grad_norm": 0.6019992155682695,
	"learning_rate": 7.658366852887723e-06,
	"loss": 0.010617994517087937,
	"step": 1950
	},
	{
	"epoch": 1.1751124437781109,
	"grad_norm": 1.1436028249646146,
	"learning_rate": 7.628757978445998e-06,
	"loss": 0.014938482642173767,
	"step": 1960
	},
	{
	"epoch": 1.1811094452773614,
	"grad_norm": 1.0338277907411968,
	"learning_rate": 7.599021152706931e-06,
	"loss": 0.012642195820808411,
	"step": 1970
	},
	{
	"epoch": 1.1871064467766117,
	"grad_norm": 0.7850332879604418,
	"learning_rate": 7.569157823070955e-06,
	"loss": 0.0130478635430336,
	"step": 1980
	},
	{
	"epoch": 1.193103448275862,
	"grad_norm": 1.0785078728531539,
	"learning_rate": 7.5391694430959095e-06,
	"loss": 0.013461634516716003,
	"step": 1990
	},
	{
	"epoch": 1.1991004497751123,
	"grad_norm": 1.0122526952666449,
	"learning_rate": 7.509057472426294e-06,
	"loss": 0.012874400615692139,
	"step": 2000
	},
	{
	"epoch": 1.2050974512743629,
	"grad_norm": 0.6498890720808242,
	"learning_rate": 7.478823376722219e-06,
	"loss": 0.011619074642658234,
	"step": 2010
	},
	{
	"epoch": 1.2110944527736132,
	"grad_norm": 0.7732153160892824,
	"learning_rate": 7.448468627588071e-06,
	"loss": 0.014993557333946228,
	"step": 2020
	},
	{
	"epoch": 1.2170914542728635,
	"grad_norm": 1.0001818936693827,
	"learning_rate": 7.4179947025008815e-06,
	"loss": 0.008736543357372284,
	"step": 2030
	},
	{
	"epoch": 1.223088455772114,
	"grad_norm": 2.39091784873849,
	"learning_rate": 7.387403084738411e-06,
	"loss": 0.012096676230430602,
	"step": 2040
	},
	{
	"epoch": 1.2290854572713643,
	"grad_norm": 2.0255336988692454,
	"learning_rate": 7.356695263306954e-06,
	"loss": 0.01338801383972168,
	"step": 2050
	},
	{
	"epoch": 1.2350824587706146,
	"grad_norm": 0.7730738024167244,
	"learning_rate": 7.32587273286887e-06,
	"loss": 0.01149948090314865,
	"step": 2060
	},
	{
	"epoch": 1.2410794602698652,
	"grad_norm": 1.2852634060083883,
	"learning_rate": 7.294936993669818e-06,
	"loss": 0.014826755225658416,
	"step": 2070
	},
	{
	"epoch": 1.2470764617691155,
	"grad_norm": 0.8930718620672152,
	"learning_rate": 7.263889551465748e-06,
	"loss": 0.010575953125953674,
	"step": 2080
	},
	{
	"epoch": 1.2530734632683658,
	"grad_norm": 0.6817450618696761,
	"learning_rate": 7.232731917449605e-06,
	"loss": 0.01065029352903366,
	"step": 2090
	},
	{
	"epoch": 1.2590704647676163,
	"grad_norm": 1.0696517480585164,
	"learning_rate": 7.201465608177772e-06,
	"loss": 0.01063266396522522,
	"step": 2100
	},
	{
	"epoch": 1.2650674662668666,
	"grad_norm": 0.5399734276458613,
	"learning_rate": 7.170092145496259e-06,
	"loss": 0.0122808575630188,
	"step": 2110
	},
	{
	"epoch": 1.271064467766117,
	"grad_norm": 0.7927460580088168,
	"learning_rate": 7.138613056466621e-06,
	"loss": 0.0102762833237648,
	"step": 2120
	},
	{
	"epoch": 1.2770614692653672,
	"grad_norm": 0.8473202461489391,
	"learning_rate": 7.107029873291638e-06,
	"loss": 0.014202943444252015,
	"step": 2130
	},
	{
	"epoch": 1.2830584707646178,
	"grad_norm": 2.4959962371544178,
	"learning_rate": 7.075344133240732e-06,
	"loss": 0.013552848994731904,
	"step": 2140
	},
	{
	"epoch": 1.289055472263868,
	"grad_norm": 1.2415279662862635,
	"learning_rate": 7.043557378575146e-06,
	"loss": 0.010717417299747466,
	"step": 2150
	},
	{
	"epoch": 1.2950524737631184,
	"grad_norm": 0.9167486552004711,
	"learning_rate": 7.011671156472878e-06,
	"loss": 0.009950728714466095,
	"step": 2160
	},
	{
	"epoch": 1.301049475262369,
	"grad_norm": 0.43711292818583564,
	"learning_rate": 6.979687018953365e-06,
	"loss": 0.008715444803237915,
	"step": 2170
	},
	{
	"epoch": 1.3070464767616192,
	"grad_norm": 0.6666518948048631,
	"learning_rate": 6.947606522801949e-06,
	"loss": 0.0103273905813694,
	"step": 2180
	},
	{
	"epoch": 1.3130434782608695,
	"grad_norm": 0.6374307846034972,
	"learning_rate": 6.915431229494102e-06,
	"loss": 0.010920720547437668,
	"step": 2190
	},
	{
	"epoch": 1.3190404797601198,
	"grad_norm": 0.58906488665692,
	"learning_rate": 6.883162705119419e-06,
	"loss": 0.010345563292503357,
	"step": 2200
	},
	{
	"epoch": 1.3250374812593704,
	"grad_norm": 0.5640045432056913,
	"learning_rate": 6.850802520305397e-06,
	"loss": 0.011169890314340592,
	"step": 2210
	},
	{
	"epoch": 1.3310344827586207,
	"grad_norm": 0.42095729336795373,
	"learning_rate": 6.8183522501409785e-06,
	"loss": 0.009798214584589005,
	"step": 2220
	},
	{
	"epoch": 1.337031484257871,
	"grad_norm": 0.7107897663552702,
	"learning_rate": 6.785813474099891e-06,
	"loss": 0.007002793997526169,
	"step": 2230
	},
	{
	"epoch": 1.3430284857571215,
	"grad_norm": 0.5599031705007916,
	"learning_rate": 6.753187775963773e-06,
	"loss": 0.007563822716474533,
	"step": 2240
	},
	{
	"epoch": 1.3490254872563718,
	"grad_norm": 0.7401332109776801,
	"learning_rate": 6.7204767437450725e-06,
	"loss": 0.007205277681350708,
	"step": 2250
	},
	{
	"epoch": 1.3550224887556221,
	"grad_norm": 0.700451010271291,
	"learning_rate": 6.687681969609768e-06,
	"loss": 0.00797586664557457,
	"step": 2260
	},
	{
	"epoch": 1.3610194902548725,
	"grad_norm": 0.8418502823581268,
	"learning_rate": 6.6548050497998585e-06,
	"loss": 0.008261550962924958,
	"step": 2270
	},
	{
	"epoch": 1.367016491754123,
	"grad_norm": 1.2083093708829893,
	"learning_rate": 6.621847584555677e-06,
	"loss": 0.008287114650011062,
	"step": 2280
	},
	{
	"epoch": 1.3730134932533733,
	"grad_norm": 0.8770533582071517,
	"learning_rate": 6.588811178038e-06,
	"loss": 0.007405246794223786,
	"step": 2290
	},
	{
	"epoch": 1.3790104947526236,
	"grad_norm": 0.6174917688185484,
	"learning_rate": 6.555697438249959e-06,
	"loss": 0.007422140240669251,
	"step": 2300
	},
	{
	"epoch": 1.3850074962518741,
	"grad_norm": 0.7718841042770771,
	"learning_rate": 6.522507976958792e-06,
	"loss": 0.006696309894323349,
	"step": 2310
	},
	{
	"epoch": 1.3910044977511244,
	"grad_norm": 0.7165810183862907,
	"learning_rate": 6.4892444096173635e-06,
	"loss": 0.008863532543182373,
	"step": 2320
	},
	{
	"epoch": 1.3970014992503748,
	"grad_norm": 0.608115357579103,
	"learning_rate": 6.455908355285567e-06,
	"loss": 0.006208262965083122,
	"step": 2330
	},
	{
	"epoch": 1.402998500749625,
	"grad_norm": 0.5898203915404459,
	"learning_rate": 6.422501436551496e-06,
	"loss": 0.007478287816047669,
	"step": 2340
	},
	{
	"epoch": 1.4089955022488756,
	"grad_norm": 0.9184641033689095,
	"learning_rate": 6.389025279452474e-06,
	"loss": 0.007693466544151306,
	"step": 2350
	},
	{
	"epoch": 1.414992503748126,
	"grad_norm": 0.645536657621612,
	"learning_rate": 6.355481513395914e-06,
	"loss": 0.008335211127996445,
	"step": 2360
	},
	{
	"epoch": 1.4209895052473764,
	"grad_norm": 0.36536738888073794,
	"learning_rate": 6.32187177108e-06,
	"loss": 0.007523158192634582,
	"step": 2370
	},
	{
	"epoch": 1.4269865067466267,
	"grad_norm": 0.7202240619815384,
	"learning_rate": 6.288197688414228e-06,
	"loss": 0.004904118180274963,
	"step": 2380
	},
	{
	"epoch": 1.432983508245877,
	"grad_norm": 0.7067580441776019,
	"learning_rate": 6.254460904439771e-06,
	"loss": 0.006895187497138977,
	"step": 2390
	},
	{
	"epoch": 1.4389805097451274,
	"grad_norm": 0.9651742768904955,
	"learning_rate": 6.220663061249706e-06,
	"loss": 0.007528834044933319,
	"step": 2400
	},
	{
	"epoch": 1.4449775112443777,
	"grad_norm": 0.5137747759340862,
	"learning_rate": 6.186805803909086e-06,
	"loss": 0.004745793715119362,
	"step": 2410
	},
	{
	"epoch": 1.4509745127436282,
	"grad_norm": 2.895489979440598,
	"learning_rate": 6.15289078037487e-06,
	"loss": 0.0070612005889415744,
	"step": 2420
	},
	{
	"epoch": 1.4569715142428785,
	"grad_norm": 0.7674191309243418,
	"learning_rate": 6.1189196414157055e-06,
	"loss": 0.006247187405824662,
	"step": 2430
	},
	{
	"epoch": 1.462968515742129,
	"grad_norm": 0.4538327005574462,
	"learning_rate": 6.084894040531591e-06,
	"loss": 0.004676883667707443,
	"step": 2440
	},
	{
	"epoch": 1.4689655172413794,
	"grad_norm": 0.6961822175861594,
	"learning_rate": 6.050815633873379e-06,
	"loss": 0.005463165044784546,
	"step": 2450
	},
	{
	"epoch": 1.4749625187406297,
	"grad_norm": 0.3818864200030401,
	"learning_rate": 6.016686080162177e-06,
	"loss": 0.005743791162967682,
	"step": 2460
	},
	{
	"epoch": 1.48095952023988,
	"grad_norm": 0.6218800446064701,
	"learning_rate": 5.9825070406086084e-06,
	"loss": 0.004535780847072601,
	"step": 2470
	},
	{
	"epoch": 1.4869565217391305,
	"grad_norm": 0.4562825432922537,
	"learning_rate": 5.948280178831949e-06,
	"loss": 0.005911724269390106,
	"step": 2480
	},
	{
	"epoch": 1.4929535232383808,
	"grad_norm": 0.2729233162646473,
	"learning_rate": 5.914007160779164e-06,
	"loss": 0.004931102693080902,
	"step": 2490
	},
	{
	"epoch": 1.4989505247376311,
	"grad_norm": 0.663195688960589,
	"learning_rate": 5.879689654643808e-06,
	"loss": 0.00664483979344368,
	"step": 2500
	},
	{
	"epoch": 1.5049475262368817,
	"grad_norm": 1.273092249357571,
	"learning_rate": 5.845329330784839e-06,
	"loss": 0.005352006107568741,
	"step": 2510
	},
	{
	"epoch": 1.510944527736132,
	"grad_norm": 0.510724703153146,
	"learning_rate": 5.810927861645307e-06,
	"loss": 0.005670171231031418,
	"step": 2520
	},
	{
	"epoch": 1.5169415292353823,
	"grad_norm": 0.45475299518788104,
	"learning_rate": 5.776486921670955e-06,
	"loss": 0.0050708547234535216,
	"step": 2530
	},
	{
	"epoch": 1.5229385307346326,
	"grad_norm": 0.489565135836941,
	"learning_rate": 5.742008187228717e-06,
	"loss": 0.004969967901706696,
	"step": 2540
	},
	{
	"epoch": 1.528935532233883,
	"grad_norm": 0.9687894756788887,
	"learning_rate": 5.707493336525123e-06,
	"loss": 0.00398874580860138,
	"step": 2550
	},
	{
	"epoch": 1.5349325337331334,
	"grad_norm": 1.6205387620945921,
	"learning_rate": 5.672944049524611e-06,
	"loss": 0.006407731026411056,
	"step": 2560
	},
	{
	"epoch": 1.540929535232384,
	"grad_norm": 0.3469102666653691,
	"learning_rate": 5.638362007867763e-06,
	"loss": 0.0029919713735580443,
	"step": 2570
	},
	{
	"epoch": 1.5469265367316343,
	"grad_norm": 0.3314134369484118,
	"learning_rate": 5.60374889478945e-06,
	"loss": 0.0055334590375423435,
	"step": 2580
	},
	{
	"epoch": 1.5529235382308846,
	"grad_norm": 0.9200546719629725,
	"learning_rate": 5.5691063950369e-06,
	"loss": 0.0060048744082450865,
	"step": 2590
	},
	{
	"epoch": 1.558920539730135,
	"grad_norm": 0.7560568067727951,
	"learning_rate": 5.534436194787705e-06,
	"loss": 0.004378331825137138,
	"step": 2600
	},
	{
	"epoch": 1.5649175412293852,
	"grad_norm": 0.47196367316768917,
	"learning_rate": 5.4997399815677324e-06,
	"loss": 0.00645904615521431,
	"step": 2610
	},
	{
	"epoch": 1.5709145427286357,
	"grad_norm": 0.6978185509730026,
	"learning_rate": 5.465019444169004e-06,
	"loss": 0.004382466897368431,
	"step": 2620
	},
	{
	"epoch": 1.576911544227886,
	"grad_norm": 0.44618001012679287,
	"learning_rate": 5.430276272567485e-06,
	"loss": 0.003658021241426468,
	"step": 2630
	},
	{
	"epoch": 1.5829085457271366,
	"grad_norm": 0.34429053500666396,
	"learning_rate": 5.395512157840832e-06,
	"loss": 0.004747949168086052,
	"step": 2640
	},
	{
	"epoch": 1.5889055472263869,
	"grad_norm": 0.2857461519537864,
	"learning_rate": 5.360728792086079e-06,
	"loss": 0.00440816655755043,
	"step": 2650
	},
	{
	"epoch": 1.5949025487256372,
	"grad_norm": 0.37428155680334885,
	"learning_rate": 5.325927868337281e-06,
	"loss": 0.002999194897711277,
	"step": 2660
	},
	{
	"epoch": 1.6008995502248875,
	"grad_norm": 0.5074559569957023,
	"learning_rate": 5.291111080483099e-06,
	"loss": 0.0034498415887355806,
	"step": 2670
	},
	{
	"epoch": 1.6068965517241378,
	"grad_norm": 0.4223823026063673,
	"learning_rate": 5.256280123184365e-06,
	"loss": 0.0036816075444221497,
	"step": 2680
	},
	{
	"epoch": 1.6128935532233883,
	"grad_norm": 0.6107607942449382,
	"learning_rate": 5.221436691791587e-06,
	"loss": 0.0037002220749855043,
	"step": 2690
	},
	{
	"epoch": 1.6188905547226387,
	"grad_norm": 0.43796288903111663,
	"learning_rate": 5.186582482262431e-06,
	"loss": 0.0032877713441848756,
	"step": 2700
	},
	{
	"epoch": 1.6248875562218892,
	"grad_norm": 0.7840144540931624,
	"learning_rate": 5.151719191079177e-06,
	"loss": 0.002722950838506222,
	"step": 2710
	},
	{
	"epoch": 1.6308845577211395,
	"grad_norm": 0.4213057546810943,
	"learning_rate": 5.116848515166144e-06,
	"loss": 0.002763654664158821,
	"step": 2720
	},
	{
	"epoch": 1.6368815592203898,
	"grad_norm": 0.22883674717133984,
	"learning_rate": 5.081972151807088e-06,
	"loss": 0.003856484591960907,
	"step": 2730
	},
	{
	"epoch": 1.6428785607196401,
	"grad_norm": 0.621830198282068,
	"learning_rate": 5.047091798562596e-06,
	"loss": 0.0027383215725421905,
	"step": 2740
	},
	{
	"epoch": 1.6488755622188904,
	"grad_norm": 0.25336521535658657,
	"learning_rate": 5.012209153187461e-06,
	"loss": 0.002146703563630581,
	"step": 2750
	},
	{
	"epoch": 1.654872563718141,
	"grad_norm": 0.7189360196089619,
	"learning_rate": 4.9773259135480355e-06,
	"loss": 0.004994399473071098,
	"step": 2760
	},
	{
	"epoch": 1.6608695652173913,
	"grad_norm": 0.3706751505337401,
	"learning_rate": 4.942443777539604e-06,
	"loss": 0.0017320241779088974,
	"step": 2770
	},
	{
	"epoch": 1.6668665667166418,
	"grad_norm": 0.06734040987909493,
	"learning_rate": 4.907564443003729e-06,
	"loss": 0.0027215242385864257,
	"step": 2780
	},
	{
	"epoch": 1.672863568215892,
	"grad_norm": 0.04409004607410812,
	"learning_rate": 4.872689607645616e-06,
	"loss": 0.0019680140540003777,
	"step": 2790
	},
	{
	"epoch": 1.6788605697151424,
	"grad_norm": 0.4285146942052719,
	"learning_rate": 4.837820968951478e-06,
	"loss": 0.0018752720206975937,
	"step": 2800
	},
	{
	"epoch": 1.6848575712143927,
	"grad_norm": 0.3666179678288489,
	"learning_rate": 4.802960224105917e-06,
	"loss": 0.002767620049417019,
	"step": 2810
	},
	{
	"epoch": 1.690854572713643,
	"grad_norm": 0.8745689296160556,
	"learning_rate": 4.7681090699093076e-06,
	"loss": 0.002765590138733387,
	"step": 2820
	},
	{
	"epoch": 1.6968515742128936,
	"grad_norm": 0.22533730796075566,
	"learning_rate": 4.7332692026952175e-06,
	"loss": 0.0023252198472619055,
	"step": 2830
	},
	{
	"epoch": 1.7028485757121439,
	"grad_norm": 0.45891305175587127,
	"learning_rate": 4.698442318247831e-06,
	"loss": 0.002209530770778656,
	"step": 2840
	},
	{
	"epoch": 1.7088455772113944,
	"grad_norm": 0.48414281947597787,
	"learning_rate": 4.6636301117194175e-06,
	"loss": 0.0023685457184910775,
	"step": 2850
	},
	{
	"epoch": 1.7148425787106447,
	"grad_norm": 0.1978346898995595,
	"learning_rate": 4.628834277547812e-06,
	"loss": 0.0019773855805397035,
	"step": 2860
	},
	{
	"epoch": 1.720839580209895,
	"grad_norm": 0.24228889354937305,
	"learning_rate": 4.594056509373955e-06,
	"loss": 0.0033055908977985384,
	"step": 2870
	},
	{
	"epoch": 1.7268365817091453,
	"grad_norm": 0.5417932963556391,
	"learning_rate": 4.559298499959444e-06,
	"loss": 0.0015053735114634037,
	"step": 2880
	},
	{
	"epoch": 1.7328335832083956,
	"grad_norm": 0.05595149163938091,
	"learning_rate": 4.524561941104144e-06,
	"loss": 0.0015690337866544723,
	"step": 2890
	},
	{
	"epoch": 1.7388305847076462,
	"grad_norm": 0.4870946742735329,
	"learning_rate": 4.489848523563844e-06,
	"loss": 0.0022526679560542105,
	"step": 2900
	},
	{
	"epoch": 1.7448275862068967,
	"grad_norm": 0.17817703608292496,
	"learning_rate": 4.455159936967964e-06,
	"loss": 0.002107449993491173,
	"step": 2910
	},
	{
	"epoch": 1.750824587706147,
	"grad_norm": 0.23549642305929913,
	"learning_rate": 4.420497869737304e-06,
	"loss": 0.0023304332047700883,
	"step": 2920
	},
	{
	"epoch": 1.7568215892053973,
	"grad_norm": 0.1542133721100359,
	"learning_rate": 4.3858640090018745e-06,
	"loss": 0.0026503754779696465,
	"step": 2930
	},
	{
	"epoch": 1.7628185907046476,
	"grad_norm": 0.2811028280533529,
	"learning_rate": 4.351260040518768e-06,
	"loss": 0.0013631435111165046,
	"step": 2940
	},
	{
	"epoch": 1.768815592203898,
	"grad_norm": 0.2867028569805445,
	"learning_rate": 4.3166876485901175e-06,
	"loss": 0.0009712180122733116,
	"step": 2950
	},
	{
	"epoch": 1.7748125937031485,
	"grad_norm": 0.3967021776922508,
	"learning_rate": 4.2821485159811004e-06,
	"loss": 0.0019603420048952104,
	"step": 2960
	},
	{
	"epoch": 1.7808095952023988,
	"grad_norm": 0.35364964950322286,
	"learning_rate": 4.2476443238380524e-06,
	"loss": 0.0020929466933012008,
	"step": 2970
	},
	{
	"epoch": 1.7868065967016493,
	"grad_norm": 0.035008758423929594,
	"learning_rate": 4.213176751606618e-06,
	"loss": 0.002725471183657646,
	"step": 2980
	},
	{
	"epoch": 1.7928035982008996,
	"grad_norm": 0.17863770189818481,
	"learning_rate": 4.178747476950022e-06,
	"loss": 0.002548707090318203,
	"step": 2990
	},
	{
	"epoch": 1.79880059970015,
	"grad_norm": 0.3789914495059916,
	"learning_rate": 4.144358175667401e-06,
	"loss": 0.0024276548996567727,
	"step": 3000
	},
	{
	"epoch": 1.8047976011994002,
	"grad_norm": 0.44540720740600165,
	"learning_rate": 4.11001052161225e-06,
	"loss": 0.001635666936635971,
	"step": 3010
	},
	{
	"epoch": 1.8107946026986506,
	"grad_norm": 0.12240730350723628,
	"learning_rate": 4.0757061866109284e-06,
	"loss": 0.002035790868103504,
	"step": 3020
	},
	{
	"epoch": 1.816791604197901,
	"grad_norm": 0.7269837454446167,
	"learning_rate": 4.041446840381309e-06,
	"loss": 0.0039451226592063906,
	"step": 3030
	},
	{
	"epoch": 1.8227886056971514,
	"grad_norm": 0.1426833292374438,
	"learning_rate": 4.007234150451488e-06,
	"loss": 0.0018614795058965682,
	"step": 3040
	},
	{
	"epoch": 1.828785607196402,
	"grad_norm": 0.44176135454573934,
	"learning_rate": 3.973069782078634e-06,
	"loss": 0.002132844366133213,
	"step": 3050
	},
	{
	"epoch": 1.8347826086956522,
	"grad_norm": 0.08366920411444181,
	"learning_rate": 3.938955398167924e-06,
	"loss": 0.0011213604360818864,
	"step": 3060
	},
	{
	"epoch": 1.8407796101949025,
	"grad_norm": 0.26598596707464867,
	"learning_rate": 3.904892659191612e-06,
	"loss": 0.001650303043425083,
	"step": 3070
	},
	{
	"epoch": 1.8467766116941529,
	"grad_norm": 0.2848301743773482,
	"learning_rate": 3.870883223108199e-06,
	"loss": 0.001640956848859787,
	"step": 3080
	},
	{
	"epoch": 1.8527736131934032,
	"grad_norm": 0.08077929425531781,
	"learning_rate": 3.836928745281742e-06,
	"loss": 0.0013208642601966857,
	"step": 3090
	},
	{
	"epoch": 1.8587706146926537,
	"grad_norm": 0.5525545653612796,
	"learning_rate": 3.803030878401274e-06,
	"loss": 0.001428498327732086,
	"step": 3100
	},
	{
	"epoch": 1.864767616191904,
	"grad_norm": 0.038688596098177624,
	"learning_rate": 3.7691912724003725e-06,
	"loss": 0.0014218797907233239,
	"step": 3110
	},
	{
	"epoch": 1.8707646176911545,
	"grad_norm": 0.4193707323985725,
	"learning_rate": 3.735411574376837e-06,
	"loss": 0.001262864749878645,
	"step": 3120
	},
	{
	"epoch": 1.8767616191904049,
	"grad_norm": 0.03212737742892218,
	"learning_rate": 3.7016934285125307e-06,
	"loss": 0.0009056521579623222,
	"step": 3130
	},
	{
	"epoch": 1.8827586206896552,
	"grad_norm": 0.2437917159487667,
	"learning_rate": 3.668038475993341e-06,
	"loss": 0.0011610159650444984,
	"step": 3140
	},
	{
	"epoch": 1.8887556221889055,
	"grad_norm": 0.3043686929497565,
	"learning_rate": 3.6344483549293143e-06,
	"loss": 0.0010252805426716805,
	"step": 3150
	},
	{
	"epoch": 1.8947526236881558,
	"grad_norm": 0.3139294697646169,
	"learning_rate": 3.6009247002748994e-06,
	"loss": 0.0012214765883982182,
	"step": 3160
	},
	{
	"epoch": 1.9007496251874063,
	"grad_norm": 0.07806834662335747,
	"learning_rate": 3.5674691437493926e-06,
	"loss": 0.0011475411243736743,
	"step": 3170
	},
	{
	"epoch": 1.9067466266866566,
	"grad_norm": 0.20441375927481553,
	"learning_rate": 3.534083313757495e-06,
	"loss": 0.0018702285364270211,
	"step": 3180
	},
	{
	"epoch": 1.9127436281859072,
	"grad_norm": 0.18911147372645712,
	"learning_rate": 3.5007688353100676e-06,
	"loss": 0.0015082253143191338,
	"step": 3190
	},
	{
	"epoch": 1.9187406296851575,
	"grad_norm": 0.16512601719719933,
	"learning_rate": 3.4675273299450264e-06,
	"loss": 0.0014218294993042946,
	"step": 3200
	},
	{
	"epoch": 1.9247376311844078,
	"grad_norm": 0.23359872392290212,
	"learning_rate": 3.434360415648423e-06,
	"loss": 0.0005122062750160695,
	"step": 3210
	},
	{
	"epoch": 1.930734632683658,
	"grad_norm": 0.1804751355783482,
	"learning_rate": 3.401269706775684e-06,
	"loss": 0.0009737418033182621,
	"step": 3220
	},
	{
	"epoch": 1.9367316341829084,
	"grad_norm": 0.23103002678467652,
	"learning_rate": 3.368256813973042e-06,
	"loss": 0.0008700978010892868,
	"step": 3230
	},
	{
	"epoch": 1.942728635682159,
	"grad_norm": 0.08348534829047724,
	"learning_rate": 3.3353233440991317e-06,
	"loss": 0.00118993716314435,
	"step": 3240
	},
	{
	"epoch": 1.9487256371814095,
	"grad_norm": 0.2803421612620039,
	"learning_rate": 3.3024709001467887e-06,
	"loss": 0.0006764805875718594,
	"step": 3250
	},
	{
	"epoch": 1.9547226386806598,
	"grad_norm": 0.01956912572182822,
	"learning_rate": 3.2697010811650133e-06,
	"loss": 0.0011283885687589645,
	"step": 3260
	},
	{
	"epoch": 1.96071964017991,
	"grad_norm": 0.23427508234616548,
	"learning_rate": 3.2370154821811496e-06,
	"loss": 0.001153472438454628,
	"step": 3270
	},
	{
	"epoch": 1.9667166416791604,
	"grad_norm": 0.07514845529294718,
	"learning_rate": 3.204415694123241e-06,
	"loss": 0.0005227739922702313,
	"step": 3280
	},
	{
	"epoch": 1.9727136431784107,
	"grad_norm": 0.026791782879728925,
	"learning_rate": 3.171903303742605e-06,
	"loss": 0.0004544252529740334,
	"step": 3290
	},
	{
	"epoch": 1.9787106446776612,
	"grad_norm": 0.0298556147979719,
	"learning_rate": 3.139479893536585e-06,
	"loss": 0.0003553445916622877,
	"step": 3300
	},
	{
	"epoch": 1.9847076461769115,
	"grad_norm": 0.039801325447289995,
	"learning_rate": 3.1071470416715414e-06,
	"loss": 0.000501357764005661,
	"step": 3310
	},
	{
	"epoch": 1.990704647676162,
	"grad_norm": 0.09059844763393914,
	"learning_rate": 3.074906321906021e-06,
	"loss": 0.00046332948841154575,
	"step": 3320
	},
	{
	"epoch": 1.9967016491754124,
	"grad_norm": 0.013699171615511596,
	"learning_rate": 3.0427593035141657e-06,
	"loss": 0.0006869334261864424,
	"step": 3330
	},
	{
	"epoch": 2.0023988005997,
	"grad_norm": 0.09716401931287477,
	"learning_rate": 3.0107075512093296e-06,
	"loss": 0.00031508624088019135,
	"step": 3340
	},
	{
	"epoch": 2.0083958020989505,
	"grad_norm": 0.017039492412108784,
	"learning_rate": 2.978752625067913e-06,
	"loss": 0.0003382320981472731,
	"step": 3350
	},
	{
	"epoch": 2.014392803598201,
	"grad_norm": 0.012376144249325193,
	"learning_rate": 2.9468960804534365e-06,
	"loss": 0.00025437292642891405,
	"step": 3360
	},
	{
	"epoch": 2.020389805097451,
	"grad_norm": 0.07583875647508195,
	"learning_rate": 2.915139467940822e-06,
	"loss": 0.0003451155265793204,
	"step": 3370
	},
	{
	"epoch": 2.0263868065967015,
	"grad_norm": 0.14343335136034452,
	"learning_rate": 2.8834843332409417e-06,
	"loss": 0.0005302869714796544,
	"step": 3380
	},
	{
	"epoch": 2.032383808095952,
	"grad_norm": 0.2399076973686298,
	"learning_rate": 2.8519322171253605e-06,
	"loss": 0.000686727836728096,
	"step": 3390
	},
	{
	"epoch": 2.0383808095952025,
	"grad_norm": 0.15913220404560705,
	"learning_rate": 2.820484655351354e-06,
	"loss": 0.00023890535812824963,
	"step": 3400
	},
	{
	"epoch": 2.044377811094453,
	"grad_norm": 0.025144323202061798,
	"learning_rate": 2.7891431785871525e-06,
	"loss": 0.00021391147747635842,
	"step": 3410
	},
	{
	"epoch": 2.050374812593703,
	"grad_norm": 0.32060550751031736,
	"learning_rate": 2.757909312337445e-06,
	"loss": 0.0003260975470766425,
	"step": 3420
	},
	{
	"epoch": 2.0563718140929534,
	"grad_norm": 0.1466529023421494,
	"learning_rate": 2.7267845768691137e-06,
	"loss": 0.0003258155658841133,
	"step": 3430
	},
	{
	"epoch": 2.0623688155922038,
	"grad_norm": 0.298118373164389,
	"learning_rate": 2.6957704871372548e-06,
	"loss": 0.0008383535780012608,
	"step": 3440
	},
	{
	"epoch": 2.068365817091454,
	"grad_norm": 0.10045121267206142,
	"learning_rate": 2.66486855271142e-06,
	"loss": 0.0004698923323303461,
	"step": 3450
	},
	{
	"epoch": 2.074362818590705,
	"grad_norm": 0.11092214594352533,
	"learning_rate": 2.63408027770216e-06,
	"loss": 0.0003082803916186094,
	"step": 3460
	},
	{
	"epoch": 2.080359820089955,
	"grad_norm": 0.059132774010371404,
	"learning_rate": 2.6034071606877997e-06,
	"loss": 0.0006355640012770891,
	"step": 3470
	},
	{
	"epoch": 2.0863568215892054,
	"grad_norm": 0.005435618647700205,
	"learning_rate": 2.5728506946415026e-06,
	"loss": 0.00021879554260522126,
	"step": 3480
	},
	{
	"epoch": 2.0923538230884557,
	"grad_norm": 0.4053518984017094,
	"learning_rate": 2.5424123668585985e-06,
	"loss": 0.0002633935306221247,
	"step": 3490
	},
	{
	"epoch": 2.098350824587706,
	"grad_norm": 0.009189909035890438,
	"learning_rate": 2.512093658884203e-06,
	"loss": 0.000517276581376791,
	"step": 3500
	},
	{
	"epoch": 2.1043478260869564,
	"grad_norm": 0.205651386091802,
	"learning_rate": 2.4818960464410866e-06,
	"loss": 0.00023343656212091446,
	"step": 3510
	},
	{
	"epoch": 2.110344827586207,
	"grad_norm": 0.007022544473411326,
	"learning_rate": 2.451820999357864e-06,
	"loss": 0.0001501690363511443,
	"step": 3520
	},
	{
	"epoch": 2.1163418290854574,
	"grad_norm": 0.14507808048119653,
	"learning_rate": 2.4218699814974374e-06,
	"loss": 0.0001178767764940858,
	"step": 3530
	},
	{
	"epoch": 2.1223388305847077,
	"grad_norm": 0.0556752492204199,
	"learning_rate": 2.392044450685758e-06,
	"loss": 0.00010345304617658258,
	"step": 3540
	},
	{
	"epoch": 2.128335832083958,
	"grad_norm": 0.012365726264308603,
	"learning_rate": 2.3623458586408553e-06,
	"loss": 0.0002090776339173317,
	"step": 3550
	},
	{
	"epoch": 2.1343328335832084,
	"grad_norm": 0.21123238049972543,
	"learning_rate": 2.33277565090219e-06,
	"loss": 0.00031952233985066414,
	"step": 3560
	},
	{
	"epoch": 2.1403298350824587,
	"grad_norm": 0.0038578319850097256,
	"learning_rate": 2.3033352667602826e-06,
	"loss": 0.00011046943254768848,
	"step": 3570
	},
	{
	"epoch": 2.146326836581709,
	"grad_norm": 0.006373930843717785,
	"learning_rate": 2.2740261391866634e-06,
	"loss": 0.0001594177447259426,
	"step": 3580
	},
	{
	"epoch": 2.1523238380809597,
	"grad_norm": 0.005753685721388461,
	"learning_rate": 2.2448496947641234e-06,
	"loss": 0.00010964295361191035,
	"step": 3590
	},
	{
	"epoch": 2.15832083958021,
	"grad_norm": 0.005854636183388129,
	"learning_rate": 2.2158073536172832e-06,
	"loss": 0.0006930826231837273,
	"step": 3600
	},
	{
	"epoch": 2.1643178410794603,
	"grad_norm": 0.02067364688180299,
	"learning_rate": 2.1869005293434552e-06,
	"loss": 0.00021630397532135248,
	"step": 3610
	},
	{
	"epoch": 2.1703148425787107,
	"grad_norm": 0.22061502564579413,
	"learning_rate": 2.1581306289438593e-06,
	"loss": 0.00018058742862194776,
	"step": 3620
	},
	{
	"epoch": 2.176311844077961,
	"grad_norm": 0.0377965905422392,
	"learning_rate": 2.1294990527551172e-06,
	"loss": 0.0001897034584544599,
	"step": 3630
	},
	{
	"epoch": 2.1823088455772113,
	"grad_norm": 0.29352186067791247,
	"learning_rate": 2.1010071943811135e-06,
	"loss": 0.00020364897791296243,
	"step": 3640
	},
	{
	"epoch": 2.1883058470764616,
	"grad_norm": 0.011428761257996708,
	"learning_rate": 2.0726564406251455e-06,
	"loss": 0.00018874803790822626,
	"step": 3650
	},
	{
	"epoch": 2.1943028485757123,
	"grad_norm": 0.11270289327200993,
	"learning_rate": 2.0444481714224384e-06,
	"loss": 0.0007160373963415623,
	"step": 3660
	},
	{
	"epoch": 2.2002998500749626,
	"grad_norm": 0.0059522383885739485,
	"learning_rate": 2.016383759772966e-06,
	"loss": 0.00020291302353143693,
	"step": 3670
	},
	{
	"epoch": 2.206296851574213,
	"grad_norm": 0.3046540173531159,
	"learning_rate": 1.988464571674629e-06,
	"loss": 0.00020479457452893257,
	"step": 3680
	},
	{
	"epoch": 2.2122938530734633,
	"grad_norm": 0.09186590578878841,
	"learning_rate": 1.9606919660567635e-06,
	"loss": 0.00012369760079309345,
	"step": 3690
	},
	{
	"epoch": 2.2182908545727136,
	"grad_norm": 0.3950520097384172,
	"learning_rate": 1.933067294714002e-06,
	"loss": 0.00013677177485078574,
	"step": 3700
	},
	{
	"epoch": 2.224287856071964,
	"grad_norm": 0.015427553878979436,
	"learning_rate": 1.9055919022404685e-06,
	"loss": 0.00022517237812280655,
	"step": 3710
	},
	{
	"epoch": 2.230284857571214,
	"grad_norm": 0.12333634291116818,
	"learning_rate": 1.8782671259643414e-06,
	"loss": 0.00032795739825814963,
	"step": 3720
	},
	{
	"epoch": 2.236281859070465,
	"grad_norm": 0.24242363451257964,
	"learning_rate": 1.85109429588275e-06,
	"loss": 0.00016028201207518578,
	"step": 3730
	},
	{
	"epoch": 2.2422788605697153,
	"grad_norm": 0.040635040172333214,
	"learning_rate": 1.8240747345970522e-06,
	"loss": 0.00014173965901136397,
	"step": 3740
	},
	{
	"epoch": 2.2482758620689656,
	"grad_norm": 0.003923728504721442,
	"learning_rate": 1.797209757248443e-06,
	"loss": 0.00028566645924001934,
	"step": 3750
	},
	{
	"epoch": 2.254272863568216,
	"grad_norm": 0.015804919263076193,
	"learning_rate": 1.7705006714539584e-06,
	"loss": 0.00027338461950421333,
	"step": 3760
	},
	{
	"epoch": 2.260269865067466,
	"grad_norm": 0.10222434673362414,
	"learning_rate": 1.7439487772428142e-06,
	"loss": 0.00013164160773158074,
	"step": 3770
	},
	{
	"epoch": 2.2662668665667165,
	"grad_norm": 0.26435570777017947,
	"learning_rate": 1.7175553669931366e-06,
	"loss": 0.00048770131543278693,
	"step": 3780
	},
	{
	"epoch": 2.2722638680659673,
	"grad_norm": 0.06742754490849341,
	"learning_rate": 1.6913217253690539e-06,
	"loss": 0.00036997650749981403,
	"step": 3790
	},
	{
	"epoch": 2.2782608695652176,
	"grad_norm": 0.022982698631413053,
	"learning_rate": 1.6652491292581756e-06,
	"loss": 0.00020245849154889583,
	"step": 3800
	},
	{
	"epoch": 2.284257871064468,
	"grad_norm": 0.14673651335306825,
	"learning_rate": 1.6393388477094275e-06,
	"loss": 0.0002849138109013438,
	"step": 3810
	},
	{
	"epoch": 2.290254872563718,
	"grad_norm": 0.01432613394887218,
	"learning_rate": 1.6135921418712959e-06,
	"loss": 9.818098042160272e-05,
	"step": 3820
	},
	{
	"epoch": 2.2962518740629685,
	"grad_norm": 0.20732246510693053,
	"learning_rate": 1.5880102649304312e-06,
	"loss": 0.0001456856494769454,
	"step": 3830
	},
	{
	"epoch": 2.302248875562219,
	"grad_norm": 0.003787513926404288,
	"learning_rate": 1.5625944620506633e-06,
	"loss": 0.00023098369129002095,
	"step": 3840
	},
	{
	"epoch": 2.308245877061469,
	"grad_norm": 0.0023646388935046124,
	"learning_rate": 1.5373459703123817e-06,
	"loss": 8.435071213170886e-05,
	"step": 3850
	},
	{
	"epoch": 2.3142428785607194,
	"grad_norm": 0.0054260307158288295,
	"learning_rate": 1.5122660186523347e-06,
	"loss": 0.0001499459845945239,
	"step": 3860
	},
	{
	"epoch": 2.32023988005997,
	"grad_norm": 0.03715216417785091,
	"learning_rate": 1.487355827803801e-06,
	"loss": 0.00027119163423776626,
	"step": 3870
	},
	{
	"epoch": 2.3262368815592205,
	"grad_norm": 0.003397844016640696,
	"learning_rate": 1.4626166102371808e-06,
	"loss": 0.0002886164700612426,
	"step": 3880
	},
	{
	"epoch": 2.332233883058471,
	"grad_norm": 0.008233810475259094,
	"learning_rate": 1.4380495701009744e-06,
	"loss": 0.00012392442440614104,
	"step": 3890
	},
	{
	"epoch": 2.338230884557721,
	"grad_norm": 0.24272950105625712,
	"learning_rate": 1.4136559031631802e-06,
	"loss": 0.0001440195133909583,
	"step": 3900
	},
	{
	"epoch": 2.3442278860569714,
	"grad_norm": 0.01997911051588864,
	"learning_rate": 1.3894367967530803e-06,
	"loss": 0.0001797779113985598,
	"step": 3910
	},
	{
	"epoch": 2.3502248875562217,
	"grad_norm": 0.00249678516483223,
	"learning_rate": 1.3653934297034616e-06,
	"loss": 9.720514644868672e-05,
	"step": 3920
	},
	{
	"epoch": 2.3562218890554725,
	"grad_norm": 0.011755295270162608,
	"learning_rate": 1.3415269722932257e-06,
	"loss": 0.0002720349933952093,
	"step": 3930
	},
	{
	"epoch": 2.362218890554723,
	"grad_norm": 0.312112503531549,
	"learning_rate": 1.3178385861904375e-06,
	"loss": 0.000972554087638855,
	"step": 3940
	},
	{
	"epoch": 2.368215892053973,
	"grad_norm": 0.006258548131869805,
	"learning_rate": 1.2943294243957732e-06,
	"loss": 0.00010051319841295481,
	"step": 3950
	},
	{
	"epoch": 2.3742128935532234,
	"grad_norm": 0.002828028774493926,
	"learning_rate": 1.2710006311864104e-06,
	"loss": 7.33192078769207e-05,
	"step": 3960
	},
	{
	"epoch": 2.3802098950524737,
	"grad_norm": 0.004227799598896886,
	"learning_rate": 1.2478533420603207e-06,
	"loss": 0.0001137706683948636,
	"step": 3970
	},
	{
	"epoch": 2.386206896551724,
	"grad_norm": 0.24349712163781712,
	"learning_rate": 1.224888683681007e-06,
	"loss": 0.0003358069574460387,
	"step": 3980
	},
	{
	"epoch": 2.3922038980509743,
	"grad_norm": 0.013679651601552613,
	"learning_rate": 1.2021077738226628e-06,
	"loss": 0.00012730241287499665,
	"step": 3990
	},
	{
	"epoch": 2.3982008995502246,
	"grad_norm": 0.021687984717511294,
	"learning_rate": 1.1795117213157698e-06,
	"loss": 0.0001658328459598124,
	"step": 4000
	},
	{
	"epoch": 2.4041979010494754,
	"grad_norm": 0.01580941909835264,
	"learning_rate": 1.1571016259931239e-06,
	"loss": 0.00010655056685209275,
	"step": 4010
	},
	{
	"epoch": 2.4101949025487257,
	"grad_norm": 0.018656185749449338,
	"learning_rate": 1.1348785786362976e-06,
	"loss": 0.0001721666194498539,
	"step": 4020
	},
	{
	"epoch": 2.416191904047976,
	"grad_norm": 0.003952035420756565,
	"learning_rate": 1.1128436609225584e-06,
	"loss": 0.0001267416402697563,
	"step": 4030
	},
	{
	"epoch": 2.4221889055472263,
	"grad_norm": 0.005613227701786692,
	"learning_rate": 1.090997945372211e-06,
	"loss": 3.215537290088832e-05,
	"step": 4040
	},
	{
	"epoch": 2.4281859070464766,
	"grad_norm": 0.029265993233271104,
	"learning_rate": 1.0693424952963948e-06,
	"loss": 9.868558263406157e-05,
	"step": 4050
	},
	{
	"epoch": 2.434182908545727,
	"grad_norm": 0.0038310557368825765,
	"learning_rate": 1.0478783647453316e-06,
	"loss": 0.00027968501672148705,
	"step": 4060
	},
	{
	"epoch": 2.4401799100449777,
	"grad_norm": 0.09503225829055226,
	"learning_rate": 1.0266065984570239e-06,
	"loss": 9.34633193537593e-05,
	"step": 4070
	},
	{
	"epoch": 2.446176911544228,
	"grad_norm": 0.2614933412939702,
	"learning_rate": 1.0055282318063919e-06,
	"loss": 0.0001761227147653699,
	"step": 4080
	},
	{
	"epoch": 2.4521739130434783,
	"grad_norm": 0.007421574273265052,
	"learning_rate": 9.846442907548942e-07,
	"loss": 0.00012275519547984005,
	"step": 4090
	},
	{
	"epoch": 2.4581709145427286,
	"grad_norm": 0.07105531851296897,
	"learning_rate": 9.639557918005743e-07,
	"loss": 0.00015743521507829427,
	"step": 4100
	},
	{
	"epoch": 2.464167916041979,
	"grad_norm": 0.09597845527238576,
	"learning_rate": 9.43463741928598e-07,
	"loss": 0.00012432982912287117,
	"step": 4110
	},
	{
	"epoch": 2.4701649175412292,
	"grad_norm": 0.13852621528751427,
	"learning_rate": 9.231691385622277e-07,
	"loss": 0.0001705039059743285,
	"step": 4120
	},
	{
	"epoch": 2.4761619190404796,
	"grad_norm": 0.0036931486698786693,
	"learning_rate": 9.030729695142853e-07,
	"loss": 7.319455035030842e-05,
	"step": 4130
	},
	{
	"epoch": 2.4821589205397303,
	"grad_norm": 0.021715735318425664,
	"learning_rate": 8.831762129390625e-07,
	"loss": 4.854604485444724e-05,
	"step": 4140
	},
	{
	"epoch": 2.4881559220389806,
	"grad_norm": 0.007900177894962055,
	"learning_rate": 8.634798372847148e-07,
	"loss": 0.00022429521195590496,
	"step": 4150
	},
	{
	"epoch": 2.494152923538231,
	"grad_norm": 0.002730312127730014,
	"learning_rate": 8.439848012461227e-07,
	"loss": 0.0002710050204768777,
	"step": 4160
	},
	{
	"epoch": 2.5001499250374812,
	"grad_norm": 0.033277520794717366,
	"learning_rate": 8.246920537182312e-07,
	"loss": 0.00016629474703222513,
	"step": 4170
	},
	{
	"epoch": 2.5061469265367315,
	"grad_norm": 0.008876454232424662,
	"learning_rate": 8.05602533749858e-07,
	"loss": 0.00020069871097803116,
	"step": 4180
	},
	{
	"epoch": 2.512143928035982,
	"grad_norm": 0.16813061123223247,
	"learning_rate": 7.867171704979932e-07,
	"loss": 9.260741644538939e-05,
	"step": 4190
	},
	{
	"epoch": 2.5181409295352326,
	"grad_norm": 0.03504277597392313,
	"learning_rate": 7.680368831825657e-07,
	"loss": 0.000269699445925653,
	"step": 4200
	},
	{
	"epoch": 2.524137931034483,
	"grad_norm": 0.2031675709631404,
	"learning_rate": 7.495625810417106e-07,
	"loss": 0.0001719924039207399,
	"step": 4210
	},
	{
	"epoch": 2.5301349325337332,
	"grad_norm": 0.0020109569870982408,
	"learning_rate": 7.312951632875053e-07,
	"loss": 7.063928060233594e-05,
	"step": 4220
	},
	{
	"epoch": 2.5361319340329835,
	"grad_norm": 0.0021684953268957864,
	"learning_rate": 7.132355190622076e-07,
	"loss": 6.589657859876752e-05,
	"step": 4230
	},
	{
	"epoch": 2.542128935532234,
	"grad_norm": 0.0053398554767928345,
	"learning_rate": 6.953845273949733e-07,
	"loss": 0.00011057076044380665,
	"step": 4240
	},
	{
	"epoch": 2.548125937031484,
	"grad_norm": 0.008301630301758598,
	"learning_rate": 6.777430571590732e-07,
	"loss": 7.171622710302472e-05,
	"step": 4250
	},
	{
	"epoch": 2.5541229385307345,
	"grad_norm": 0.0035648527725669755,
	"learning_rate": 6.603119670295998e-07,
	"loss": 0.00011826218105852604,
	"step": 4260
	},
	{
	"epoch": 2.5601199400299848,
	"grad_norm": 0.0025969028900765167,
	"learning_rate": 6.430921054416773e-07,
	"loss": 2.9090914176777004e-05,
	"step": 4270
	},
	{
	"epoch": 2.5661169415292355,
	"grad_norm": 0.002010977962404487,
	"learning_rate": 6.260843105491577e-07,
	"loss": 4.955612821504474e-05,
	"step": 4280
	},
	{
	"epoch": 2.572113943028486,
	"grad_norm": 0.0060959869084811266,
	"learning_rate": 6.092894101838315e-07,
	"loss": 7.064694073051214e-05,
	"step": 4290
	},
	{
	"epoch": 2.578110944527736,
	"grad_norm": 0.14660903752720336,
	"learning_rate": 5.927082218151293e-07,
	"loss": 0.00013926166575402022,
	"step": 4300
	},
	{
	"epoch": 2.5841079460269865,
	"grad_norm": 0.008847973030263486,
	"learning_rate": 5.763415525103372e-07,
	"loss": 0.0001977612730115652,
	"step": 4310
	},
	{
	"epoch": 2.5901049475262368,
	"grad_norm": 0.03385793129646808,
	"learning_rate": 5.601901988953073e-07,
	"loss": 0.00019506249809637667,
	"step": 4320
	},
	{
	"epoch": 2.5961019490254875,
	"grad_norm": 0.004351321891021449,
	"learning_rate": 5.442549471156922e-07,
	"loss": 0.0002707177307456732,
	"step": 4330
	},
	{
	"epoch": 2.602098950524738,
	"grad_norm": 0.20230693318728651,
	"learning_rate": 5.285365727986708e-07,
	"loss": 9.98120871372521e-05,
	"step": 4340
	},
	{
	"epoch": 2.608095952023988,
	"grad_norm": 0.0017257852666605175,
	"learning_rate": 5.130358410152008e-07,
	"loss": 0.00010577249340713025,
	"step": 4350
	},
	{
	"epoch": 2.6140929535232384,
	"grad_norm": 0.0033426231949093247,
	"learning_rate": 4.977535062427796e-07,
	"loss": 5.586037877947092e-05,
	"step": 4360
	},
	{
	"epoch": 2.6200899550224888,
	"grad_norm": 0.002760838950017781,
	"learning_rate": 4.82690312328723e-07,
	"loss": 0.00023619804996997119,
	"step": 4370
	},
	{
	"epoch": 2.626086956521739,
	"grad_norm": 0.0018970164576556166,
	"learning_rate": 4.678469924539525e-07,
	"loss": 7.213128847070039e-05,
	"step": 4380
	},
	{
	"epoch": 2.6320839580209894,
	"grad_norm": 0.0016709373881888241,
	"learning_rate": 4.532242690973182e-07,
	"loss": 0.00013169851154088975,
	"step": 4390
	},
	{
	"epoch": 2.6380809595202397,
	"grad_norm": 0.0034893522997727425,
	"learning_rate": 4.38822854000423e-07,
	"loss": 0.00010110214352607728,
	"step": 4400
	},
	{
	"epoch": 2.64407796101949,
	"grad_norm": 0.004256718046477718,
	"learning_rate": 4.246434481329892e-07,
	"loss": 8.836719789542257e-05,
	"step": 4410
	},
	{
	"epoch": 2.6500749625187408,
	"grad_norm": 0.0028947742332821485,
	"learning_rate": 4.1068674165873177e-07,
	"loss": 0.00018591651460155844,
	"step": 4420
	},
	{
	"epoch": 2.656071964017991,
	"grad_norm": 0.058279762541049246,
	"learning_rate": 3.969534139017728e-07,
	"loss": 5.3574988851323727e-05,
	"step": 4430
	},
	{
	"epoch": 2.6620689655172414,
	"grad_norm": 0.09631181116881746,
	"learning_rate": 3.8344413331356853e-07,
	"loss": 0.00022052752319723368,
	"step": 4440
	},
	{
	"epoch": 2.6680659670164917,
	"grad_norm": 0.05400733835014621,
	"learning_rate": 3.701595574403799e-07,
	"loss": 8.408076828345657e-05,
	"step": 4450
	},
	{
	"epoch": 2.674062968515742,
	"grad_norm": 0.03578584367204414,
	"learning_rate": 3.5710033289126255e-07,
	"loss": 0.00016756949480623007,
	"step": 4460
	},
	{
	"epoch": 2.6800599700149927,
	"grad_norm": 0.0022655272259947943,
	"learning_rate": 3.442670953065991e-07,
	"loss": 6.750763859599828e-05,
	"step": 4470
	},
	{
	"epoch": 2.686056971514243,
	"grad_norm": 0.0022240884102984262,
	"learning_rate": 3.3166046932715414e-07,
	"loss": 0.00012028507189825177,
	"step": 4480
	},
	{
	"epoch": 2.6920539730134934,
	"grad_norm": 0.004619422352335246,
	"learning_rate": 3.1928106856367824e-07,
	"loss": 0.0002119823591783643,
	"step": 4490
	},
	{
	"epoch": 2.6980509745127437,
	"grad_norm": 0.003643679816253472,
	"learning_rate": 3.0712949556703175e-07,
	"loss": 0.00016507470281794666,
	"step": 4500
	},
	{
	"epoch": 2.704047976011994,
	"grad_norm": 0.0018426742190434686,
	"learning_rate": 2.9520634179886544e-07,
	"loss": 0.0001338446862064302,
	"step": 4510
	},
	{
	"epoch": 2.7100449775112443,
	"grad_norm": 0.0018204132394634871,
	"learning_rate": 2.8351218760282516e-07,
	"loss": 4.272074438631535e-05,
	"step": 4520
	},
	{
	"epoch": 2.7160419790104946,
	"grad_norm": 0.06003552708347106,
	"learning_rate": 2.7204760217631074e-07,
	"loss": 0.00019897529855370522,
	"step": 4530
	},
	{
	"epoch": 2.722038980509745,
	"grad_norm": 0.0026436751524386063,
	"learning_rate": 2.608131435427641e-07,
	"loss": 6.081213359721005e-05,
	"step": 4540
	},
	{
	"epoch": 2.7280359820089957,
	"grad_norm": 0.004379395238450374,
	"learning_rate": 2.498093585245132e-07,
	"loss": 0.00012997064040973783,
	"step": 4550
	},
	{
	"epoch": 2.734032983508246,
	"grad_norm": 0.002217729171631346,
	"learning_rate": 2.3903678271615437e-07,
	"loss": 4.725663166027516e-05,
	"step": 4560
	},
	{
	"epoch": 2.7400299850074963,
	"grad_norm": 0.10558376724378146,
	"learning_rate": 2.2849594045848412e-07,
	"loss": 9.887558408081532e-05,
	"step": 4570
	},
	{
	"epoch": 2.7460269865067466,
	"grad_norm": 0.050238946364215845,
	"learning_rate": 2.181873448129751e-07,
	"loss": 0.00010824603959918022,
	"step": 4580
	},
	{
	"epoch": 2.752023988005997,
	"grad_norm": 0.0038357391962297173,
	"learning_rate": 2.0811149753680647e-07,
	"loss": 4.437556490302086e-05,
	"step": 4590
	},
	{
	"epoch": 2.758020989505247,
	"grad_norm": 0.08119805172380345,
	"learning_rate": 1.9826888905843943e-07,
	"loss": 6.780261173844337e-05,
	"step": 4600
	},
	{
	"epoch": 2.764017991004498,
	"grad_norm": 0.0031400163791590776,
	"learning_rate": 1.8865999845374794e-07,
	"loss": 8.227718062698841e-05,
	"step": 4610
	},
	{
	"epoch": 2.7700149925037483,
	"grad_norm": 0.02231553814297758,
	"learning_rate": 1.7928529342269796e-07,
	"loss": 0.00013522050576284528,
	"step": 4620
	},
	{
	"epoch": 2.7760119940029986,
	"grad_norm": 0.005688224586819576,
	"learning_rate": 1.7014523026658515e-07,
	"loss": 0.0002912352792918682,
	"step": 4630
	},
	{
	"epoch": 2.782008995502249,
	"grad_norm": 0.04274655435629151,
	"learning_rate": 1.6124025386582366e-07,
	"loss": 6.563850329257547e-05,
	"step": 4640
	},
	{
	"epoch": 2.788005997001499,
	"grad_norm": 0.023870841731819466,
	"learning_rate": 1.5257079765829296e-07,
	"loss": 0.0001261313445866108,
	"step": 4650
	},
	{
	"epoch": 2.7940029985007495,
	"grad_norm": 0.02308264385431564,
	"learning_rate": 1.441372836182392e-07,
	"loss": 5.091931670904159e-05,
	"step": 4660
	},
	{
	"epoch": 2.8,
	"grad_norm": 0.013773139345435148,
	"learning_rate": 1.3594012223573983e-07,
	"loss": 4.0650839218869804e-05,
	"step": 4670
	},
	{
	"epoch": 2.80599700149925,
	"grad_norm": 0.007417058378187365,
	"learning_rate": 1.2797971249671915e-07,
	"loss": 4.403659841045737e-05,
	"step": 4680
	},
	{
	"epoch": 2.811994002998501,
	"grad_norm": 0.004895756319397226,
	"learning_rate": 1.2025644186353103e-07,
	"loss": 6.0337805189192294e-05,
	"step": 4690
	},
	{
	"epoch": 2.817991004497751,
	"grad_norm": 0.005456175920086177,
	"learning_rate": 1.1277068625609899e-07,
	"loss": 8.372058509849012e-05,
	"step": 4700
	},
	{
	"epoch": 2.8239880059970015,
	"grad_norm": 0.0061993042337634445,
	"learning_rate": 1.0552281003361753e-07,
	"loss": 2.793670282699168e-05,
	"step": 4710
	},
	{
	"epoch": 2.829985007496252,
	"grad_norm": 0.0033776167701644565,
	"learning_rate": 9.851316597681959e-08,
	"loss": 0.00010963287204504013,
	"step": 4720
	},
	{
	"epoch": 2.835982008995502,
	"grad_norm": 0.008130663860473652,
	"learning_rate": 9.174209527080369e-08,
	"loss": 0.00027154290582984684,
	"step": 4730
	},
	{
	"epoch": 2.841979010494753,
	"grad_norm": 0.0032630844236524953,
	"learning_rate": 8.520992748842938e-08,
	"loss": 0.00012200160417705775,
	"step": 4740
	},
	{
	"epoch": 2.847976011994003,
	"grad_norm": 0.002158250428361804,
	"learning_rate": 7.891698057427232e-08,
	"loss": 6.704623228870332e-05,
	"step": 4750
	},
	{
	"epoch": 2.8539730134932535,
	"grad_norm": 0.0019393096800296016,
	"learning_rate": 7.286356082915169e-08,
	"loss": 0.0003062809584662318,
	"step": 4760
	},
	{
	"epoch": 2.859970014992504,
	"grad_norm": 0.015184585871985764,
	"learning_rate": 6.704996289521981e-08,
	"loss": 4.403400234878063e-05,
	"step": 4770
	},
	{
	"epoch": 2.865967016491754,
	"grad_norm": 0.0024209786212707686,
	"learning_rate": 6.147646974162313e-08,
	"loss": 0.0001349588856101036,
	"step": 4780
	},
	{
	"epoch": 2.8719640179910044,
	"grad_norm": 0.0697025519383071,
	"learning_rate": 5.614335265072435e-08,
	"loss": 4.611912881955504e-05,
	"step": 4790
	},
	{
	"epoch": 2.8779610194902547,
	"grad_norm": 0.001964363714663423,
	"learning_rate": 5.105087120490404e-08,
	"loss": 6.344455759972334e-05,
	"step": 4800
	},
	{
	"epoch": 2.883958020989505,
	"grad_norm": 0.1816531792762563,
	"learning_rate": 4.619927327392082e-08,
	"loss": 0.00011217539431527258,
	"step": 4810
	},
	{
	"epoch": 2.8899550224887554,
	"grad_norm": 0.004812766142295,
	"learning_rate": 4.158879500284985e-08,
	"loss": 2.2781861480325462e-05,
	"step": 4820
	},
	{
	"epoch": 2.895952023988006,
	"grad_norm": 0.008389095427513725,
	"learning_rate": 3.721966080058703e-08,
	"loss": 6.626155227422715e-05,
	"step": 4830
	},
	{
	"epoch": 2.9019490254872564,
	"grad_norm": 0.02204039757430177,
	"learning_rate": 3.309208332892777e-08,
	"loss": 3.878698626067489e-05,
	"step": 4840
	},
	{
	"epoch": 2.9079460269865067,
	"grad_norm": 0.0049266294882024195,
	"learning_rate": 2.9206263492213004e-08,
	"loss": 0.0001440607476979494,
	"step": 4850
	},
	{
	"epoch": 2.913943028485757,
	"grad_norm": 0.11578749986927613,
	"learning_rate": 2.5562390427554263e-08,
	"loss": 5.902083357796073e-05,
	"step": 4860
	},
	{
	"epoch": 2.9199400299850073,
	"grad_norm": 0.1220848501779149,
	"learning_rate": 2.2160641495625467e-08,
	"loss": 0.0001411622972227633,
	"step": 4870
	},
	{
	"epoch": 2.925937031484258,
	"grad_norm": 0.001763201178938392,
	"learning_rate": 1.9001182272030406e-08,
	"loss": 7.73983309045434e-05,
	"step": 4880
	},
	{
	"epoch": 2.9319340329835084,
	"grad_norm": 0.004818074379255404,
	"learning_rate": 1.6084166539242496e-08,
	"loss": 0.00011157422559335828,
	"step": 4890
	},
	{
	"epoch": 2.9379310344827587,
	"grad_norm": 0.014723222254443662,
	"learning_rate": 1.340973627912301e-08,
	"loss": 5.1000103121623396e-05,
	"step": 4900
	},
	{
	"epoch": 2.943928035982009,
	"grad_norm": 0.006935624408487933,
	"learning_rate": 1.0978021666005479e-08,
	"loss": 2.564325404819101e-05,
	"step": 4910
	},
	{
	"epoch": 2.9499250374812593,
	"grad_norm": 0.001954817281729771,
	"learning_rate": 8.789141060365215e-09,
	"loss": 7.659155526198446e-05,
	"step": 4920
	},
	{
	"epoch": 2.9559220389805096,
	"grad_norm": 0.0022976631988290102,
	"learning_rate": 6.843201003051692e-09,
	"loss": 2.1746587299276145e-05,
	"step": 4930
	},
	{
	"epoch": 2.96191904047976,
	"grad_norm": 0.003868082243962635,
	"learning_rate": 5.140296210110473e-09,
	"loss": 1.8990872194990516e-05,
	"step": 4940
	},
	{
	"epoch": 2.9679160419790103,
	"grad_norm": 0.07650369766178686,
	"learning_rate": 3.6805095681652357e-09,
	"loss": 5.960226990282535e-05,
	"step": 4950
	},
	{
	"epoch": 2.973913043478261,
	"grad_norm": 0.003664785410086155,
	"learning_rate": 2.463912130388768e-09,
	"loss": 4.071901203133166e-05,
	"step": 4960
	},
	{
	"epoch": 2.9799100449775113,
	"grad_norm": 0.01137579923369469,
	"learning_rate": 1.4905631130440746e-09,
	"loss": 0.0001039734110236168,
	"step": 4970
	},
	{
	"epoch": 2.9859070464767616,
	"grad_norm": 0.0207412116725464,
	"learning_rate": 7.60509892598349e-10,
	"loss": 8.716167649254203e-05,
	"step": 4980
	},
	{
	"epoch": 2.991904047976012,
	"grad_norm": 0.0860235592992905,
	"learning_rate": 2.737880034214824e-10,
	"loss": 6.557384040206671e-05,
	"step": 4990
	},
	{
	"epoch": 2.9979010494752623,
	"grad_norm": 0.002040770997191797,
	"learning_rate": 3.042113605411512e-11,
	"loss": 3.6511517828330396e-05,
	"step": 5000
	},
	{
	"epoch": 3.0,
	"step": 5004,
	"total_flos": 4.391744823418159e+18,
	"train_loss": 0.061801959001679146,
	"train_runtime": 804258.665,
	"train_samples_per_second": 0.05,
	"train_steps_per_second": 0.006
	}
	],
	"logging_steps": 10,
	"max_steps": 5004,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 3,
	"save_steps": 10000,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": true
	},
	"attributes": {}
	}
	},
	"total_flos": 4.391744823418159e+18,
	"train_batch_size": 1,
	"trial_name": null,
	"trial_params": null
	}