Instructions to use VisionXLab/FIRM-Video-8B-Qwen3VL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use VisionXLab/FIRM-Video-8B-Qwen3VL with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="VisionXLab/FIRM-Video-8B-Qwen3VL")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("VisionXLab/FIRM-Video-8B-Qwen3VL")
model = AutoModelForMultimodalLM.from_pretrained("VisionXLab/FIRM-Video-8B-Qwen3VL", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use VisionXLab/FIRM-Video-8B-Qwen3VL with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "VisionXLab/FIRM-Video-8B-Qwen3VL"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VisionXLab/FIRM-Video-8B-Qwen3VL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/VisionXLab/FIRM-Video-8B-Qwen3VL

SGLang

How to use VisionXLab/FIRM-Video-8B-Qwen3VL with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "VisionXLab/FIRM-Video-8B-Qwen3VL" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VisionXLab/FIRM-Video-8B-Qwen3VL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "VisionXLab/FIRM-Video-8B-Qwen3VL" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "VisionXLab/FIRM-Video-8B-Qwen3VL",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use VisionXLab/FIRM-Video-8B-Qwen3VL with Docker Model Runner:
```
docker model run hf.co/VisionXLab/FIRM-Video-8B-Qwen3VL
```

FIRM-Video-8B-Qwen3VL / trainer_state.json

zpy777

Upload folder using huggingface_hub

fbc5e82 verified 2 months ago

Raw History Blame Contribute Delete

103 kB

	{
	"best_global_step": null,
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 2.0,
	"eval_steps": 500,
	"global_step": 5504,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"epoch": 0.003633720930232558,
	"grad_norm": 34.2308235168457,
	"learning_rate": 1.6304347826086958e-07,
	"loss": 2.4326148986816407,
	"step": 10
	},
	{
	"epoch": 0.007267441860465116,
	"grad_norm": 32.4608268737793,
	"learning_rate": 3.442028985507247e-07,
	"loss": 2.368633842468262,
	"step": 20
	},
	{
	"epoch": 0.010901162790697675,
	"grad_norm": 29.76852035522461,
	"learning_rate": 5.253623188405797e-07,
	"loss": 2.205740547180176,
	"step": 30
	},
	{
	"epoch": 0.014534883720930232,
	"grad_norm": 12.119311332702637,
	"learning_rate": 7.065217391304348e-07,
	"loss": 1.7575836181640625,
	"step": 40
	},
	{
	"epoch": 0.018168604651162792,
	"grad_norm": 5.649290084838867,
	"learning_rate": 8.8768115942029e-07,
	"loss": 1.50073184967041,
	"step": 50
	},
	{
	"epoch": 0.02180232558139535,
	"grad_norm": 4.6185126304626465,
	"learning_rate": 1.068840579710145e-06,
	"loss": 1.3841896057128906,
	"step": 60
	},
	{
	"epoch": 0.025436046511627907,
	"grad_norm": 3.9817209243774414,
	"learning_rate": 1.25e-06,
	"loss": 1.259902000427246,
	"step": 70
	},
	{
	"epoch": 0.029069767441860465,
	"grad_norm": 3.747006416320801,
	"learning_rate": 1.4311594202898552e-06,
	"loss": 1.1269563674926757,
	"step": 80
	},
	{
	"epoch": 0.032703488372093026,
	"grad_norm": 3.4612467288970947,
	"learning_rate": 1.6123188405797101e-06,
	"loss": 1.0907072067260741,
	"step": 90
	},
	{
	"epoch": 0.036337209302325583,
	"grad_norm": 3.7003870010375977,
	"learning_rate": 1.7934782608695653e-06,
	"loss": 1.0542475700378418,
	"step": 100
	},
	{
	"epoch": 0.03997093023255814,
	"grad_norm": 3.5332090854644775,
	"learning_rate": 1.9746376811594204e-06,
	"loss": 1.0133766174316405,
	"step": 110
	},
	{
	"epoch": 0.0436046511627907,
	"grad_norm": 3.4193594455718994,
	"learning_rate": 2.1557971014492757e-06,
	"loss": 0.9582798957824707,
	"step": 120
	},
	{
	"epoch": 0.047238372093023256,
	"grad_norm": 3.7489328384399414,
	"learning_rate": 2.3369565217391307e-06,
	"loss": 0.9633728981018066,
	"step": 130
	},
	{
	"epoch": 0.050872093023255814,
	"grad_norm": 3.4787702560424805,
	"learning_rate": 2.518115942028986e-06,
	"loss": 0.958065128326416,
	"step": 140
	},
	{
	"epoch": 0.05450581395348837,
	"grad_norm": 3.477475643157959,
	"learning_rate": 2.6992753623188405e-06,
	"loss": 0.9650903701782226,
	"step": 150
	},
	{
	"epoch": 0.05813953488372093,
	"grad_norm": 3.538022041320801,
	"learning_rate": 2.880434782608696e-06,
	"loss": 0.947903060913086,
	"step": 160
	},
	{
	"epoch": 0.06177325581395349,
	"grad_norm": 3.6848044395446777,
	"learning_rate": 3.061594202898551e-06,
	"loss": 0.9341224670410156,
	"step": 170
	},
	{
	"epoch": 0.06540697674418605,
	"grad_norm": 3.290121555328369,
	"learning_rate": 3.242753623188406e-06,
	"loss": 0.9166387557983399,
	"step": 180
	},
	{
	"epoch": 0.0690406976744186,
	"grad_norm": 3.1919069290161133,
	"learning_rate": 3.4239130434782614e-06,
	"loss": 0.9012920379638671,
	"step": 190
	},
	{
	"epoch": 0.07267441860465117,
	"grad_norm": 3.583104133605957,
	"learning_rate": 3.6050724637681163e-06,
	"loss": 0.9445178031921386,
	"step": 200
	},
	{
	"epoch": 0.07630813953488372,
	"grad_norm": 3.459635019302368,
	"learning_rate": 3.7862318840579717e-06,
	"loss": 0.9214607238769531,
	"step": 210
	},
	{
	"epoch": 0.07994186046511628,
	"grad_norm": 3.3845698833465576,
	"learning_rate": 3.967391304347827e-06,
	"loss": 0.8824331283569335,
	"step": 220
	},
	{
	"epoch": 0.08357558139534883,
	"grad_norm": 3.390432119369507,
	"learning_rate": 4.148550724637682e-06,
	"loss": 0.9067767143249512,
	"step": 230
	},
	{
	"epoch": 0.0872093023255814,
	"grad_norm": 3.236952304840088,
	"learning_rate": 4.329710144927536e-06,
	"loss": 0.920140552520752,
	"step": 240
	},
	{
	"epoch": 0.09084302325581395,
	"grad_norm": 3.384547233581543,
	"learning_rate": 4.510869565217392e-06,
	"loss": 0.8972224235534668,
	"step": 250
	},
	{
	"epoch": 0.09447674418604651,
	"grad_norm": 3.213975429534912,
	"learning_rate": 4.692028985507246e-06,
	"loss": 0.8767337799072266,
	"step": 260
	},
	{
	"epoch": 0.09811046511627906,
	"grad_norm": 2.90049409866333,
	"learning_rate": 4.873188405797102e-06,
	"loss": 0.8697729110717773,
	"step": 270
	},
	{
	"epoch": 0.10174418604651163,
	"grad_norm": 3.3197343349456787,
	"learning_rate": 4.999995937615819e-06,
	"loss": 0.9078285217285156,
	"step": 280
	},
	{
	"epoch": 0.10537790697674419,
	"grad_norm": 3.3703126907348633,
	"learning_rate": 4.999923717819873e-06,
	"loss": 0.9200371742248535,
	"step": 290
	},
	{
	"epoch": 0.10901162790697674,
	"grad_norm": 3.1578056812286377,
	"learning_rate": 4.999761225821655e-06,
	"loss": 0.8725878715515136,
	"step": 300
	},
	{
	"epoch": 0.11264534883720931,
	"grad_norm": 3.0317864418029785,
	"learning_rate": 4.999508467488746e-06,
	"loss": 0.856719970703125,
	"step": 310
	},
	{
	"epoch": 0.11627906976744186,
	"grad_norm": 3.2835702896118164,
	"learning_rate": 4.999165451948247e-06,
	"loss": 0.8956731796264649,
	"step": 320
	},
	{
	"epoch": 0.11991279069767442,
	"grad_norm": 3.286651611328125,
	"learning_rate": 4.998732191586442e-06,
	"loss": 0.8485555648803711,
	"step": 330
	},
	{
	"epoch": 0.12354651162790697,
	"grad_norm": 2.9587960243225098,
	"learning_rate": 4.998208702048356e-06,
	"loss": 0.8765426635742187,
	"step": 340
	},
	{
	"epoch": 0.12718023255813954,
	"grad_norm": 3.0377676486968994,
	"learning_rate": 4.997595002237186e-06,
	"loss": 0.8789534568786621,
	"step": 350
	},
	{
	"epoch": 0.1308139534883721,
	"grad_norm": 3.0630507469177246,
	"learning_rate": 4.9968911143136225e-06,
	"loss": 0.8233914375305176,
	"step": 360
	},
	{
	"epoch": 0.13444767441860464,
	"grad_norm": 3.360434055328369,
	"learning_rate": 4.996097063695047e-06,
	"loss": 0.8773544311523438,
	"step": 370
	},
	{
	"epoch": 0.1380813953488372,
	"grad_norm": 3.1635208129882812,
	"learning_rate": 4.9952128790546115e-06,
	"loss": 0.8583930969238281,
	"step": 380
	},
	{
	"epoch": 0.14171511627906977,
	"grad_norm": 3.3118343353271484,
	"learning_rate": 4.9942385923202096e-06,
	"loss": 0.8331462860107421,
	"step": 390
	},
	{
	"epoch": 0.14534883720930233,
	"grad_norm": 2.983292579650879,
	"learning_rate": 4.993174238673318e-06,
	"loss": 0.8543838500976563,
	"step": 400
	},
	{
	"epoch": 0.14898255813953487,
	"grad_norm": 3.0217156410217285,
	"learning_rate": 4.9920198565477295e-06,
	"loss": 0.8610661506652832,
	"step": 410
	},
	{
	"epoch": 0.15261627906976744,
	"grad_norm": 2.80354380607605,
	"learning_rate": 4.990775487628162e-06,
	"loss": 0.833210563659668,
	"step": 420
	},
	{
	"epoch": 0.15625,
	"grad_norm": 3.045804500579834,
	"learning_rate": 4.989441176848755e-06,
	"loss": 0.870855712890625,
	"step": 430
	},
	{
	"epoch": 0.15988372093023256,
	"grad_norm": 2.7781946659088135,
	"learning_rate": 4.988016972391448e-06,
	"loss": 0.8238213539123536,
	"step": 440
	},
	{
	"epoch": 0.16351744186046513,
	"grad_norm": 2.781770706176758,
	"learning_rate": 4.98650292568424e-06,
	"loss": 0.8713046073913574,
	"step": 450
	},
	{
	"epoch": 0.16715116279069767,
	"grad_norm": 3.03906512260437,
	"learning_rate": 4.984899091399331e-06,
	"loss": 0.8518487930297851,
	"step": 460
	},
	{
	"epoch": 0.17078488372093023,
	"grad_norm": 3.0925002098083496,
	"learning_rate": 4.98320552745115e-06,
	"loss": 0.840611743927002,
	"step": 470
	},
	{
	"epoch": 0.1744186046511628,
	"grad_norm": 2.668984889984131,
	"learning_rate": 4.981422294994261e-06,
	"loss": 0.8196014404296875,
	"step": 480
	},
	{
	"epoch": 0.17805232558139536,
	"grad_norm": 2.7378602027893066,
	"learning_rate": 4.9795494584211575e-06,
	"loss": 0.8391487121582031,
	"step": 490
	},
	{
	"epoch": 0.1816860465116279,
	"grad_norm": 3.0461721420288086,
	"learning_rate": 4.9775870853599365e-06,
	"loss": 0.8854215621948243,
	"step": 500
	},
	{
	"epoch": 0.18531976744186046,
	"grad_norm": 2.9957311153411865,
	"learning_rate": 4.975535246671853e-06,
	"loss": 0.8136476516723633,
	"step": 510
	},
	{
	"epoch": 0.18895348837209303,
	"grad_norm": 2.7893149852752686,
	"learning_rate": 4.97339401644877e-06,
	"loss": 0.8376171112060546,
	"step": 520
	},
	{
	"epoch": 0.1925872093023256,
	"grad_norm": 2.820272922515869,
	"learning_rate": 4.9711634720104714e-06,
	"loss": 0.8519861221313476,
	"step": 530
	},
	{
	"epoch": 0.19622093023255813,
	"grad_norm": 2.8363044261932373,
	"learning_rate": 4.968843693901879e-06,
	"loss": 0.8639166831970215,
	"step": 540
	},
	{
	"epoch": 0.1998546511627907,
	"grad_norm": 2.7104744911193848,
	"learning_rate": 4.96643476589014e-06,
	"loss": 0.8391305923461914,
	"step": 550
	},
	{
	"epoch": 0.20348837209302326,
	"grad_norm": 2.6412911415100098,
	"learning_rate": 4.963936774961605e-06,
	"loss": 0.8537864685058594,
	"step": 560
	},
	{
	"epoch": 0.20712209302325582,
	"grad_norm": 3.1969637870788574,
	"learning_rate": 4.961349811318681e-06,
	"loss": 0.8276303291320801,
	"step": 570
	},
	{
	"epoch": 0.21075581395348839,
	"grad_norm": 2.824836015701294,
	"learning_rate": 4.958673968376582e-06,
	"loss": 0.8226048469543457,
	"step": 580
	},
	{
	"epoch": 0.21438953488372092,
	"grad_norm": 2.7200229167938232,
	"learning_rate": 4.955909342759949e-06,
	"loss": 0.8440999984741211,
	"step": 590
	},
	{
	"epoch": 0.2180232558139535,
	"grad_norm": 2.751291036605835,
	"learning_rate": 4.953056034299367e-06,
	"loss": 0.8424409866333008,
	"step": 600
	},
	{
	"epoch": 0.22165697674418605,
	"grad_norm": 3.263523817062378,
	"learning_rate": 4.9501141460277536e-06,
	"loss": 0.8259700775146485,
	"step": 610
	},
	{
	"epoch": 0.22529069767441862,
	"grad_norm": 2.875143527984619,
	"learning_rate": 4.9470837841766446e-06,
	"loss": 0.8352272033691406,
	"step": 620
	},
	{
	"epoch": 0.22892441860465115,
	"grad_norm": 2.642139196395874,
	"learning_rate": 4.943965058172354e-06,
	"loss": 0.8158163070678711,
	"step": 630
	},
	{
	"epoch": 0.23255813953488372,
	"grad_norm": 2.5755996704101562,
	"learning_rate": 4.940758080632023e-06,
	"loss": 0.7944734573364258,
	"step": 640
	},
	{
	"epoch": 0.23619186046511628,
	"grad_norm": 3.11088490486145,
	"learning_rate": 4.937462967359556e-06,
	"loss": 0.822010612487793,
	"step": 650
	},
	{
	"epoch": 0.23982558139534885,
	"grad_norm": 2.9258079528808594,
	"learning_rate": 4.934079837341436e-06,
	"loss": 0.8047715187072754,
	"step": 660
	},
	{
	"epoch": 0.24345930232558138,
	"grad_norm": 2.5476019382476807,
	"learning_rate": 4.93060881274243e-06,
	"loss": 0.8176538467407226,
	"step": 670
	},
	{
	"epoch": 0.24709302325581395,
	"grad_norm": 2.7018723487854004,
	"learning_rate": 4.927050018901176e-06,
	"loss": 0.8218058586120606,
	"step": 680
	},
	{
	"epoch": 0.2507267441860465,
	"grad_norm": 2.8049678802490234,
	"learning_rate": 4.923403584325659e-06,
	"loss": 0.8292177200317383,
	"step": 690
	},
	{
	"epoch": 0.2543604651162791,
	"grad_norm": 2.6390583515167236,
	"learning_rate": 4.919669640688567e-06,
	"loss": 0.8091305732727051,
	"step": 700
	},
	{
	"epoch": 0.25799418604651164,
	"grad_norm": 2.4425857067108154,
	"learning_rate": 4.91584832282254e-06,
	"loss": 0.8342226028442383,
	"step": 710
	},
	{
	"epoch": 0.2616279069767442,
	"grad_norm": 2.49047589302063,
	"learning_rate": 4.911939768715302e-06,
	"loss": 0.8127988815307617,
	"step": 720
	},
	{
	"epoch": 0.26526162790697677,
	"grad_norm": 2.6440837383270264,
	"learning_rate": 4.907944119504674e-06,
	"loss": 0.8182649612426758,
	"step": 730
	},
	{
	"epoch": 0.2688953488372093,
	"grad_norm": 2.7478673458099365,
	"learning_rate": 4.903861519473479e-06,
	"loss": 0.8209170341491699,
	"step": 740
	},
	{
	"epoch": 0.27252906976744184,
	"grad_norm": 2.9532063007354736,
	"learning_rate": 4.899692116044334e-06,
	"loss": 0.8080958366394043,
	"step": 750
	},
	{
	"epoch": 0.2761627906976744,
	"grad_norm": 3.037907600402832,
	"learning_rate": 4.895436059774325e-06,
	"loss": 0.8339145660400391,
	"step": 760
	},
	{
	"epoch": 0.279796511627907,
	"grad_norm": 2.694559335708618,
	"learning_rate": 4.89109350434957e-06,
	"loss": 0.8278702735900879,
	"step": 770
	},
	{
	"epoch": 0.28343023255813954,
	"grad_norm": 2.666208267211914,
	"learning_rate": 4.886664606579671e-06,
	"loss": 0.7949569702148438,
	"step": 780
	},
	{
	"epoch": 0.2870639534883721,
	"grad_norm": 2.8232085704803467,
	"learning_rate": 4.88214952639205e-06,
	"loss": 0.826717758178711,
	"step": 790
	},
	{
	"epoch": 0.29069767441860467,
	"grad_norm": 2.638852834701538,
	"learning_rate": 4.877548426826173e-06,
	"loss": 0.8023736953735352,
	"step": 800
	},
	{
	"epoch": 0.29433139534883723,
	"grad_norm": 2.5360491275787354,
	"learning_rate": 4.872861474027667e-06,
	"loss": 0.8074925422668457,
	"step": 810
	},
	{
	"epoch": 0.29796511627906974,
	"grad_norm": 2.6113131046295166,
	"learning_rate": 4.868088837242318e-06,
	"loss": 0.8043436050415039,
	"step": 820
	},
	{
	"epoch": 0.3015988372093023,
	"grad_norm": 2.6305062770843506,
	"learning_rate": 4.863230688809955e-06,
	"loss": 0.7932482719421386,
	"step": 830
	},
	{
	"epoch": 0.30523255813953487,
	"grad_norm": 2.6593618392944336,
	"learning_rate": 4.858287204158236e-06,
	"loss": 0.824673843383789,
	"step": 840
	},
	{
	"epoch": 0.30886627906976744,
	"grad_norm": 2.8339638710021973,
	"learning_rate": 4.853258561796307e-06,
	"loss": 0.7906547069549561,
	"step": 850
	},
	{
	"epoch": 0.3125,
	"grad_norm": 2.6146087646484375,
	"learning_rate": 4.848144943308355e-06,
	"loss": 0.7977964401245117,
	"step": 860
	},
	{
	"epoch": 0.31613372093023256,
	"grad_norm": 2.471606969833374,
	"learning_rate": 4.842946533347056e-06,
	"loss": 0.7975066184997559,
	"step": 870
	},
	{
	"epoch": 0.31976744186046513,
	"grad_norm": 2.57033371925354,
	"learning_rate": 4.837663519626904e-06,
	"loss": 0.7908267021179199,
	"step": 880
	},
	{
	"epoch": 0.3234011627906977,
	"grad_norm": 2.6324503421783447,
	"learning_rate": 4.8322960929174325e-06,
	"loss": 0.7818597793579102,
	"step": 890
	},
	{
	"epoch": 0.32703488372093026,
	"grad_norm": 2.5802016258239746,
	"learning_rate": 4.826844447036329e-06,
	"loss": 0.791273546218872,
	"step": 900
	},
	{
	"epoch": 0.33066860465116277,
	"grad_norm": 2.489678382873535,
	"learning_rate": 4.82130877884243e-06,
	"loss": 0.7943994522094726,
	"step": 910
	},
	{
	"epoch": 0.33430232558139533,
	"grad_norm": 2.4453723430633545,
	"learning_rate": 4.8156892882286206e-06,
	"loss": 0.8073820114135742,
	"step": 920
	},
	{
	"epoch": 0.3379360465116279,
	"grad_norm": 2.7187907695770264,
	"learning_rate": 4.809986178114608e-06,
	"loss": 0.7841668128967285,
	"step": 930
	},
	{
	"epoch": 0.34156976744186046,
	"grad_norm": 2.3783340454101562,
	"learning_rate": 4.804199654439601e-06,
	"loss": 0.8067031860351562,
	"step": 940
	},
	{
	"epoch": 0.345203488372093,
	"grad_norm": 2.6058096885681152,
	"learning_rate": 4.798329926154868e-06,
	"loss": 0.797321081161499,
	"step": 950
	},
	{
	"epoch": 0.3488372093023256,
	"grad_norm": 2.717211961746216,
	"learning_rate": 4.792377205216198e-06,
	"loss": 0.8069614410400391,
	"step": 960
	},
	{
	"epoch": 0.35247093023255816,
	"grad_norm": 2.3637256622314453,
	"learning_rate": 4.78634170657624e-06,
	"loss": 0.7931838989257812,
	"step": 970
	},
	{
	"epoch": 0.3561046511627907,
	"grad_norm": 2.5873794555664062,
	"learning_rate": 4.780223648176746e-06,
	"loss": 0.7875592231750488,
	"step": 980
	},
	{
	"epoch": 0.35973837209302323,
	"grad_norm": 2.4815046787261963,
	"learning_rate": 4.774023250940699e-06,
	"loss": 0.8080557823181153,
	"step": 990
	},
	{
	"epoch": 0.3633720930232558,
	"grad_norm": 2.5485689640045166,
	"learning_rate": 4.767740738764337e-06,
	"loss": 0.7884207248687745,
	"step": 1000
	},
	{
	"epoch": 0.36700581395348836,
	"grad_norm": 2.548661231994629,
	"learning_rate": 4.761376338509065e-06,
	"loss": 0.7941656589508057,
	"step": 1010
	},
	{
	"epoch": 0.3706395348837209,
	"grad_norm": 2.677043914794922,
	"learning_rate": 4.754930279993266e-06,
	"loss": 0.7980897903442383,
	"step": 1020
	},
	{
	"epoch": 0.3742732558139535,
	"grad_norm": 2.370474100112915,
	"learning_rate": 4.748402795984004e-06,
	"loss": 0.7892168998718262,
	"step": 1030
	},
	{
	"epoch": 0.37790697674418605,
	"grad_norm": 2.592254161834717,
	"learning_rate": 4.741794122188611e-06,
	"loss": 0.7861084938049316,
	"step": 1040
	},
	{
	"epoch": 0.3815406976744186,
	"grad_norm": 2.4242539405822754,
	"learning_rate": 4.735104497246185e-06,
	"loss": 0.7972273349761962,
	"step": 1050
	},
	{
	"epoch": 0.3851744186046512,
	"grad_norm": 2.468698740005493,
	"learning_rate": 4.728334162718967e-06,
	"loss": 0.7848814964294434,
	"step": 1060
	},
	{
	"epoch": 0.38880813953488375,
	"grad_norm": 2.5513319969177246,
	"learning_rate": 4.72148336308362e-06,
	"loss": 0.8222492218017579,
	"step": 1070
	},
	{
	"epoch": 0.39244186046511625,
	"grad_norm": 2.4857430458068848,
	"learning_rate": 4.7145523457224005e-06,
	"loss": 0.7939577579498291,
	"step": 1080
	},
	{
	"epoch": 0.3960755813953488,
	"grad_norm": 2.5042176246643066,
	"learning_rate": 4.707541360914225e-06,
	"loss": 0.7967430114746094,
	"step": 1090
	},
	{
	"epoch": 0.3997093023255814,
	"grad_norm": 2.50421142578125,
	"learning_rate": 4.700450661825635e-06,
	"loss": 0.7649580478668213,
	"step": 1100
	},
	{
	"epoch": 0.40334302325581395,
	"grad_norm": 2.4666318893432617,
	"learning_rate": 4.693280504501651e-06,
	"loss": 0.7870227813720703,
	"step": 1110
	},
	{
	"epoch": 0.4069767441860465,
	"grad_norm": 2.3815321922302246,
	"learning_rate": 4.686031147856531e-06,
	"loss": 0.8183629989624024,
	"step": 1120
	},
	{
	"epoch": 0.4106104651162791,
	"grad_norm": 2.4221959114074707,
	"learning_rate": 4.678702853664418e-06,
	"loss": 0.7800597667694091,
	"step": 1130
	},
	{
	"epoch": 0.41424418604651164,
	"grad_norm": 2.4387614727020264,
	"learning_rate": 4.671295886549887e-06,
	"loss": 0.7944596290588379,
	"step": 1140
	},
	{
	"epoch": 0.4178779069767442,
	"grad_norm": 2.5819296836853027,
	"learning_rate": 4.6638105139783955e-06,
	"loss": 0.7856440067291259,
	"step": 1150
	},
	{
	"epoch": 0.42151162790697677,
	"grad_norm": 2.115025520324707,
	"learning_rate": 4.656247006246617e-06,
	"loss": 0.7786880493164062,
	"step": 1160
	},
	{
	"epoch": 0.4251453488372093,
	"grad_norm": 2.465291738510132,
	"learning_rate": 4.648605636472683e-06,
	"loss": 0.7897854804992676,
	"step": 1170
	},
	{
	"epoch": 0.42877906976744184,
	"grad_norm": 2.4061923027038574,
	"learning_rate": 4.640886680586326e-06,
	"loss": 0.7431286334991455,
	"step": 1180
	},
	{
	"epoch": 0.4324127906976744,
	"grad_norm": 2.412606954574585,
	"learning_rate": 4.6330904173189105e-06,
	"loss": 0.7887528896331787,
	"step": 1190
	},
	{
	"epoch": 0.436046511627907,
	"grad_norm": 2.4691309928894043,
	"learning_rate": 4.625217128193366e-06,
	"loss": 0.8178772926330566,
	"step": 1200
	},
	{
	"epoch": 0.43968023255813954,
	"grad_norm": 2.4143872261047363,
	"learning_rate": 4.617267097514028e-06,
	"loss": 0.7813047409057617,
	"step": 1210
	},
	{
	"epoch": 0.4433139534883721,
	"grad_norm": 2.5029988288879395,
	"learning_rate": 4.609240612356367e-06,
	"loss": 0.7546382427215577,
	"step": 1220
	},
	{
	"epoch": 0.44694767441860467,
	"grad_norm": 2.3429665565490723,
	"learning_rate": 4.6011379625566245e-06,
	"loss": 0.8102869033813477,
	"step": 1230
	},
	{
	"epoch": 0.45058139534883723,
	"grad_norm": 2.2588775157928467,
	"learning_rate": 4.5929594407013434e-06,
	"loss": 0.7714152336120605,
	"step": 1240
	},
	{
	"epoch": 0.45421511627906974,
	"grad_norm": 2.393068313598633,
	"learning_rate": 4.584705342116807e-06,
	"loss": 0.7810145854949951,
	"step": 1250
	},
	{
	"epoch": 0.4578488372093023,
	"grad_norm": 2.47301983833313,
	"learning_rate": 4.576375964858373e-06,
	"loss": 0.7458586692810059,
	"step": 1260
	},
	{
	"epoch": 0.46148255813953487,
	"grad_norm": 2.4680774211883545,
	"learning_rate": 4.567971609699709e-06,
	"loss": 0.7881102085113525,
	"step": 1270
	},
	{
	"epoch": 0.46511627906976744,
	"grad_norm": 2.3784594535827637,
	"learning_rate": 4.559492580121936e-06,
	"loss": 0.7897718906402588,
	"step": 1280
	},
	{
	"epoch": 0.46875,
	"grad_norm": 2.3376259803771973,
	"learning_rate": 4.550939182302666e-06,
	"loss": 0.7819802761077881,
	"step": 1290
	},
	{
	"epoch": 0.47238372093023256,
	"grad_norm": 2.5781242847442627,
	"learning_rate": 4.5423117251049455e-06,
	"loss": 0.7619577884674072,
	"step": 1300
	},
	{
	"epoch": 0.47601744186046513,
	"grad_norm": 2.6463913917541504,
	"learning_rate": 4.533610520066104e-06,
	"loss": 0.8002893447875976,
	"step": 1310
	},
	{
	"epoch": 0.4796511627906977,
	"grad_norm": 2.5348868370056152,
	"learning_rate": 4.524835881386508e-06,
	"loss": 0.797673225402832,
	"step": 1320
	},
	{
	"epoch": 0.48328488372093026,
	"grad_norm": 2.4320013523101807,
	"learning_rate": 4.515988125918206e-06,
	"loss": 0.7804751873016358,
	"step": 1330
	},
	{
	"epoch": 0.48691860465116277,
	"grad_norm": 2.1748507022857666,
	"learning_rate": 4.507067573153496e-06,
	"loss": 0.7891386032104493,
	"step": 1340
	},
	{
	"epoch": 0.49055232558139533,
	"grad_norm": 2.336627721786499,
	"learning_rate": 4.498074545213386e-06,
	"loss": 0.7624752998352051,
	"step": 1350
	},
	{
	"epoch": 0.4941860465116279,
	"grad_norm": 2.323540687561035,
	"learning_rate": 4.489009366835956e-06,
	"loss": 0.778096866607666,
	"step": 1360
	},
	{
	"epoch": 0.49781976744186046,
	"grad_norm": 2.4476687908172607,
	"learning_rate": 4.479872365364644e-06,
	"loss": 0.7627400398254395,
	"step": 1370
	},
	{
	"epoch": 0.501453488372093,
	"grad_norm": 2.5058326721191406,
	"learning_rate": 4.470663870736414e-06,
	"loss": 0.7500468730926514,
	"step": 1380
	},
	{
	"epoch": 0.5050872093023255,
	"grad_norm": 2.414111375808716,
	"learning_rate": 4.461384215469847e-06,
	"loss": 0.7830061912536621,
	"step": 1390
	},
	{
	"epoch": 0.5087209302325582,
	"grad_norm": 2.4094791412353516,
	"learning_rate": 4.452033734653136e-06,
	"loss": 0.7941655158996582,
	"step": 1400
	},
	{
	"epoch": 0.5123546511627907,
	"grad_norm": 2.2999656200408936,
	"learning_rate": 4.442612765931981e-06,
	"loss": 0.7837831974029541,
	"step": 1410
	},
	{
	"epoch": 0.5159883720930233,
	"grad_norm": 2.477172374725342,
	"learning_rate": 4.433121649497401e-06,
	"loss": 0.7861865997314453,
	"step": 1420
	},
	{
	"epoch": 0.5196220930232558,
	"grad_norm": 2.3494627475738525,
	"learning_rate": 4.423560728073444e-06,
	"loss": 0.7856459617614746,
	"step": 1430
	},
	{
	"epoch": 0.5232558139534884,
	"grad_norm": 2.4817135334014893,
	"learning_rate": 4.413930346904819e-06,
	"loss": 0.7574215412139893,
	"step": 1440
	},
	{
	"epoch": 0.5268895348837209,
	"grad_norm": 2.7172696590423584,
	"learning_rate": 4.404230853744424e-06,
	"loss": 0.7790991306304932,
	"step": 1450
	},
	{
	"epoch": 0.5305232558139535,
	"grad_norm": 2.328077793121338,
	"learning_rate": 4.3944625988407896e-06,
	"loss": 0.7736337184906006,
	"step": 1460
	},
	{
	"epoch": 0.534156976744186,
	"grad_norm": 2.3900513648986816,
	"learning_rate": 4.384625934925431e-06,
	"loss": 0.7492892265319824,
	"step": 1470
	},
	{
	"epoch": 0.5377906976744186,
	"grad_norm": 2.4754834175109863,
	"learning_rate": 4.374721217200113e-06,
	"loss": 0.7459090232849122,
	"step": 1480
	},
	{
	"epoch": 0.5414244186046512,
	"grad_norm": 2.372633218765259,
	"learning_rate": 4.364748803324021e-06,
	"loss": 0.7739748001098633,
	"step": 1490
	},
	{
	"epoch": 0.5450581395348837,
	"grad_norm": 2.440763473510742,
	"learning_rate": 4.354709053400846e-06,
	"loss": 0.7933945655822754,
	"step": 1500
	},
	{
	"epoch": 0.5486918604651163,
	"grad_norm": 2.379467010498047,
	"learning_rate": 4.344602329965788e-06,
	"loss": 0.7429629802703858,
	"step": 1510
	},
	{
	"epoch": 0.5523255813953488,
	"grad_norm": 2.4870030879974365,
	"learning_rate": 4.334428997972454e-06,
	"loss": 0.7675507068634033,
	"step": 1520
	},
	{
	"epoch": 0.5559593023255814,
	"grad_norm": 2.2411887645721436,
	"learning_rate": 4.324189424779686e-06,
	"loss": 0.7561450004577637,
	"step": 1530
	},
	{
	"epoch": 0.559593023255814,
	"grad_norm": 2.42997407913208,
	"learning_rate": 4.313883980138296e-06,
	"loss": 0.7534577369689941,
	"step": 1540
	},
	{
	"epoch": 0.5632267441860465,
	"grad_norm": 2.348618507385254,
	"learning_rate": 4.303513036177711e-06,
	"loss": 0.8024792671203613,
	"step": 1550
	},
	{
	"epoch": 0.5668604651162791,
	"grad_norm": 2.283494710922241,
	"learning_rate": 4.293076967392539e-06,
	"loss": 0.7551123619079589,
	"step": 1560
	},
	{
	"epoch": 0.5704941860465116,
	"grad_norm": 2.291511058807373,
	"learning_rate": 4.282576150629044e-06,
	"loss": 0.8130410194396973,
	"step": 1570
	},
	{
	"epoch": 0.5741279069767442,
	"grad_norm": 3.0013701915740967,
	"learning_rate": 4.272010965071538e-06,
	"loss": 0.7796205520629883,
	"step": 1580
	},
	{
	"epoch": 0.5777616279069767,
	"grad_norm": 2.2320775985717773,
	"learning_rate": 4.2613817922286875e-06,
	"loss": 0.7652807235717773,
	"step": 1590
	},
	{
	"epoch": 0.5813953488372093,
	"grad_norm": 2.3117880821228027,
	"learning_rate": 4.250689015919741e-06,
	"loss": 0.761961030960083,
	"step": 1600
	},
	{
	"epoch": 0.5850290697674418,
	"grad_norm": 2.3330628871917725,
	"learning_rate": 4.239933022260668e-06,
	"loss": 0.8002090454101562,
	"step": 1610
	},
	{
	"epoch": 0.5886627906976745,
	"grad_norm": 2.322035789489746,
	"learning_rate": 4.2291141996502115e-06,
	"loss": 0.7480731964111328,
	"step": 1620
	},
	{
	"epoch": 0.592296511627907,
	"grad_norm": 2.3511040210723877,
	"learning_rate": 4.21823293875587e-06,
	"loss": 0.7778585433959961,
	"step": 1630
	},
	{
	"epoch": 0.5959302325581395,
	"grad_norm": 2.365743398666382,
	"learning_rate": 4.207289632499787e-06,
	"loss": 0.760723876953125,
	"step": 1640
	},
	{
	"epoch": 0.5995639534883721,
	"grad_norm": 2.3175790309906006,
	"learning_rate": 4.1962846760445606e-06,
	"loss": 0.7410211563110352,
	"step": 1650
	},
	{
	"epoch": 0.6031976744186046,
	"grad_norm": 2.391818046569824,
	"learning_rate": 4.185218466778981e-06,
	"loss": 0.7819056034088134,
	"step": 1660
	},
	{
	"epoch": 0.6068313953488372,
	"grad_norm": 3.155374765396118,
	"learning_rate": 4.174091404303671e-06,
	"loss": 0.7714629173278809,
	"step": 1670
	},
	{
	"epoch": 0.6104651162790697,
	"grad_norm": 2.527911424636841,
	"learning_rate": 4.162903890416666e-06,
	"loss": 0.7814093112945557,
	"step": 1680
	},
	{
	"epoch": 0.6140988372093024,
	"grad_norm": 2.2402873039245605,
	"learning_rate": 4.151656329098899e-06,
	"loss": 0.7395771026611329,
	"step": 1690
	},
	{
	"epoch": 0.6177325581395349,
	"grad_norm": 2.416172504425049,
	"learning_rate": 4.1403491264996144e-06,
	"loss": 0.7594813346862793,
	"step": 1700
	},
	{
	"epoch": 0.6213662790697675,
	"grad_norm": 2.4122121334075928,
	"learning_rate": 4.1289826909217034e-06,
	"loss": 0.753940200805664,
	"step": 1710
	},
	{
	"epoch": 0.625,
	"grad_norm": 2.3994522094726562,
	"learning_rate": 4.117557432806958e-06,
	"loss": 0.7865163803100585,
	"step": 1720
	},
	{
	"epoch": 0.6286337209302325,
	"grad_norm": 2.2479474544525146,
	"learning_rate": 4.106073764721252e-06,
	"loss": 0.7595019340515137,
	"step": 1730
	},
	{
	"epoch": 0.6322674418604651,
	"grad_norm": 2.4755139350891113,
	"learning_rate": 4.094532101339641e-06,
	"loss": 0.7513357162475586,
	"step": 1740
	},
	{
	"epoch": 0.6359011627906976,
	"grad_norm": 2.4134581089019775,
	"learning_rate": 4.08293285943139e-06,
	"loss": 0.7758606910705567,
	"step": 1750
	},
	{
	"epoch": 0.6395348837209303,
	"grad_norm": 2.3151111602783203,
	"learning_rate": 4.071276457844925e-06,
	"loss": 0.7795231342315674,
	"step": 1760
	},
	{
	"epoch": 0.6431686046511628,
	"grad_norm": 2.557997226715088,
	"learning_rate": 4.059563317492705e-06,
	"loss": 0.7613862037658692,
	"step": 1770
	},
	{
	"epoch": 0.6468023255813954,
	"grad_norm": 2.5041115283966064,
	"learning_rate": 4.0477938613360265e-06,
	"loss": 0.761210823059082,
	"step": 1780
	},
	{
	"epoch": 0.6504360465116279,
	"grad_norm": 2.360748767852783,
	"learning_rate": 4.035968514369748e-06,
	"loss": 0.7812687873840332,
	"step": 1790
	},
	{
	"epoch": 0.6540697674418605,
	"grad_norm": 2.282150983810425,
	"learning_rate": 4.024087703606945e-06,
	"loss": 0.7689810752868652,
	"step": 1800
	},
	{
	"epoch": 0.657703488372093,
	"grad_norm": 2.2814013957977295,
	"learning_rate": 4.012151858063487e-06,
	"loss": 0.7490324020385742,
	"step": 1810
	},
	{
	"epoch": 0.6613372093023255,
	"grad_norm": 2.361901044845581,
	"learning_rate": 4.0001614087425515e-06,
	"loss": 0.7782148361206055,
	"step": 1820
	},
	{
	"epoch": 0.6649709302325582,
	"grad_norm": 2.2950687408447266,
	"learning_rate": 3.988116788619055e-06,
	"loss": 0.7573251247406005,
	"step": 1830
	},
	{
	"epoch": 0.6686046511627907,
	"grad_norm": 2.345322847366333,
	"learning_rate": 3.976018432624025e-06,
	"loss": 0.77418212890625,
	"step": 1840
	},
	{
	"epoch": 0.6722383720930233,
	"grad_norm": 2.338822364807129,
	"learning_rate": 3.963866777628885e-06,
	"loss": 0.7585583686828613,
	"step": 1850
	},
	{
	"epoch": 0.6758720930232558,
	"grad_norm": 2.348903179168701,
	"learning_rate": 3.95166226242969e-06,
	"loss": 0.7470979690551758,
	"step": 1860
	},
	{
	"epoch": 0.6795058139534884,
	"grad_norm": 2.3524892330169678,
	"learning_rate": 3.939405327731268e-06,
	"loss": 0.7816557884216309,
	"step": 1870
	},
	{
	"epoch": 0.6831395348837209,
	"grad_norm": 2.2748517990112305,
	"learning_rate": 3.927096416131324e-06,
	"loss": 0.7381314277648926,
	"step": 1880
	},
	{
	"epoch": 0.6867732558139535,
	"grad_norm": 2.3923628330230713,
	"learning_rate": 3.914735972104442e-06,
	"loss": 0.7566698074340821,
	"step": 1890
	},
	{
	"epoch": 0.690406976744186,
	"grad_norm": 2.2513277530670166,
	"learning_rate": 3.902324441986041e-06,
	"loss": 0.7712888717651367,
	"step": 1900
	},
	{
	"epoch": 0.6940406976744186,
	"grad_norm": 2.2769501209259033,
	"learning_rate": 3.889862273956261e-06,
	"loss": 0.7604453086853027,
	"step": 1910
	},
	{
	"epoch": 0.6976744186046512,
	"grad_norm": 2.287050485610962,
	"learning_rate": 3.877349918023772e-06,
	"loss": 0.7452004432678223,
	"step": 1920
	},
	{
	"epoch": 0.7013081395348837,
	"grad_norm": 2.4155802726745605,
	"learning_rate": 3.864787826009533e-06,
	"loss": 0.7583384990692139,
	"step": 1930
	},
	{
	"epoch": 0.7049418604651163,
	"grad_norm": 2.3672666549682617,
	"learning_rate": 3.852176451530468e-06,
	"loss": 0.7433928489685059,
	"step": 1940
	},
	{
	"epoch": 0.7085755813953488,
	"grad_norm": 2.3730006217956543,
	"learning_rate": 3.839516249983093e-06,
	"loss": 0.721006441116333,
	"step": 1950
	},
	{
	"epoch": 0.7122093023255814,
	"grad_norm": 2.2506585121154785,
	"learning_rate": 3.826807678527067e-06,
	"loss": 0.7675203323364258,
	"step": 1960
	},
	{
	"epoch": 0.715843023255814,
	"grad_norm": 2.26613450050354,
	"learning_rate": 3.814051196068684e-06,
	"loss": 0.7598457336425781,
	"step": 1970
	},
	{
	"epoch": 0.7194767441860465,
	"grad_norm": 3.0172502994537354,
	"learning_rate": 3.8012472632443065e-06,
	"loss": 0.749884843826294,
	"step": 1980
	},
	{
	"epoch": 0.7231104651162791,
	"grad_norm": 3.941880226135254,
	"learning_rate": 3.7883963424037286e-06,
	"loss": 0.7476849555969238,
	"step": 1990
	},
	{
	"epoch": 0.7267441860465116,
	"grad_norm": 2.3040683269500732,
	"learning_rate": 3.775498897593478e-06,
	"loss": 0.7777666091918946,
	"step": 2000
	},
	{
	"epoch": 0.7303779069767442,
	"grad_norm": 2.299778699874878,
	"learning_rate": 3.762555394540066e-06,
	"loss": 0.7330509185791015,
	"step": 2010
	},
	{
	"epoch": 0.7340116279069767,
	"grad_norm": 2.2814173698425293,
	"learning_rate": 3.7495663006331633e-06,
	"loss": 0.7531011581420899,
	"step": 2020
	},
	{
	"epoch": 0.7376453488372093,
	"grad_norm": 2.3034424781799316,
	"learning_rate": 3.7365320849087262e-06,
	"loss": 0.7587457656860351,
	"step": 2030
	},
	{
	"epoch": 0.7412790697674418,
	"grad_norm": 2.358973264694214,
	"learning_rate": 3.7234532180320605e-06,
	"loss": 0.7624511241912841,
	"step": 2040
	},
	{
	"epoch": 0.7449127906976745,
	"grad_norm": 2.2100989818573,
	"learning_rate": 3.710330172280821e-06,
	"loss": 0.7730766296386719,
	"step": 2050
	},
	{
	"epoch": 0.748546511627907,
	"grad_norm": 2.31831693649292,
	"learning_rate": 3.6971634215279664e-06,
	"loss": 0.7575939178466797,
	"step": 2060
	},
	{
	"epoch": 0.7521802325581395,
	"grad_norm": 2.2341463565826416,
	"learning_rate": 3.683953441224636e-06,
	"loss": 0.726379108428955,
	"step": 2070
	},
	{
	"epoch": 0.7558139534883721,
	"grad_norm": 2.4005143642425537,
	"learning_rate": 3.6707007083829913e-06,
	"loss": 0.7606348037719727,
	"step": 2080
	},
	{
	"epoch": 0.7594476744186046,
	"grad_norm": 2.299570083618164,
	"learning_rate": 3.6574057015589877e-06,
	"loss": 0.7550271034240723,
	"step": 2090
	},
	{
	"epoch": 0.7630813953488372,
	"grad_norm": 2.247924566268921,
	"learning_rate": 3.6440689008350904e-06,
	"loss": 0.7482584953308106,
	"step": 2100
	},
	{
	"epoch": 0.7667151162790697,
	"grad_norm": 2.256894826889038,
	"learning_rate": 3.630690787802944e-06,
	"loss": 0.7816937446594239,
	"step": 2110
	},
	{
	"epoch": 0.7703488372093024,
	"grad_norm": 2.3339056968688965,
	"learning_rate": 3.6172718455459783e-06,
	"loss": 0.7460580825805664,
	"step": 2120
	},
	{
	"epoch": 0.7739825581395349,
	"grad_norm": 2.6155683994293213,
	"learning_rate": 3.6038125586219663e-06,
	"loss": 0.7558025360107422,
	"step": 2130
	},
	{
	"epoch": 0.7776162790697675,
	"grad_norm": 2.3319284915924072,
	"learning_rate": 3.5903134130455285e-06,
	"loss": 0.7401683807373047,
	"step": 2140
	},
	{
	"epoch": 0.78125,
	"grad_norm": 2.23057222366333,
	"learning_rate": 3.5767748962705756e-06,
	"loss": 0.7545011520385743,
	"step": 2150
	},
	{
	"epoch": 0.7848837209302325,
	"grad_norm": 2.2631583213806152,
	"learning_rate": 3.563197497172717e-06,
	"loss": 0.7662585735321045,
	"step": 2160
	},
	{
	"epoch": 0.7885174418604651,
	"grad_norm": 2.337887763977051,
	"learning_rate": 3.5495817060315975e-06,
	"loss": 0.7608307361602783,
	"step": 2170
	},
	{
	"epoch": 0.7921511627906976,
	"grad_norm": 3.1069204807281494,
	"learning_rate": 3.535928014513201e-06,
	"loss": 0.7439197063446045,
	"step": 2180
	},
	{
	"epoch": 0.7957848837209303,
	"grad_norm": 2.2606189250946045,
	"learning_rate": 3.5222369156520927e-06,
	"loss": 0.7460708141326904,
	"step": 2190
	},
	{
	"epoch": 0.7994186046511628,
	"grad_norm": 2.22314453125,
	"learning_rate": 3.5085089038336157e-06,
	"loss": 0.7369312763214111,
	"step": 2200
	},
	{
	"epoch": 0.8030523255813954,
	"grad_norm": 2.3230905532836914,
	"learning_rate": 3.49474447477604e-06,
	"loss": 0.7450941562652588,
	"step": 2210
	},
	{
	"epoch": 0.8066860465116279,
	"grad_norm": 2.4673714637756348,
	"learning_rate": 3.4809441255126598e-06,
	"loss": 0.738589096069336,
	"step": 2220
	},
	{
	"epoch": 0.8103197674418605,
	"grad_norm": 2.3054916858673096,
	"learning_rate": 3.4671083543738505e-06,
	"loss": 0.7657737731933594,
	"step": 2230
	},
	{
	"epoch": 0.813953488372093,
	"grad_norm": 2.3647828102111816,
	"learning_rate": 3.453237660969068e-06,
	"loss": 0.7373433113098145,
	"step": 2240
	},
	{
	"epoch": 0.8175872093023255,
	"grad_norm": 2.2854654788970947,
	"learning_rate": 3.439332546168812e-06,
	"loss": 0.7454282760620117,
	"step": 2250
	},
	{
	"epoch": 0.8212209302325582,
	"grad_norm": 2.323105812072754,
	"learning_rate": 3.425393512086538e-06,
	"loss": 0.718260669708252,
	"step": 2260
	},
	{
	"epoch": 0.8248546511627907,
	"grad_norm": 2.3059773445129395,
	"learning_rate": 3.4114210620605248e-06,
	"loss": 0.7474150657653809,
	"step": 2270
	},
	{
	"epoch": 0.8284883720930233,
	"grad_norm": 2.3977224826812744,
	"learning_rate": 3.3974157006357013e-06,
	"loss": 0.7327244281768799,
	"step": 2280
	},
	{
	"epoch": 0.8321220930232558,
	"grad_norm": 2.17206072807312,
	"learning_rate": 3.383377933545429e-06,
	"loss": 0.7716388702392578,
	"step": 2290
	},
	{
	"epoch": 0.8357558139534884,
	"grad_norm": 2.394636392593384,
	"learning_rate": 3.3693082676932326e-06,
	"loss": 0.7505688667297363,
	"step": 2300
	},
	{
	"epoch": 0.8393895348837209,
	"grad_norm": 2.2230443954467773,
	"learning_rate": 3.3552072111345047e-06,
	"loss": 0.7731509685516358,
	"step": 2310
	},
	{
	"epoch": 0.8430232558139535,
	"grad_norm": 2.244063138961792,
	"learning_rate": 3.341075273058153e-06,
	"loss": 0.7528995037078857,
	"step": 2320
	},
	{
	"epoch": 0.846656976744186,
	"grad_norm": 2.3284997940063477,
	"learning_rate": 3.3269129637682178e-06,
	"loss": 0.7626793861389161,
	"step": 2330
	},
	{
	"epoch": 0.8502906976744186,
	"grad_norm": 2.4036169052124023,
	"learning_rate": 3.3127207946654434e-06,
	"loss": 0.7294770240783691,
	"step": 2340
	},
	{
	"epoch": 0.8539244186046512,
	"grad_norm": 2.2926857471466064,
	"learning_rate": 3.29849927822881e-06,
	"loss": 0.7291061878204346,
	"step": 2350
	},
	{
	"epoch": 0.8575581395348837,
	"grad_norm": 2.279606580734253,
	"learning_rate": 3.2842489279970314e-06,
	"loss": 0.7511893749237061,
	"step": 2360
	},
	{
	"epoch": 0.8611918604651163,
	"grad_norm": 2.4814586639404297,
	"learning_rate": 3.2699702585500072e-06,
	"loss": 0.7769060134887695,
	"step": 2370
	},
	{
	"epoch": 0.8648255813953488,
	"grad_norm": 2.3231747150421143,
	"learning_rate": 3.2556637854902456e-06,
	"loss": 0.7389485836029053,
	"step": 2380
	},
	{
	"epoch": 0.8684593023255814,
	"grad_norm": 2.297140121459961,
	"learning_rate": 3.2413300254242417e-06,
	"loss": 0.7335830211639405,
	"step": 2390
	},
	{
	"epoch": 0.872093023255814,
	"grad_norm": 2.3252971172332764,
	"learning_rate": 3.2269694959438236e-06,
	"loss": 0.7166270732879638,
	"step": 2400
	},
	{
	"epoch": 0.8757267441860465,
	"grad_norm": 2.3775274753570557,
	"learning_rate": 3.212582715607463e-06,
	"loss": 0.7769906997680665,
	"step": 2410
	},
	{
	"epoch": 0.8793604651162791,
	"grad_norm": 2.162703514099121,
	"learning_rate": 3.1981702039215483e-06,
	"loss": 0.7350566387176514,
	"step": 2420
	},
	{
	"epoch": 0.8829941860465116,
	"grad_norm": 2.266566276550293,
	"learning_rate": 3.183732481321629e-06,
	"loss": 0.734774398803711,
	"step": 2430
	},
	{
	"epoch": 0.8866279069767442,
	"grad_norm": 2.4028637409210205,
	"learning_rate": 3.1692700691536184e-06,
	"loss": 0.7473856449127197,
	"step": 2440
	},
	{
	"epoch": 0.8902616279069767,
	"grad_norm": 2.1481335163116455,
	"learning_rate": 3.154783489654969e-06,
	"loss": 0.7348992347717285,
	"step": 2450
	},
	{
	"epoch": 0.8938953488372093,
	"grad_norm": 2.378777027130127,
	"learning_rate": 3.1402732659358187e-06,
	"loss": 0.7362776756286621,
	"step": 2460
	},
	{
	"epoch": 0.8975290697674418,
	"grad_norm": 2.073789596557617,
	"learning_rate": 3.1257399219600926e-06,
	"loss": 0.7534358024597168,
	"step": 2470
	},
	{
	"epoch": 0.9011627906976745,
	"grad_norm": 2.1593196392059326,
	"learning_rate": 3.1111839825265933e-06,
	"loss": 0.7116081237792968,
	"step": 2480
	},
	{
	"epoch": 0.904796511627907,
	"grad_norm": 2.356607437133789,
	"learning_rate": 3.0966059732500396e-06,
	"loss": 0.7617850303649902,
	"step": 2490
	},
	{
	"epoch": 0.9084302325581395,
	"grad_norm": 2.373751163482666,
	"learning_rate": 3.0820064205420967e-06,
	"loss": 0.7370933532714844,
	"step": 2500
	},
	{
	"epoch": 0.9120639534883721,
	"grad_norm": 2.187333106994629,
	"learning_rate": 3.0673858515923603e-06,
	"loss": 0.7156447410583496,
	"step": 2510
	},
	{
	"epoch": 0.9156976744186046,
	"grad_norm": 2.398669958114624,
	"learning_rate": 3.052744794349323e-06,
	"loss": 0.7433481216430664,
	"step": 2520
	},
	{
	"epoch": 0.9193313953488372,
	"grad_norm": 2.1346797943115234,
	"learning_rate": 3.0380837775013072e-06,
	"loss": 0.7418402671813965,
	"step": 2530
	},
	{
	"epoch": 0.9229651162790697,
	"grad_norm": 2.488369941711426,
	"learning_rate": 3.0234033304573797e-06,
	"loss": 0.7505437850952148,
	"step": 2540
	},
	{
	"epoch": 0.9265988372093024,
	"grad_norm": 2.360621929168701,
	"learning_rate": 3.0087039833282293e-06,
	"loss": 0.7594944477081299,
	"step": 2550
	},
	{
	"epoch": 0.9302325581395349,
	"grad_norm": 2.3394200801849365,
	"learning_rate": 2.9939862669070274e-06,
	"loss": 0.7139914989471435,
	"step": 2560
	},
	{
	"epoch": 0.9338662790697675,
	"grad_norm": 2.230193853378296,
	"learning_rate": 2.9792507126502595e-06,
	"loss": 0.7415294647216797,
	"step": 2570
	},
	{
	"epoch": 0.9375,
	"grad_norm": 2.3186755180358887,
	"learning_rate": 2.964497852658535e-06,
	"loss": 0.7491122245788574,
	"step": 2580
	},
	{
	"epoch": 0.9411337209302325,
	"grad_norm": 2.326460123062134,
	"learning_rate": 2.9497282196573733e-06,
	"loss": 0.7225167274475097,
	"step": 2590
	},
	{
	"epoch": 0.9447674418604651,
	"grad_norm": 2.2080297470092773,
	"learning_rate": 2.934942346977967e-06,
	"loss": 0.7309268951416016,
	"step": 2600
	},
	{
	"epoch": 0.9484011627906976,
	"grad_norm": 2.218813180923462,
	"learning_rate": 2.9201407685379242e-06,
	"loss": 0.7576736450195313,
	"step": 2610
	},
	{
	"epoch": 0.9520348837209303,
	"grad_norm": 2.2128567695617676,
	"learning_rate": 2.905324018821985e-06,
	"loss": 0.7640575408935547,
	"step": 2620
	},
	{
	"epoch": 0.9556686046511628,
	"grad_norm": 2.526268720626831,
	"learning_rate": 2.890492632862728e-06,
	"loss": 0.7474720001220703,
	"step": 2630
	},
	{
	"epoch": 0.9593023255813954,
	"grad_norm": 2.3193838596343994,
	"learning_rate": 2.875647146221241e-06,
	"loss": 0.731212043762207,
	"step": 2640
	},
	{
	"epoch": 0.9629360465116279,
	"grad_norm": 2.3635311126708984,
	"learning_rate": 2.860788094967793e-06,
	"loss": 0.7533315181732178,
	"step": 2650
	},
	{
	"epoch": 0.9665697674418605,
	"grad_norm": 2.2626616954803467,
	"learning_rate": 2.845916015662467e-06,
	"loss": 0.7371150970458984,
	"step": 2660
	},
	{
	"epoch": 0.970203488372093,
	"grad_norm": 2.267836570739746,
	"learning_rate": 2.8310314453357902e-06,
	"loss": 0.750466251373291,
	"step": 2670
	},
	{
	"epoch": 0.9738372093023255,
	"grad_norm": 2.321403980255127,
	"learning_rate": 2.816134921469339e-06,
	"loss": 0.7606642723083497,
	"step": 2680
	},
	{
	"epoch": 0.9774709302325582,
	"grad_norm": 2.2467598915100098,
	"learning_rate": 2.8012269819763337e-06,
	"loss": 0.7392509460449219,
	"step": 2690
	},
	{
	"epoch": 0.9811046511627907,
	"grad_norm": 2.381352424621582,
	"learning_rate": 2.7863081651822122e-06,
	"loss": 0.7417672157287598,
	"step": 2700
	},
	{
	"epoch": 0.9847383720930233,
	"grad_norm": 2.2145116329193115,
	"learning_rate": 2.7713790098051906e-06,
	"loss": 0.7176500797271729,
	"step": 2710
	},
	{
	"epoch": 0.9883720930232558,
	"grad_norm": 2.23537278175354,
	"learning_rate": 2.7564400549368127e-06,
	"loss": 0.7362884521484375,
	"step": 2720
	},
	{
	"epoch": 0.9920058139534884,
	"grad_norm": 2.492530345916748,
	"learning_rate": 2.74149184002248e-06,
	"loss": 0.7469696044921875,
	"step": 2730
	},
	{
	"epoch": 0.9956395348837209,
	"grad_norm": 2.3141868114471436,
	"learning_rate": 2.726534904841974e-06,
	"loss": 0.7469913482666015,
	"step": 2740
	},
	{
	"epoch": 0.9992732558139535,
	"grad_norm": 2.286506175994873,
	"learning_rate": 2.7115697894899675e-06,
	"loss": 0.7334429740905761,
	"step": 2750
	},
	{
	"epoch": 1.002906976744186,
	"grad_norm": 2.1241118907928467,
	"learning_rate": 2.696597034356517e-06,
	"loss": 0.684693431854248,
	"step": 2760
	},
	{
	"epoch": 1.0065406976744187,
	"grad_norm": 2.3246991634368896,
	"learning_rate": 2.6816171801075507e-06,
	"loss": 0.641628646850586,
	"step": 2770
	},
	{
	"epoch": 1.010174418604651,
	"grad_norm": 2.4200432300567627,
	"learning_rate": 2.6666307676653485e-06,
	"loss": 0.6292952537536621,
	"step": 2780
	},
	{
	"epoch": 1.0138081395348837,
	"grad_norm": 2.2513413429260254,
	"learning_rate": 2.651638338189003e-06,
	"loss": 0.6401253700256347,
	"step": 2790
	},
	{
	"epoch": 1.0174418604651163,
	"grad_norm": 2.48384952545166,
	"learning_rate": 2.6366404330548847e-06,
	"loss": 0.6396333694458007,
	"step": 2800
	},
	{
	"epoch": 1.021075581395349,
	"grad_norm": 2.3938305377960205,
	"learning_rate": 2.6216375938370887e-06,
	"loss": 0.6436416625976562,
	"step": 2810
	},
	{
	"epoch": 1.0247093023255813,
	"grad_norm": 2.364605665206909,
	"learning_rate": 2.6066303622878802e-06,
	"loss": 0.6255351543426514,
	"step": 2820
	},
	{
	"epoch": 1.028343023255814,
	"grad_norm": 2.3612289428710938,
	"learning_rate": 2.591619280318131e-06,
	"loss": 0.6562280178070068,
	"step": 2830
	},
	{
	"epoch": 1.0319767441860466,
	"grad_norm": 2.2469143867492676,
	"learning_rate": 2.5766048899777524e-06,
	"loss": 0.6595710754394531,
	"step": 2840
	},
	{
	"epoch": 1.035610465116279,
	"grad_norm": 2.2446906566619873,
	"learning_rate": 2.561587733436119e-06,
	"loss": 0.6045775413513184,
	"step": 2850
	},
	{
	"epoch": 1.0392441860465116,
	"grad_norm": 2.3548073768615723,
	"learning_rate": 2.5465683529624953e-06,
	"loss": 0.6162781715393066,
	"step": 2860
	},
	{
	"epoch": 1.0428779069767442,
	"grad_norm": 2.2470242977142334,
	"learning_rate": 2.5315472909064502e-06,
	"loss": 0.6290149688720703,
	"step": 2870
	},
	{
	"epoch": 1.0465116279069768,
	"grad_norm": 2.373875856399536,
	"learning_rate": 2.5165250896782763e-06,
	"loss": 0.6437867641448974,
	"step": 2880
	},
	{
	"epoch": 1.0501453488372092,
	"grad_norm": 2.300671339035034,
	"learning_rate": 2.5015022917293983e-06,
	"loss": 0.6200816631317139,
	"step": 2890
	},
	{
	"epoch": 1.0537790697674418,
	"grad_norm": 2.3263323307037354,
	"learning_rate": 2.486479439532793e-06,
	"loss": 0.6472393035888672,
	"step": 2900
	},
	{
	"epoch": 1.0574127906976745,
	"grad_norm": 2.3209452629089355,
	"learning_rate": 2.4714570755633915e-06,
	"loss": 0.6044593811035156,
	"step": 2910
	},
	{
	"epoch": 1.0610465116279069,
	"grad_norm": 2.421056032180786,
	"learning_rate": 2.456435742278499e-06,
	"loss": 0.6231337070465088,
	"step": 2920
	},
	{
	"epoch": 1.0646802325581395,
	"grad_norm": 2.3039190769195557,
	"learning_rate": 2.441415982098201e-06,
	"loss": 0.6458144187927246,
	"step": 2930
	},
	{
	"epoch": 1.068313953488372,
	"grad_norm": 2.2657155990600586,
	"learning_rate": 2.4263983373857773e-06,
	"loss": 0.6138556480407715,
	"step": 2940
	},
	{
	"epoch": 1.0719476744186047,
	"grad_norm": 2.1495866775512695,
	"learning_rate": 2.4113833504281187e-06,
	"loss": 0.6035123348236084,
	"step": 2950
	},
	{
	"epoch": 1.0755813953488371,
	"grad_norm": 2.314849376678467,
	"learning_rate": 2.396371563416146e-06,
	"loss": 0.6484601974487305,
	"step": 2960
	},
	{
	"epoch": 1.0792151162790697,
	"grad_norm": 2.3036186695098877,
	"learning_rate": 2.3813635184252285e-06,
	"loss": 0.6278446197509766,
	"step": 2970
	},
	{
	"epoch": 1.0828488372093024,
	"grad_norm": 2.2794058322906494,
	"learning_rate": 2.3663597573956113e-06,
	"loss": 0.649768590927124,
	"step": 2980
	},
	{
	"epoch": 1.086482558139535,
	"grad_norm": 2.4643797874450684,
	"learning_rate": 2.3513608221128477e-06,
	"loss": 0.6214523315429688,
	"step": 2990
	},
	{
	"epoch": 1.0901162790697674,
	"grad_norm": 2.7062759399414062,
	"learning_rate": 2.336367254188229e-06,
	"loss": 0.6410791397094726,
	"step": 3000
	},
	{
	"epoch": 1.09375,
	"grad_norm": 2.3651273250579834,
	"learning_rate": 2.3213795950392355e-06,
	"loss": 0.6172582149505615,
	"step": 3010
	},
	{
	"epoch": 1.0973837209302326,
	"grad_norm": 2.371607780456543,
	"learning_rate": 2.3063983858699808e-06,
	"loss": 0.6253729820251465,
	"step": 3020
	},
	{
	"epoch": 1.101017441860465,
	"grad_norm": 2.2308261394500732,
	"learning_rate": 2.291424167651668e-06,
	"loss": 0.6136109352111816,
	"step": 3030
	},
	{
	"epoch": 1.1046511627906976,
	"grad_norm": 2.4627668857574463,
	"learning_rate": 2.2764574811030597e-06,
	"loss": 0.6419172286987305,
	"step": 3040
	},
	{
	"epoch": 1.1082848837209303,
	"grad_norm": 2.308950424194336,
	"learning_rate": 2.2614988666709464e-06,
	"loss": 0.618984317779541,
	"step": 3050
	},
	{
	"epoch": 1.1119186046511629,
	"grad_norm": 2.305047035217285,
	"learning_rate": 2.24654886451064e-06,
	"loss": 0.6118502140045166,
	"step": 3060
	},
	{
	"epoch": 1.1155523255813953,
	"grad_norm": 2.3247389793395996,
	"learning_rate": 2.2316080144664577e-06,
	"loss": 0.625124740600586,
	"step": 3070
	},
	{
	"epoch": 1.119186046511628,
	"grad_norm": 2.4493560791015625,
	"learning_rate": 2.216676856052237e-06,
	"loss": 0.6551538467407226,
	"step": 3080
	},
	{
	"epoch": 1.1228197674418605,
	"grad_norm": 2.132716178894043,
	"learning_rate": 2.2017559284318517e-06,
	"loss": 0.6467277526855468,
	"step": 3090
	},
	{
	"epoch": 1.1264534883720931,
	"grad_norm": 2.328946113586426,
	"learning_rate": 2.18684577039974e-06,
	"loss": 0.6202045440673828,
	"step": 3100
	},
	{
	"epoch": 1.1300872093023255,
	"grad_norm": 2.432185649871826,
	"learning_rate": 2.171946920361449e-06,
	"loss": 0.6500596046447754,
	"step": 3110
	},
	{
	"epoch": 1.1337209302325582,
	"grad_norm": 2.2914774417877197,
	"learning_rate": 2.157059916314199e-06,
	"loss": 0.6548232078552246,
	"step": 3120
	},
	{
	"epoch": 1.1373546511627908,
	"grad_norm": 2.4901769161224365,
	"learning_rate": 2.142185295827448e-06,
	"loss": 0.6379193305969239,
	"step": 3130
	},
	{
	"epoch": 1.1409883720930232,
	"grad_norm": 4.886627674102783,
	"learning_rate": 2.1273235960234844e-06,
	"loss": 0.6132206916809082,
	"step": 3140
	},
	{
	"epoch": 1.1446220930232558,
	"grad_norm": 3.491535186767578,
	"learning_rate": 2.1124753535580308e-06,
	"loss": 0.6380853652954102,
	"step": 3150
	},
	{
	"epoch": 1.1482558139534884,
	"grad_norm": 2.478628396987915,
	"learning_rate": 2.0976411046008683e-06,
	"loss": 0.6365983009338378,
	"step": 3160
	},
	{
	"epoch": 1.151889534883721,
	"grad_norm": 2.4086923599243164,
	"learning_rate": 2.0828213848164693e-06,
	"loss": 0.6142704486846924,
	"step": 3170
	},
	{
	"epoch": 1.1555232558139534,
	"grad_norm": 2.326162576675415,
	"learning_rate": 2.0680167293446585e-06,
	"loss": 0.6183312416076661,
	"step": 3180
	},
	{
	"epoch": 1.159156976744186,
	"grad_norm": 2.202583074569702,
	"learning_rate": 2.05322767278129e-06,
	"loss": 0.6308890819549561,
	"step": 3190
	},
	{
	"epoch": 1.1627906976744187,
	"grad_norm": 2.3551995754241943,
	"learning_rate": 2.0384547491589416e-06,
	"loss": 0.6144701957702636,
	"step": 3200
	},
	{
	"epoch": 1.166424418604651,
	"grad_norm": 2.440127372741699,
	"learning_rate": 2.023698491927629e-06,
	"loss": 0.6246836662292481,
	"step": 3210
	},
	{
	"epoch": 1.1700581395348837,
	"grad_norm": 2.36844801902771,
	"learning_rate": 2.008959433935547e-06,
	"loss": 0.6098629474639893,
	"step": 3220
	},
	{
	"epoch": 1.1736918604651163,
	"grad_norm": 2.477142572402954,
	"learning_rate": 1.9942381074098248e-06,
	"loss": 0.6160079002380371,
	"step": 3230
	},
	{
	"epoch": 1.177325581395349,
	"grad_norm": 2.25296688079834,
	"learning_rate": 1.9795350439373085e-06,
	"loss": 0.6055639266967774,
	"step": 3240
	},
	{
	"epoch": 1.1809593023255813,
	"grad_norm": 2.4230899810791016,
	"learning_rate": 1.9648507744453657e-06,
	"loss": 0.6263333797454834,
	"step": 3250
	},
	{
	"epoch": 1.184593023255814,
	"grad_norm": 2.275339365005493,
	"learning_rate": 1.9501858291827157e-06,
	"loss": 0.6378993511199951,
	"step": 3260
	},
	{
	"epoch": 1.1882267441860466,
	"grad_norm": 2.4432897567749023,
	"learning_rate": 1.935540737700278e-06,
	"loss": 0.6045506954193115,
	"step": 3270
	},
	{
	"epoch": 1.191860465116279,
	"grad_norm": 2.359424591064453,
	"learning_rate": 1.9209160288320517e-06,
	"loss": 0.6010058879852295,
	"step": 3280
	},
	{
	"epoch": 1.1954941860465116,
	"grad_norm": 2.453836679458618,
	"learning_rate": 1.906312230676024e-06,
	"loss": 0.6423003196716308,
	"step": 3290
	},
	{
	"epoch": 1.1991279069767442,
	"grad_norm": 2.323133707046509,
	"learning_rate": 1.891729870575092e-06,
	"loss": 0.6270840644836426,
	"step": 3300
	},
	{
	"epoch": 1.2027616279069768,
	"grad_norm": 2.602797031402588,
	"learning_rate": 1.8771694750980277e-06,
	"loss": 0.6354727268218994,
	"step": 3310
	},
	{
	"epoch": 1.2063953488372092,
	"grad_norm": 2.456242799758911,
	"learning_rate": 1.8626315700204606e-06,
	"loss": 0.6264838218688965,
	"step": 3320
	},
	{
	"epoch": 1.2100290697674418,
	"grad_norm": 2.510651111602783,
	"learning_rate": 1.848116680305892e-06,
	"loss": 0.6387592315673828,
	"step": 3330
	},
	{
	"epoch": 1.2136627906976745,
	"grad_norm": 2.453080892562866,
	"learning_rate": 1.833625330086739e-06,
	"loss": 0.6230563640594482,
	"step": 3340
	},
	{
	"epoch": 1.2172965116279069,
	"grad_norm": 2.348503589630127,
	"learning_rate": 1.8191580426454064e-06,
	"loss": 0.6384520053863525,
	"step": 3350
	},
	{
	"epoch": 1.2209302325581395,
	"grad_norm": 2.6607465744018555,
	"learning_rate": 1.8047153403953948e-06,
	"loss": 0.638615083694458,
	"step": 3360
	},
	{
	"epoch": 1.224563953488372,
	"grad_norm": 2.3196043968200684,
	"learning_rate": 1.7902977448624334e-06,
	"loss": 0.6396076202392578,
	"step": 3370
	},
	{
	"epoch": 1.2281976744186047,
	"grad_norm": 2.2998158931732178,
	"learning_rate": 1.7759057766656468e-06,
	"loss": 0.61277756690979,
	"step": 3380
	},
	{
	"epoch": 1.2318313953488371,
	"grad_norm": 2.343247890472412,
	"learning_rate": 1.7615399554987588e-06,
	"loss": 0.6225319385528565,
	"step": 3390
	},
	{
	"epoch": 1.2354651162790697,
	"grad_norm": 2.4423811435699463,
	"learning_rate": 1.7472008001113244e-06,
	"loss": 0.6134791851043702,
	"step": 3400
	},
	{
	"epoch": 1.2390988372093024,
	"grad_norm": 2.306137800216675,
	"learning_rate": 1.7328888282899956e-06,
	"loss": 0.6216968536376953,
	"step": 3410
	},
	{
	"epoch": 1.2427325581395348,
	"grad_norm": 2.430939197540283,
	"learning_rate": 1.7186045568398274e-06,
	"loss": 0.6352485656738281,
	"step": 3420
	},
	{
	"epoch": 1.2463662790697674,
	"grad_norm": 2.4834225177764893,
	"learning_rate": 1.7043485015656164e-06,
	"loss": 0.650179386138916,
	"step": 3430
	},
	{
	"epoch": 1.25,
	"grad_norm": 2.3769431114196777,
	"learning_rate": 1.6901211772532717e-06,
	"loss": 0.6062415599822998,
	"step": 3440
	},
	{
	"epoch": 1.2536337209302326,
	"grad_norm": 2.4731128215789795,
	"learning_rate": 1.675923097651228e-06,
	"loss": 0.6470130920410156,
	"step": 3450
	},
	{
	"epoch": 1.2572674418604652,
	"grad_norm": 2.276545763015747,
	"learning_rate": 1.6617547754518964e-06,
	"loss": 0.6461814403533935,
	"step": 3460
	},
	{
	"epoch": 1.2609011627906976,
	"grad_norm": 2.3915622234344482,
	"learning_rate": 1.6476167222731448e-06,
	"loss": 0.6211596012115479,
	"step": 3470
	},
	{
	"epoch": 1.2645348837209303,
	"grad_norm": 2.4467568397521973,
	"learning_rate": 1.6335094486398295e-06,
	"loss": 0.6250475883483887,
	"step": 3480
	},
	{
	"epoch": 1.2681686046511627,
	"grad_norm": 2.391965389251709,
	"learning_rate": 1.6194334639653588e-06,
	"loss": 0.6195459365844727,
	"step": 3490
	},
	{
	"epoch": 1.2718023255813953,
	"grad_norm": 2.2584242820739746,
	"learning_rate": 1.6053892765332957e-06,
	"loss": 0.6222965717315674,
	"step": 3500
	},
	{
	"epoch": 1.275436046511628,
	"grad_norm": 2.3458127975463867,
	"learning_rate": 1.5913773934790062e-06,
	"loss": 0.6226613044738769,
	"step": 3510
	},
	{
	"epoch": 1.2790697674418605,
	"grad_norm": 2.39019513130188,
	"learning_rate": 1.577398320771344e-06,
	"loss": 0.6492767333984375,
	"step": 3520
	},
	{
	"epoch": 1.2827034883720931,
	"grad_norm": 2.4112114906311035,
	"learning_rate": 1.5634525631943844e-06,
	"loss": 0.6178762912750244,
	"step": 3530
	},
	{
	"epoch": 1.2863372093023255,
	"grad_norm": 2.499526262283325,
	"learning_rate": 1.5495406243291925e-06,
	"loss": 0.6175832748413086,
	"step": 3540
	},
	{
	"epoch": 1.2899709302325582,
	"grad_norm": 2.208752155303955,
	"learning_rate": 1.5356630065356395e-06,
	"loss": 0.639375114440918,
	"step": 3550
	},
	{
	"epoch": 1.2936046511627908,
	"grad_norm": 2.570621967315674,
	"learning_rate": 1.5218202109342655e-06,
	"loss": 0.5913646697998047,
	"step": 3560
	},
	{
	"epoch": 1.2972383720930232,
	"grad_norm": 4.118480205535889,
	"learning_rate": 1.508012737388181e-06,
	"loss": 0.5979381561279297,
	"step": 3570
	},
	{
	"epoch": 1.3008720930232558,
	"grad_norm": 2.428710699081421,
	"learning_rate": 1.4942410844850144e-06,
	"loss": 0.6250282287597656,
	"step": 3580
	},
	{
	"epoch": 1.3045058139534884,
	"grad_norm": 2.431718111038208,
	"learning_rate": 1.4805057495189178e-06,
	"loss": 0.6236115455627441,
	"step": 3590
	},
	{
	"epoch": 1.308139534883721,
	"grad_norm": 2.667093515396118,
	"learning_rate": 1.4668072284725974e-06,
	"loss": 0.6282271385192871,
	"step": 3600
	},
	{
	"epoch": 1.3117732558139534,
	"grad_norm": 2.326963424682617,
	"learning_rate": 1.4531460159994125e-06,
	"loss": 0.6030844688415528,
	"step": 3610
	},
	{
	"epoch": 1.315406976744186,
	"grad_norm": 2.3418161869049072,
	"learning_rate": 1.4395226054055084e-06,
	"loss": 0.627055311203003,
	"step": 3620
	},
	{
	"epoch": 1.3190406976744187,
	"grad_norm": 2.4146857261657715,
	"learning_rate": 1.4259374886320064e-06,
	"loss": 0.621110200881958,
	"step": 3630
	},
	{
	"epoch": 1.322674418604651,
	"grad_norm": 4.005244255065918,
	"learning_rate": 1.4123911562372411e-06,
	"loss": 0.5961332321166992,
	"step": 3640
	},
	{
	"epoch": 1.3263081395348837,
	"grad_norm": 2.275883197784424,
	"learning_rate": 1.398884097379038e-06,
	"loss": 0.6394064903259278,
	"step": 3650
	},
	{
	"epoch": 1.3299418604651163,
	"grad_norm": 2.2539658546447754,
	"learning_rate": 1.3854167997970607e-06,
	"loss": 0.6367066383361817,
	"step": 3660
	},
	{
	"epoch": 1.333575581395349,
	"grad_norm": 2.4578914642333984,
	"learning_rate": 1.3719897497951934e-06,
	"loss": 0.6221925735473632,
	"step": 3670
	},
	{
	"epoch": 1.3372093023255813,
	"grad_norm": 2.477123975753784,
	"learning_rate": 1.3586034322239777e-06,
	"loss": 0.639109754562378,
	"step": 3680
	},
	{
	"epoch": 1.340843023255814,
	"grad_norm": 2.5295298099517822,
	"learning_rate": 1.3452583304631118e-06,
	"loss": 0.5931121826171875,
	"step": 3690
	},
	{
	"epoch": 1.3444767441860466,
	"grad_norm": 2.397085666656494,
	"learning_rate": 1.3319549264039874e-06,
	"loss": 0.609772777557373,
	"step": 3700
	},
	{
	"epoch": 1.348110465116279,
	"grad_norm": 2.383049726486206,
	"learning_rate": 1.318693700432298e-06,
	"loss": 0.631618881225586,
	"step": 3710
	},
	{
	"epoch": 1.3517441860465116,
	"grad_norm": 2.4240267276763916,
	"learning_rate": 1.305475131410682e-06,
	"loss": 0.6085801601409913,
	"step": 3720
	},
	{
	"epoch": 1.3553779069767442,
	"grad_norm": 2.3766026496887207,
	"learning_rate": 1.2922996966614407e-06,
	"loss": 0.6195814132690429,
	"step": 3730
	},
	{
	"epoch": 1.3590116279069768,
	"grad_norm": 2.2502548694610596,
	"learning_rate": 1.2791678719492931e-06,
	"loss": 0.6285619735717773,
	"step": 3740
	},
	{
	"epoch": 1.3626453488372092,
	"grad_norm": 2.6200804710388184,
	"learning_rate": 1.2660801314642038e-06,
	"loss": 0.6259998321533203,
	"step": 3750
	},
	{
	"epoch": 1.3662790697674418,
	"grad_norm": 2.4135735034942627,
	"learning_rate": 1.2530369478042567e-06,
	"loss": 0.622979736328125,
	"step": 3760
	},
	{
	"epoch": 1.3699127906976745,
	"grad_norm": 2.509988784790039,
	"learning_rate": 1.2400387919585876e-06,
	"loss": 0.6295654296875,
	"step": 3770
	},
	{
	"epoch": 1.3735465116279069,
	"grad_norm": 2.4443678855895996,
	"learning_rate": 1.2270861332903774e-06,
	"loss": 0.6230918884277343,
	"step": 3780
	},
	{
	"epoch": 1.3771802325581395,
	"grad_norm": 2.2816250324249268,
	"learning_rate": 1.2141794395199112e-06,
	"loss": 0.609403419494629,
	"step": 3790
	},
	{
	"epoch": 1.380813953488372,
	"grad_norm": 2.3419740200042725,
	"learning_rate": 1.2013191767076756e-06,
	"loss": 0.6306521415710449,
	"step": 3800
	},
	{
	"epoch": 1.3844476744186047,
	"grad_norm": 2.3549256324768066,
	"learning_rate": 1.1885058092375385e-06,
	"loss": 0.6321505546569824,
	"step": 3810
	},
	{
	"epoch": 1.3880813953488373,
	"grad_norm": 2.384006977081299,
	"learning_rate": 1.1757397997999778e-06,
	"loss": 0.6087956428527832,
	"step": 3820
	},
	{
	"epoch": 1.3917151162790697,
	"grad_norm": 2.491466522216797,
	"learning_rate": 1.1630216093753756e-06,
	"loss": 0.6140663623809814,
	"step": 3830
	},
	{
	"epoch": 1.3953488372093024,
	"grad_norm": 2.4473798274993896,
	"learning_rate": 1.1503516972173675e-06,
	"loss": 0.6443946361541748,
	"step": 3840
	},
	{
	"epoch": 1.3989825581395348,
	"grad_norm": 2.3487229347229004,
	"learning_rate": 1.1377305208362613e-06,
	"loss": 0.600891399383545,
	"step": 3850
	},
	{
	"epoch": 1.4026162790697674,
	"grad_norm": 2.3079259395599365,
	"learning_rate": 1.1251585359825193e-06,
	"loss": 0.6274770736694336,
	"step": 3860
	},
	{
	"epoch": 1.40625,
	"grad_norm": 2.477717876434326,
	"learning_rate": 1.1126361966302983e-06,
	"loss": 0.6162134170532226,
	"step": 3870
	},
	{
	"epoch": 1.4098837209302326,
	"grad_norm": 2.4388327598571777,
	"learning_rate": 1.1001639549610546e-06,
	"loss": 0.6384091377258301,
	"step": 3880
	},
	{
	"epoch": 1.4135174418604652,
	"grad_norm": 2.4514715671539307,
	"learning_rate": 1.0877422613472196e-06,
	"loss": 0.5987993717193604,
	"step": 3890
	},
	{
	"epoch": 1.4171511627906976,
	"grad_norm": 2.4952847957611084,
	"learning_rate": 1.075371564335937e-06,
	"loss": 0.6450319290161133,
	"step": 3900
	},
	{
	"epoch": 1.4207848837209303,
	"grad_norm": 2.2558183670043945,
	"learning_rate": 1.0630523106328605e-06,
	"loss": 0.6321861267089843,
	"step": 3910
	},
	{
	"epoch": 1.4244186046511627,
	"grad_norm": 2.474931478500366,
	"learning_rate": 1.0507849450860277e-06,
	"loss": 0.6270703315734864,
	"step": 3920
	},
	{
	"epoch": 1.4280523255813953,
	"grad_norm": 2.485100269317627,
	"learning_rate": 1.0385699106697983e-06,
	"loss": 0.617896842956543,
	"step": 3930
	},
	{
	"epoch": 1.431686046511628,
	"grad_norm": 2.4523518085479736,
	"learning_rate": 1.026407648468854e-06,
	"loss": 0.6216073989868164,
	"step": 3940
	},
	{
	"epoch": 1.4353197674418605,
	"grad_norm": 2.2941200733184814,
	"learning_rate": 1.0142985976622718e-06,
	"loss": 0.6087827205657959,
	"step": 3950
	},
	{
	"epoch": 1.4389534883720931,
	"grad_norm": 2.665576219558716,
	"learning_rate": 1.0022431955076686e-06,
	"loss": 0.6322806358337403,
	"step": 3960
	},
	{
	"epoch": 1.4425872093023255,
	"grad_norm": 2.5570075511932373,
	"learning_rate": 9.902418773254064e-07,
	"loss": 0.6422297477722168,
	"step": 3970
	},
	{
	"epoch": 1.4462209302325582,
	"grad_norm": 2.2849698066711426,
	"learning_rate": 9.782950764828788e-07,
	"loss": 0.6348532676696778,
	"step": 3980
	},
	{
	"epoch": 1.4498546511627908,
	"grad_norm": 2.311509609222412,
	"learning_rate": 9.664032243788565e-07,
	"loss": 0.618861198425293,
	"step": 3990
	},
	{
	"epoch": 1.4534883720930232,
	"grad_norm": 2.485456705093384,
	"learning_rate": 9.545667504279147e-07,
	"loss": 0.6145537376403809,
	"step": 4000
	},
	{
	"epoch": 1.4571220930232558,
	"grad_norm": 2.4726078510284424,
	"learning_rate": 9.427860820449236e-07,
	"loss": 0.6032392501831054,
	"step": 4010
	},
	{
	"epoch": 1.4607558139534884,
	"grad_norm": 2.5328330993652344,
	"learning_rate": 9.310616446296139e-07,
	"loss": 0.6275752544403076,
	"step": 4020
	},
	{
	"epoch": 1.464389534883721,
	"grad_norm": 2.358928918838501,
	"learning_rate": 9.193938615512193e-07,
	"loss": 0.6322643280029296,
	"step": 4030
	},
	{
	"epoch": 1.4680232558139534,
	"grad_norm": 2.4950027465820312,
	"learning_rate": 9.077831541331842e-07,
	"loss": 0.6126497268676758,
	"step": 4040
	},
	{
	"epoch": 1.471656976744186,
	"grad_norm": 2.3938448429107666,
	"learning_rate": 8.962299416379533e-07,
	"loss": 0.6140788078308106,
	"step": 4050
	},
	{
	"epoch": 1.4752906976744187,
	"grad_norm": 2.461545467376709,
	"learning_rate": 8.847346412518312e-07,
	"loss": 0.6246371269226074,
	"step": 4060
	},
	{
	"epoch": 1.478924418604651,
	"grad_norm": 2.4481334686279297,
	"learning_rate": 8.732976680699153e-07,
	"loss": 0.6154314994812011,
	"step": 4070
	},
	{
	"epoch": 1.4825581395348837,
	"grad_norm": 2.342137336730957,
	"learning_rate": 8.619194350811094e-07,
	"loss": 0.6052567005157471,
	"step": 4080
	},
	{
	"epoch": 1.4861918604651163,
	"grad_norm": 2.5140323638916016,
	"learning_rate": 8.506003531532103e-07,
	"loss": 0.609114122390747,
	"step": 4090
	},
	{
	"epoch": 1.489825581395349,
	"grad_norm": 2.3659894466400146,
	"learning_rate": 8.393408310180722e-07,
	"loss": 0.6255186080932618,
	"step": 4100
	},
	{
	"epoch": 1.4934593023255813,
	"grad_norm": 2.436582326889038,
	"learning_rate": 8.281412752568435e-07,
	"loss": 0.6080513000488281,
	"step": 4110
	},
	{
	"epoch": 1.497093023255814,
	"grad_norm": 2.4563467502593994,
	"learning_rate": 8.170020902852896e-07,
	"loss": 0.6291277885437012,
	"step": 4120
	},
	{
	"epoch": 1.5007267441860463,
	"grad_norm": 2.4873809814453125,
	"learning_rate": 8.059236783391885e-07,
	"loss": 0.616195821762085,
	"step": 4130
	},
	{
	"epoch": 1.504360465116279,
	"grad_norm": 2.308582067489624,
	"learning_rate": 7.949064394598033e-07,
	"loss": 0.6022687435150147,
	"step": 4140
	},
	{
	"epoch": 1.5079941860465116,
	"grad_norm": 2.5051050186157227,
	"learning_rate": 7.839507714794389e-07,
	"loss": 0.5941274166107178,
	"step": 4150
	},
	{
	"epoch": 1.5116279069767442,
	"grad_norm": 2.2698898315429688,
	"learning_rate": 7.73057070007078e-07,
	"loss": 0.6412195205688477,
	"step": 4160
	},
	{
	"epoch": 1.5152616279069768,
	"grad_norm": 2.3436214923858643,
	"learning_rate": 7.622257284140927e-07,
	"loss": 0.6041569709777832,
	"step": 4170
	},
	{
	"epoch": 1.5188953488372094,
	"grad_norm": 2.3799853324890137,
	"learning_rate": 7.514571378200408e-07,
	"loss": 0.6206464767456055,
	"step": 4180
	},
	{
	"epoch": 1.5225290697674418,
	"grad_norm": 2.434307813644409,
	"learning_rate": 7.407516870785405e-07,
	"loss": 0.6407901763916015,
	"step": 4190
	},
	{
	"epoch": 1.5261627906976745,
	"grad_norm": 2.472949743270874,
	"learning_rate": 7.301097627632367e-07,
	"loss": 0.6143634796142579,
	"step": 4200
	},
	{
	"epoch": 1.5297965116279069,
	"grad_norm": 2.4636948108673096,
	"learning_rate": 7.195317491538308e-07,
	"loss": 0.5821886539459229,
	"step": 4210
	},
	{
	"epoch": 1.5334302325581395,
	"grad_norm": 2.294044256210327,
	"learning_rate": 7.090180282222109e-07,
	"loss": 0.6156596183776856,
	"step": 4220
	},
	{
	"epoch": 1.537063953488372,
	"grad_norm": 2.440505027770996,
	"learning_rate": 6.985689796186593e-07,
	"loss": 0.6063512802124024,
	"step": 4230
	},
	{
	"epoch": 1.5406976744186047,
	"grad_norm": 2.4071571826934814,
	"learning_rate": 6.881849806581405e-07,
	"loss": 0.5945966243743896,
	"step": 4240
	},
	{
	"epoch": 1.5443313953488373,
	"grad_norm": 2.3198697566986084,
	"learning_rate": 6.778664063066773e-07,
	"loss": 0.600504207611084,
	"step": 4250
	},
	{
	"epoch": 1.5479651162790697,
	"grad_norm": 2.531416654586792,
	"learning_rate": 6.676136291678118e-07,
	"loss": 0.5918593406677246,
	"step": 4260
	},
	{
	"epoch": 1.5515988372093024,
	"grad_norm": 2.509495973587036,
	"learning_rate": 6.574270194691485e-07,
	"loss": 0.6285495281219482,
	"step": 4270
	},
	{
	"epoch": 1.5552325581395348,
	"grad_norm": 2.3980696201324463,
	"learning_rate": 6.473069450489888e-07,
	"loss": 0.6094050884246827,
	"step": 4280
	},
	{
	"epoch": 1.5588662790697674,
	"grad_norm": 2.5676488876342773,
	"learning_rate": 6.37253771343044e-07,
	"loss": 0.6188784599304199,
	"step": 4290
	},
	{
	"epoch": 1.5625,
	"grad_norm": 2.501662015914917,
	"learning_rate": 6.272678613712437e-07,
	"loss": 0.6202298164367676,
	"step": 4300
	},
	{
	"epoch": 1.5661337209302326,
	"grad_norm": 2.583395004272461,
	"learning_rate": 6.173495757246251e-07,
	"loss": 0.606217336654663,
	"step": 4310
	},
	{
	"epoch": 1.5697674418604652,
	"grad_norm": 2.364949941635132,
	"learning_rate": 6.074992725523102e-07,
	"loss": 0.6397464752197266,
	"step": 4320
	},
	{
	"epoch": 1.5734011627906976,
	"grad_norm": 2.5047953128814697,
	"learning_rate": 5.977173075485778e-07,
	"loss": 0.5977835178375244,
	"step": 4330
	},
	{
	"epoch": 1.5770348837209303,
	"grad_norm": 2.3925271034240723,
	"learning_rate": 5.880040339400142e-07,
	"loss": 0.6198395729064942,
	"step": 4340
	},
	{
	"epoch": 1.5806686046511627,
	"grad_norm": 2.5963175296783447,
	"learning_rate": 5.783598024727618e-07,
	"loss": 0.6271990299224853,
	"step": 4350
	},
	{
	"epoch": 1.5843023255813953,
	"grad_norm": 2.498198986053467,
	"learning_rate": 5.687849613998534e-07,
	"loss": 0.6185245037078857,
	"step": 4360
	},
	{
	"epoch": 1.587936046511628,
	"grad_norm": 2.4176995754241943,
	"learning_rate": 5.592798564686339e-07,
	"loss": 0.625272560119629,
	"step": 4370
	},
	{
	"epoch": 1.5915697674418605,
	"grad_norm": 2.509249687194824,
	"learning_rate": 5.498448309082777e-07,
	"loss": 0.6272983551025391,
	"step": 4380
	},
	{
	"epoch": 1.5952034883720931,
	"grad_norm": 2.402564287185669,
	"learning_rate": 5.40480225417395e-07,
	"loss": 0.6388293266296386,
	"step": 4390
	},
	{
	"epoch": 1.5988372093023255,
	"grad_norm": 2.4651103019714355,
	"learning_rate": 5.311863781517296e-07,
	"loss": 0.5957953453063964,
	"step": 4400
	},
	{
	"epoch": 1.6024709302325582,
	"grad_norm": 2.329270362854004,
	"learning_rate": 5.219636247119448e-07,
	"loss": 0.6173959732055664,
	"step": 4410
	},
	{
	"epoch": 1.6061046511627906,
	"grad_norm": 2.3564770221710205,
	"learning_rate": 5.128122981315068e-07,
	"loss": 0.6107733726501465,
	"step": 4420
	},
	{
	"epoch": 1.6097383720930232,
	"grad_norm": 2.292123556137085,
	"learning_rate": 5.03732728864663e-07,
	"loss": 0.6385003089904785,
	"step": 4430
	},
	{
	"epoch": 1.6133720930232558,
	"grad_norm": 2.306692600250244,
	"learning_rate": 4.947252447745013e-07,
	"loss": 0.6241935729980469,
	"step": 4440
	},
	{
	"epoch": 1.6170058139534884,
	"grad_norm": 2.324307680130005,
	"learning_rate": 4.857901711211161e-07,
	"loss": 0.6390767097473145,
	"step": 4450
	},
	{
	"epoch": 1.620639534883721,
	"grad_norm": 2.299142360687256,
	"learning_rate": 4.769278305498632e-07,
	"loss": 0.6130090236663819,
	"step": 4460
	},
	{
	"epoch": 1.6242732558139537,
	"grad_norm": 2.4672698974609375,
	"learning_rate": 4.681385430797086e-07,
	"loss": 0.6364339828491211,
	"step": 4470
	},
	{
	"epoch": 1.627906976744186,
	"grad_norm": 2.2892332077026367,
	"learning_rate": 4.594226260916701e-07,
	"loss": 0.6140618324279785,
	"step": 4480
	},
	{
	"epoch": 1.6315406976744184,
	"grad_norm": 2.3146252632141113,
	"learning_rate": 4.507803943173592e-07,
	"loss": 0.6106718063354493,
	"step": 4490
	},
	{
	"epoch": 1.635174418604651,
	"grad_norm": 2.391108751296997,
	"learning_rate": 4.422121598276161e-07,
	"loss": 0.620957899093628,
	"step": 4500
	},
	{
	"epoch": 1.6388081395348837,
	"grad_norm": 2.462740659713745,
	"learning_rate": 4.3371823202124156e-07,
	"loss": 0.607684326171875,
	"step": 4510
	},
	{
	"epoch": 1.6424418604651163,
	"grad_norm": 2.3729045391082764,
	"learning_rate": 4.2529891761382e-07,
	"loss": 0.599039363861084,
	"step": 4520
	},
	{
	"epoch": 1.646075581395349,
	"grad_norm": 2.4546728134155273,
	"learning_rate": 4.1695452062665067e-07,
	"loss": 0.6334895610809326,
	"step": 4530
	},
	{
	"epoch": 1.6497093023255816,
	"grad_norm": 2.4358999729156494,
	"learning_rate": 4.0868534237576476e-07,
	"loss": 0.6073498725891113,
	"step": 4540
	},
	{
	"epoch": 1.653343023255814,
	"grad_norm": 2.365082263946533,
	"learning_rate": 4.0049168146104487e-07,
	"loss": 0.6107266902923584,
	"step": 4550
	},
	{
	"epoch": 1.6569767441860463,
	"grad_norm": 2.5459234714508057,
	"learning_rate": 3.923738337554467e-07,
	"loss": 0.6061570167541503,
	"step": 4560
	},
	{
	"epoch": 1.660610465116279,
	"grad_norm": 2.8649494647979736,
	"learning_rate": 3.843320923943089e-07,
	"loss": 0.6015692710876465,
	"step": 4570
	},
	{
	"epoch": 1.6642441860465116,
	"grad_norm": 2.256239891052246,
	"learning_rate": 3.7636674776477507e-07,
	"loss": 0.6040020942687988,
	"step": 4580
	},
	{
	"epoch": 1.6678779069767442,
	"grad_norm": 2.3947911262512207,
	"learning_rate": 3.6847808749530005e-07,
	"loss": 0.6294768333435059,
	"step": 4590
	},
	{
	"epoch": 1.6715116279069768,
	"grad_norm": 2.424443244934082,
	"learning_rate": 3.6066639644527195e-07,
	"loss": 0.6310386180877685,
	"step": 4600
	},
	{
	"epoch": 1.6751453488372094,
	"grad_norm": 2.4617273807525635,
	"learning_rate": 3.529319566947184e-07,
	"loss": 0.6166575431823731,
	"step": 4610
	},
	{
	"epoch": 1.6787790697674418,
	"grad_norm": 2.3722071647644043,
	"learning_rate": 3.4527504753412517e-07,
	"loss": 0.6409936428070069,
	"step": 4620
	},
	{
	"epoch": 1.6824127906976745,
	"grad_norm": 2.332188844680786,
	"learning_rate": 3.3769594545435103e-07,
	"loss": 0.6028295040130616,
	"step": 4630
	},
	{
	"epoch": 1.6860465116279069,
	"grad_norm": 2.485243082046509,
	"learning_rate": 3.301949241366398e-07,
	"loss": 0.5945775032043457,
	"step": 4640
	},
	{
	"epoch": 1.6896802325581395,
	"grad_norm": 2.476738214492798,
	"learning_rate": 3.227722544427428e-07,
	"loss": 0.5858177185058594,
	"step": 4650
	},
	{
	"epoch": 1.693313953488372,
	"grad_norm": 2.422708511352539,
	"learning_rate": 3.154282044051335e-07,
	"loss": 0.5783253192901612,
	"step": 4660
	},
	{
	"epoch": 1.6969476744186047,
	"grad_norm": 2.4745054244995117,
	"learning_rate": 3.081630392173329e-07,
	"loss": 0.6289084911346435,
	"step": 4670
	},
	{
	"epoch": 1.7005813953488373,
	"grad_norm": 2.678463935852051,
	"learning_rate": 3.009770212243293e-07,
	"loss": 0.6344997406005859,
	"step": 4680
	},
	{
	"epoch": 1.7042151162790697,
	"grad_norm": 2.6704978942871094,
	"learning_rate": 2.93870409913109e-07,
	"loss": 0.6260345935821533,
	"step": 4690
	},
	{
	"epoch": 1.7078488372093024,
	"grad_norm": 2.3599631786346436,
	"learning_rate": 2.8684346190328393e-07,
	"loss": 0.6227920055389404,
	"step": 4700
	},
	{
	"epoch": 1.7114825581395348,
	"grad_norm": 2.3389649391174316,
	"learning_rate": 2.798964309378255e-07,
	"loss": 0.6140339851379395,
	"step": 4710
	},
	{
	"epoch": 1.7151162790697674,
	"grad_norm": 2.3903820514678955,
	"learning_rate": 2.7302956787390006e-07,
	"loss": 0.6161631107330322,
	"step": 4720
	},
	{
	"epoch": 1.71875,
	"grad_norm": 2.3136348724365234,
	"learning_rate": 2.662431206738167e-07,
	"loss": 0.6303337097167969,
	"step": 4730
	},
	{
	"epoch": 1.7223837209302326,
	"grad_norm": 2.365283966064453,
	"learning_rate": 2.595373343960658e-07,
	"loss": 0.6121698379516601,
	"step": 4740
	},
	{
	"epoch": 1.7260174418604652,
	"grad_norm": 2.4423670768737793,
	"learning_rate": 2.5291245118647304e-07,
	"loss": 0.6158035278320313,
	"step": 4750
	},
	{
	"epoch": 1.7296511627906976,
	"grad_norm": 2.5949673652648926,
	"learning_rate": 2.4636871026945757e-07,
	"loss": 0.5731970787048339,
	"step": 4760
	},
	{
	"epoch": 1.7332848837209303,
	"grad_norm": 2.4564154148101807,
	"learning_rate": 2.399063479393912e-07,
	"loss": 0.6520524501800538,
	"step": 4770
	},
	{
	"epoch": 1.7369186046511627,
	"grad_norm": 2.3764851093292236,
	"learning_rate": 2.335255975520656e-07,
	"loss": 0.6177213191986084,
	"step": 4780
	},
	{
	"epoch": 1.7405523255813953,
	"grad_norm": 2.363431930541992,
	"learning_rate": 2.272266895162664e-07,
	"loss": 0.6088789939880371,
	"step": 4790
	},
	{
	"epoch": 1.744186046511628,
	"grad_norm": 2.4964349269866943,
	"learning_rate": 2.2100985128545498e-07,
	"loss": 0.6220262527465821,
	"step": 4800
	},
	{
	"epoch": 1.7478197674418605,
	"grad_norm": 2.278346538543701,
	"learning_rate": 2.1487530734955242e-07,
	"loss": 0.6197375297546387,
	"step": 4810
	},
	{
	"epoch": 1.7514534883720931,
	"grad_norm": 2.4007949829101562,
	"learning_rate": 2.0882327922683354e-07,
	"loss": 0.5987682819366456,
	"step": 4820
	},
	{
	"epoch": 1.7550872093023255,
	"grad_norm": 2.465545415878296,
	"learning_rate": 2.0285398545592999e-07,
	"loss": 0.6542253494262695,
	"step": 4830
	},
	{
	"epoch": 1.7587209302325582,
	"grad_norm": 2.3019957542419434,
	"learning_rate": 1.969676415879368e-07,
	"loss": 0.6183276653289795,
	"step": 4840
	},
	{
	"epoch": 1.7623546511627906,
	"grad_norm": 2.345837354660034,
	"learning_rate": 1.9116446017862917e-07,
	"loss": 0.6046629905700683,
	"step": 4850
	},
	{
	"epoch": 1.7659883720930232,
	"grad_norm": 2.3908026218414307,
	"learning_rate": 1.8544465078078693e-07,
	"loss": 0.62733154296875,
	"step": 4860
	},
	{
	"epoch": 1.7696220930232558,
	"grad_norm": 2.5572052001953125,
	"learning_rate": 1.798084199366293e-07,
	"loss": 0.5897133827209473,
	"step": 4870
	},
	{
	"epoch": 1.7732558139534884,
	"grad_norm": 2.3685665130615234,
	"learning_rate": 1.7425597117035537e-07,
	"loss": 0.6033139228820801,
	"step": 4880
	},
	{
	"epoch": 1.776889534883721,
	"grad_norm": 2.377469778060913,
	"learning_rate": 1.6878750498079273e-07,
	"loss": 0.60375075340271,
	"step": 4890
	},
	{
	"epoch": 1.7805232558139537,
	"grad_norm": 2.524209976196289,
	"learning_rate": 1.6340321883416222e-07,
	"loss": 0.6266297817230224,
	"step": 4900
	},
	{
	"epoch": 1.784156976744186,
	"grad_norm": 2.5692672729492188,
	"learning_rate": 1.5810330715694266e-07,
	"loss": 0.5777091979980469,
	"step": 4910
	},
	{
	"epoch": 1.7877906976744184,
	"grad_norm": 2.2759571075439453,
	"learning_rate": 1.5288796132885393e-07,
	"loss": 0.6004066944122315,
	"step": 4920
	},
	{
	"epoch": 1.791424418604651,
	"grad_norm": 2.351780652999878,
	"learning_rate": 1.4775736967594379e-07,
	"loss": 0.5945886611938477,
	"step": 4930
	},
	{
	"epoch": 1.7950581395348837,
	"grad_norm": 2.4352078437805176,
	"learning_rate": 1.4271171746378764e-07,
	"loss": 0.6205580711364747,
	"step": 4940
	},
	{
	"epoch": 1.7986918604651163,
	"grad_norm": 2.3724617958068848,
	"learning_rate": 1.3775118689080048e-07,
	"loss": 0.6045650482177735,
	"step": 4950
	},
	{
	"epoch": 1.802325581395349,
	"grad_norm": 2.3964314460754395,
	"learning_rate": 1.3287595708165457e-07,
	"loss": 0.6370014190673828,
	"step": 4960
	},
	{
	"epoch": 1.8059593023255816,
	"grad_norm": 2.438803195953369,
	"learning_rate": 1.2808620408081536e-07,
	"loss": 0.6139168739318848,
	"step": 4970
	},
	{
	"epoch": 1.809593023255814,
	"grad_norm": 2.499262809753418,
	"learning_rate": 1.2338210084617996e-07,
	"loss": 0.6294375896453858,
	"step": 4980
	},
	{
	"epoch": 1.8132267441860463,
	"grad_norm": 2.547861337661743,
	"learning_rate": 1.1876381724283515e-07,
	"loss": 0.6226459503173828,
	"step": 4990
	},
	{
	"epoch": 1.816860465116279,
	"grad_norm": 2.4133615493774414,
	"learning_rate": 1.142315200369229e-07,
	"loss": 0.6258872032165528,
	"step": 5000
	},
	{
	"epoch": 1.8204941860465116,
	"grad_norm": 2.4785385131835938,
	"learning_rate": 1.0978537288961616e-07,
	"loss": 0.5929446220397949,
	"step": 5010
	},
	{
	"epoch": 1.8241279069767442,
	"grad_norm": 2.5542328357696533,
	"learning_rate": 1.0542553635121134e-07,
	"loss": 0.6152782440185547,
	"step": 5020
	},
	{
	"epoch": 1.8277616279069768,
	"grad_norm": 2.50476336479187,
	"learning_rate": 1.0115216785533149e-07,
	"loss": 0.6148166656494141,
	"step": 5030
	},
	{
	"epoch": 1.8313953488372094,
	"grad_norm": 2.236177444458008,
	"learning_rate": 9.696542171323902e-08,
	"loss": 0.6066837310791016,
	"step": 5040
	},
	{
	"epoch": 1.8350290697674418,
	"grad_norm": 2.636986017227173,
	"learning_rate": 9.286544910826395e-08,
	"loss": 0.6134783267974854,
	"step": 5050
	},
	{
	"epoch": 1.8386627906976745,
	"grad_norm": 2.5881102085113525,
	"learning_rate": 8.885239809034718e-08,
	"loss": 0.6172745704650879,
	"step": 5060
	},
	{
	"epoch": 1.8422965116279069,
	"grad_norm": 2.4413042068481445,
	"learning_rate": 8.492641357069148e-08,
	"loss": 0.5996236324310302,
	"step": 5070
	},
	{
	"epoch": 1.8459302325581395,
	"grad_norm": 2.3867828845977783,
	"learning_rate": 8.10876373165298e-08,
	"loss": 0.6087081432342529,
	"step": 5080
	},
	{
	"epoch": 1.849563953488372,
	"grad_norm": 2.477936267852783,
	"learning_rate": 7.733620794600604e-08,
	"loss": 0.6194024562835694,
	"step": 5090
	},
	{
	"epoch": 1.8531976744186047,
	"grad_norm": 2.5035672187805176,
	"learning_rate": 7.367226092317076e-08,
	"loss": 0.5913207054138183,
	"step": 5100
	},
	{
	"epoch": 1.8568313953488373,
	"grad_norm": 2.389775037765503,
	"learning_rate": 7.009592855308722e-08,
	"loss": 0.6251027107238769,
	"step": 5110
	},
	{
	"epoch": 1.8604651162790697,
	"grad_norm": 2.504149913787842,
	"learning_rate": 6.66073399770556e-08,
	"loss": 0.6004135131835937,
	"step": 5120
	},
	{
	"epoch": 1.8640988372093024,
	"grad_norm": 2.542309284210205,
	"learning_rate": 6.320662116794912e-08,
	"loss": 0.6172690868377686,
	"step": 5130
	},
	{
	"epoch": 1.8677325581395348,
	"grad_norm": 2.4836597442626953,
	"learning_rate": 5.989389492566527e-08,
	"loss": 0.6046713829040528,
	"step": 5140
	},
	{
	"epoch": 1.8713662790697674,
	"grad_norm": 2.4051828384399414,
	"learning_rate": 5.66692808726918e-08,
	"loss": 0.6101225852966309,
	"step": 5150
	},
	{
	"epoch": 1.875,
	"grad_norm": 2.442782163619995,
	"learning_rate": 5.3532895449785735e-08,
	"loss": 0.6214385986328125,
	"step": 5160
	},
	{
	"epoch": 1.8786337209302326,
	"grad_norm": 2.393648386001587,
	"learning_rate": 5.0484851911770664e-08,
	"loss": 0.5991032600402832,
	"step": 5170
	},
	{
	"epoch": 1.8822674418604652,
	"grad_norm": 2.3515219688415527,
	"learning_rate": 4.7525260323446897e-08,
	"loss": 0.5910218238830567,
	"step": 5180
	},
	{
	"epoch": 1.8859011627906976,
	"grad_norm": 2.3698923587799072,
	"learning_rate": 4.4654227555614716e-08,
	"loss": 0.600614356994629,
	"step": 5190
	},
	{
	"epoch": 1.8895348837209303,
	"grad_norm": 2.3536767959594727,
	"learning_rate": 4.187185728121823e-08,
	"loss": 0.5952644348144531,
	"step": 5200
	},
	{
	"epoch": 1.8931686046511627,
	"grad_norm": 2.4382128715515137,
	"learning_rate": 3.9178249971599523e-08,
	"loss": 0.6196311950683594,
	"step": 5210
	},
	{
	"epoch": 1.8968023255813953,
	"grad_norm": 2.2585620880126953,
	"learning_rate": 3.657350289287265e-08,
	"loss": 0.6128178596496582,
	"step": 5220
	},
	{
	"epoch": 1.900436046511628,
	"grad_norm": 2.4336864948272705,
	"learning_rate": 3.4057710102409215e-08,
	"loss": 0.605217981338501,
	"step": 5230
	},
	{
	"epoch": 1.9040697674418605,
	"grad_norm": 2.490455150604248,
	"learning_rate": 3.163096244544361e-08,
	"loss": 0.6238677024841308,
	"step": 5240
	},
	{
	"epoch": 1.9077034883720931,
	"grad_norm": 2.308756113052368,
	"learning_rate": 2.929334755179175e-08,
	"loss": 0.6050383567810058,
	"step": 5250
	},
	{
	"epoch": 1.9113372093023255,
	"grad_norm": 2.29961895942688,
	"learning_rate": 2.7044949832686908e-08,
	"loss": 0.6162363529205322,
	"step": 5260
	},
	{
	"epoch": 1.9149709302325582,
	"grad_norm": 2.4525465965270996,
	"learning_rate": 2.4885850477732178e-08,
	"loss": 0.6142619609832763,
	"step": 5270
	},
	{
	"epoch": 1.9186046511627906,
	"grad_norm": 2.365469455718994,
	"learning_rate": 2.281612745196782e-08,
	"loss": 0.6296330451965332,
	"step": 5280
	},
	{
	"epoch": 1.9222383720930232,
	"grad_norm": 2.580822706222534,
	"learning_rate": 2.083585549305628e-08,
	"loss": 0.5965895652770996,
	"step": 5290
	},
	{
	"epoch": 1.9258720930232558,
	"grad_norm": 2.6384425163269043,
	"learning_rate": 1.8945106108584634e-08,
	"loss": 0.6235435485839844,
	"step": 5300
	},
	{
	"epoch": 1.9295058139534884,
	"grad_norm": 2.9369654655456543,
	"learning_rate": 1.7143947573479704e-08,
	"loss": 0.6292021751403809,
	"step": 5310
	},
	{
	"epoch": 1.933139534883721,
	"grad_norm": 2.353226900100708,
	"learning_rate": 1.543244492754531e-08,
	"loss": 0.6019412517547608,
	"step": 5320
	},
	{
	"epoch": 1.9367732558139537,
	"grad_norm": 2.441413402557373,
	"learning_rate": 1.3810659973112206e-08,
	"loss": 0.6056236267089844,
	"step": 5330
	},
	{
	"epoch": 1.940406976744186,
	"grad_norm": 2.224850654602051,
	"learning_rate": 1.227865127280764e-08,
	"loss": 0.6034550666809082,
	"step": 5340
	},
	{
	"epoch": 1.9440406976744184,
	"grad_norm": 2.3518519401550293,
	"learning_rate": 1.0836474147438991e-08,
	"loss": 0.6212761878967286,
	"step": 5350
	},
	{
	"epoch": 1.947674418604651,
	"grad_norm": 2.3524904251098633,
	"learning_rate": 9.484180673997312e-09,
	"loss": 0.6164274215698242,
	"step": 5360
	},
	{
	"epoch": 1.9513081395348837,
	"grad_norm": 2.451040267944336,
	"learning_rate": 8.22181968377661e-09,
	"loss": 0.6036581993103027,
	"step": 5370
	},
	{
	"epoch": 1.9549418604651163,
	"grad_norm": 2.387059450149536,
	"learning_rate": 7.0494367606105396e-09,
	"loss": 0.6041526794433594,
	"step": 5380
	},
	{
	"epoch": 1.958575581395349,
	"grad_norm": 2.419077157974243,
	"learning_rate": 5.967074239226212e-09,
	"loss": 0.6136129856109619,
	"step": 5390
	},
	{
	"epoch": 1.9622093023255816,
	"grad_norm": 2.4791665077209473,
	"learning_rate": 4.974771203715145e-09,
	"loss": 0.6109216690063477,
	"step": 5400
	},
	{
	"epoch": 1.965843023255814,
	"grad_norm": 2.418416738510132,
	"learning_rate": 4.07256348612356e-09,
	"loss": 0.6217376708984375,
	"step": 5410
	},
	{
	"epoch": 1.9694767441860463,
	"grad_norm": 2.4548721313476562,
	"learning_rate": 3.260483665155911e-09,
	"loss": 0.5937500953674316,
	"step": 5420
	},
	{
	"epoch": 1.973110465116279,
	"grad_norm": 2.4309005737304688,
	"learning_rate": 2.538561065000833e-09,
	"loss": 0.6115482330322266,
	"step": 5430
	},
	{
	"epoch": 1.9767441860465116,
	"grad_norm": 2.4169089794158936,
	"learning_rate": 1.906821754270871e-09,
	"loss": 0.6274826049804687,
	"step": 5440
	},
	{
	"epoch": 1.9803779069767442,
	"grad_norm": 2.7668118476867676,
	"learning_rate": 1.365288545061849e-09,
	"loss": 0.6413361072540283,
	"step": 5450
	},
	{
	"epoch": 1.9840116279069768,
	"grad_norm": 2.582724094390869,
	"learning_rate": 9.139809921282494e-10,
	"loss": 0.5973198890686036,
	"step": 5460
	},
	{
	"epoch": 1.9876453488372094,
	"grad_norm": 2.540160655975342,
	"learning_rate": 5.529153921779439e-10,
	"loss": 0.6123784065246582,
	"step": 5470
	},
	{
	"epoch": 1.9912790697674418,
	"grad_norm": 2.4696357250213623,
	"learning_rate": 2.821047832837764e-10,
	"loss": 0.5966281890869141,
	"step": 5480
	},
	{
	"epoch": 1.9949127906976745,
	"grad_norm": 2.642086982727051,
	"learning_rate": 1.0155894441227221e-10,
	"loss": 0.6046372890472412,
	"step": 5490
	},
	{
	"epoch": 1.9985465116279069,
	"grad_norm": 2.455078601837158,
	"learning_rate": 1.1284395070310095e-11,
	"loss": 0.613033390045166,
	"step": 5500
	},
	{
	"epoch": 2.0,
	"step": 5504,
	"total_flos": 2.1250283267503948e+19,
	"train_loss": 0.7199515468159388,
	"train_runtime": 22331.278,
	"train_samples_per_second": 7.885,
	"train_steps_per_second": 0.246
	}
	],
	"logging_steps": 10,
	"max_steps": 5504,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 2,
	"save_steps": 500,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": true
	},
	"attributes": {}
	}
	},
	"total_flos": 2.1250283267503948e+19,
	"train_batch_size": 2,
	"trial_name": null,
	"trial_params": null
	}