Instructions to use cs-552-2026-flab/general_knowledge_model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cs-552-2026-flab/general_knowledge_model with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cs-552-2026-flab/general_knowledge_model")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("cs-552-2026-flab/general_knowledge_model")
model = AutoModelForCausalLM.from_pretrained("cs-552-2026-flab/general_knowledge_model", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use cs-552-2026-flab/general_knowledge_model with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cs-552-2026-flab/general_knowledge_model"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cs-552-2026-flab/general_knowledge_model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/cs-552-2026-flab/general_knowledge_model

SGLang

How to use cs-552-2026-flab/general_knowledge_model with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cs-552-2026-flab/general_knowledge_model" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cs-552-2026-flab/general_knowledge_model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cs-552-2026-flab/general_knowledge_model" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cs-552-2026-flab/general_knowledge_model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use cs-552-2026-flab/general_knowledge_model with Docker Model Runner:
```
docker model run hf.co/cs-552-2026-flab/general_knowledge_model
```

amir-ammar commited on May 27

Commit

d3bd402

verified ·

1 Parent(s): ff2c737

remove stale checkpoint-3005

Browse files

Files changed (11) hide show

checkpoint-3005/chat_template.jinja +0 -90
checkpoint-3005/config.json +0 -63
checkpoint-3005/generation_config.json +0 -12
checkpoint-3005/model.safetensors +0 -3
checkpoint-3005/optimizer.pt +0 -3
checkpoint-3005/rng_state.pth +0 -3
checkpoint-3005/scheduler.pt +0 -3
checkpoint-3005/tokenizer.json +0 -3
checkpoint-3005/tokenizer_config.json +0 -30
checkpoint-3005/trainer_state.json +0 -3034
checkpoint-3005/training_args.bin +0 -3

checkpoint-3005/chat_template.jinja DELETED Viewed

@@ -1,90 +0,0 @@
-{%- set enable_thinking = false %}
-{%- if tools %}
-    {{- '<|im_start|>system\n' }}
-    {%- if messages[0].role == 'system' %}
-        {{- messages[0].content + '\n\n' }}
-    {%- endif %}
-    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
-    {%- for tool in tools %}
-        {{- "\n" }}
-        {{- tool | tojson }}
-    {%- endfor %}
-    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
-{%- else %}
-    {%- if messages[0].role == 'system' %}
-        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
-    {%- endif %}
-{%- endif %}
-{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
-{%- for message in messages[::-1] %}
-    {%- set index = (messages|length - 1) - loop.index0 %}
-    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
-        {%- set ns.multi_step_tool = false %}
-        {%- set ns.last_query_index = index %}
-    {%- endif %}
-{%- endfor %}
-{%- for message in messages %}
-    {%- if message.content is string %}
-        {%- set content = message.content %}
-    {%- else %}
-        {%- set content = '' %}
-    {%- endif %}
-    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
-        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
-    {%- elif message.role == "assistant" %}
-        {%- set reasoning_content = '' %}
-        {%- if message.reasoning_content is string %}
-            {%- set reasoning_content = message.reasoning_content %}
-        {%- else %}
-            {%- if '</think>' in content %}
-                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
-                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
-            {%- endif %}
-        {%- endif %}
-        {%- if loop.index0 > ns.last_query_index %}
-            {%- if loop.last or (not loop.last and reasoning_content) %}
-                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
-            {%- else %}
-                {{- '<|im_start|>' + message.role + '\n' + content }}
-            {%- endif %}
-        {%- else %}
-            {{- '<|im_start|>' + message.role + '\n' + content }}
-        {%- endif %}
-        {%- if message.tool_calls %}
-            {%- for tool_call in message.tool_calls %}
-                {%- if (loop.first and content) or (not loop.first) %}
-                    {{- '\n' }}
-                {%- endif %}
-                {%- if tool_call.function %}
-                    {%- set tool_call = tool_call.function %}
-                {%- endif %}
-                {{- '<tool_call>\n{"name": "' }}
-                {{- tool_call.name }}
-                {{- '", "arguments": ' }}
-                {%- if tool_call.arguments is string %}
-                    {{- tool_call.arguments }}
-                {%- else %}
-                    {{- tool_call.arguments | tojson }}
-                {%- endif %}
-                {{- '}\n</tool_call>' }}
-            {%- endfor %}
-        {%- endif %}
-        {{- '<|im_end|>\n' }}
-    {%- elif message.role == "tool" %}
-        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
-            {{- '<|im_start|>user' }}
-        {%- endif %}
-        {{- '\n<tool_response>\n' }}
-        {{- content }}
-        {{- '\n</tool_response>' }}
-        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
-            {{- '<|im_end|>\n' }}
-        {%- endif %}
-    {%- endif %}
-{%- endfor %}
-{%- if add_generation_prompt %}
-    {{- '<|im_start|>assistant\n' }}
-    {%- if enable_thinking is defined and enable_thinking is false %}
-        {{- '<think>\n\n</think>\n\n' }}
-    {%- endif %}
-{%- endif %}

checkpoint-3005/config.json DELETED Viewed

@@ -1,63 +0,0 @@
-{
-  "architectures": [
-    "Qwen3ForCausalLM"
-  ],
-  "attention_bias": false,
-  "attention_dropout": 0.0,
-  "bos_token_id": null,
-  "dtype": "bfloat16",
-  "eos_token_id": 151645,
-  "head_dim": 128,
-  "hidden_act": "silu",
-  "hidden_size": 2048,
-  "initializer_range": 0.02,
-  "intermediate_size": 6144,
-  "layer_types": [
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention",
-    "full_attention"
-  ],
-  "max_position_embeddings": 40960,
-  "max_window_layers": 28,
-  "model_type": "qwen3",
-  "num_attention_heads": 16,
-  "num_hidden_layers": 28,
-  "num_key_value_heads": 8,
-  "pad_token_id": 151643,
-  "rms_norm_eps": 1e-06,
-  "rope_parameters": {
-    "rope_theta": 1000000,
-    "rope_type": "default"
-  },
-  "sliding_window": null,
-  "tie_word_embeddings": true,
-  "transformers_version": "5.7.0",
-  "use_cache": false,
-  "use_sliding_window": false,
-  "vocab_size": 151936
-}

checkpoint-3005/generation_config.json DELETED Viewed

@@ -1,12 +0,0 @@
-{
-  "do_sample": true,
-  "eos_token_id": [
-    151645,
-    151643
-  ],
-  "pad_token_id": 151643,
-  "temperature": 0.6,
-  "top_k": 20,
-  "top_p": 0.95,
-  "transformers_version": "5.7.0"
-}

checkpoint-3005/model.safetensors DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:fde9fe623bfa6989e3da983646e719ec1779b6ad922a247fe1d71a057cc64d44
-size 3441185608

checkpoint-3005/optimizer.pt DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:acf692f9b43588c61699e8ecf7e9bb4e1eaeb2bd5db135ce9b7721acba3c4654
-size 6882567855

checkpoint-3005/rng_state.pth DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
-size 14645

checkpoint-3005/scheduler.pt DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:9f3d5e2a3202f6694ba113c66b7f406f6367b844d54f552d4606ebb93a7c5473
-size 1465

checkpoint-3005/tokenizer.json DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
-size 11422650

checkpoint-3005/tokenizer_config.json DELETED Viewed

@@ -1,30 +0,0 @@
-{
-  "add_prefix_space": false,
-  "backend": "tokenizers",
-  "bos_token": null,
-  "clean_up_tokenization_spaces": false,
-  "eos_token": "<|im_end|>",
-  "errors": "replace",
-  "extra_special_tokens": [
-    "<|im_start|>",
-    "<|im_end|>",
-    "<|object_ref_start|>",
-    "<|object_ref_end|>",
-    "<|box_start|>",
-    "<|box_end|>",
-    "<|quad_start|>",
-    "<|quad_end|>",
-    "<|vision_start|>",
-    "<|vision_end|>",
-    "<|vision_pad|>",
-    "<|image_pad|>",
-    "<|video_pad|>"
-  ],
-  "is_local": false,
-  "local_files_only": false,
-  "model_max_length": 131072,
-  "pad_token": "<|endoftext|>",
-  "split_special_tokens": false,
-  "tokenizer_class": "Qwen2Tokenizer",
-  "unk_token": null
-}

checkpoint-3005/trainer_state.json DELETED Viewed

@@ -1,3034 +0,0 @@
-{
-  "best_global_step": null,
-  "best_metric": null,
-  "best_model_checkpoint": null,
-  "epoch": 1.0,
-  "eval_steps": 500,
-  "global_step": 3005,
-  "is_hyper_param_search": false,
-  "is_local_process_zero": true,
-  "is_world_process_zero": true,
-  "log_history": [
-    {
-      "entropy": 1.10129873752594,
-      "epoch": 0.0033278562366105785,
-      "grad_norm": 108.0,
-      "learning_rate": 4.945054945054946e-07,
-      "loss": 3.767025375366211,
-      "mean_token_accuracy": 0.4454367410391569,
-      "num_tokens": 58896.0,
-      "step": 10
-    },
-    {
-      "entropy": 1.0888467606157064,
-      "epoch": 0.006655712473221157,
-      "grad_norm": 109.0,
-      "learning_rate": 1.0439560439560442e-06,
-      "loss": 3.8026988983154295,
-      "mean_token_accuracy": 0.44370908215641974,
-      "num_tokens": 113880.0,
-      "step": 20
-    },
-    {
-      "entropy": 1.1470010358840228,
-      "epoch": 0.009983568709831735,
-      "grad_norm": 72.0,
-      "learning_rate": 1.5934065934065933e-06,
-      "loss": 3.680803680419922,
-      "mean_token_accuracy": 0.44784557111561296,
-      "num_tokens": 170458.0,
-      "step": 30
-    },
-    {
-      "entropy": 1.257061143964529,
-      "epoch": 0.013311424946442314,
-      "grad_norm": 43.25,
-      "learning_rate": 2.1428571428571427e-06,
-      "loss": 3.5028457641601562,
-      "mean_token_accuracy": 0.4467772927135229,
-      "num_tokens": 228784.0,
-      "step": 40
-    },
-    {
-      "entropy": 1.3516330607235432,
-      "epoch": 0.01663928118305289,
-      "grad_norm": 34.0,
-      "learning_rate": 2.6923076923076923e-06,
-      "loss": 3.2657550811767577,
-      "mean_token_accuracy": 0.4732152370736003,
-      "num_tokens": 282823.0,
-      "step": 50
-    },
-    {
-      "entropy": 1.4362287972122432,
-      "epoch": 0.01996713741966347,
-      "grad_norm": 22.875,
-      "learning_rate": 3.2417582417582424e-06,
-      "loss": 2.9696491241455076,
-      "mean_token_accuracy": 0.5022022368386387,
-      "num_tokens": 338410.0,
-      "step": 60
-    },
-    {
-      "entropy": 1.541035571694374,
-      "epoch": 0.02329499365627405,
-      "grad_norm": 16.125,
-      "learning_rate": 3.7912087912087915e-06,
-      "loss": 2.6919633865356447,
-      "mean_token_accuracy": 0.5306562084704638,
-      "num_tokens": 396586.0,
-      "step": 70
-    },
-    {
-      "entropy": 1.545711586251855,
-      "epoch": 0.026622849892884628,
-      "grad_norm": 15.25,
-      "learning_rate": 4.340659340659341e-06,
-      "loss": 2.3435085296630858,
-      "mean_token_accuracy": 0.5775316689163447,
-      "num_tokens": 450987.0,
-      "step": 80
-    },
-    {
-      "entropy": 1.5379757933318614,
-      "epoch": 0.029950706129495207,
-      "grad_norm": 8.5625,
-      "learning_rate": 4.890109890109891e-06,
-      "loss": 2.2016986846923827,
-      "mean_token_accuracy": 0.6109725600108504,
-      "num_tokens": 507007.0,
-      "step": 90
-    },
-    {
-      "entropy": 1.524170184135437,
-      "epoch": 0.03327856236610578,
-      "grad_norm": 6.9375,
-      "learning_rate": 4.999907016059834e-06,
-      "loss": 2.084031105041504,
-      "mean_token_accuracy": 0.6345648266375065,
-      "num_tokens": 565620.0,
-      "step": 100
-    },
-    {
-      "entropy": 1.5098746936768292,
-      "epoch": 0.036606418602716365,
-      "grad_norm": 5.4375,
-      "learning_rate": 4.999529280657373e-06,
-      "loss": 1.9416280746459962,
-      "mean_token_accuracy": 0.6530486961826683,
-      "num_tokens": 619148.0,
-      "step": 110
-    },
-    {
-      "entropy": 1.5675622571259737,
-      "epoch": 0.03993427483932694,
-      "grad_norm": 7.03125,
-      "learning_rate": 4.998861026166586e-06,
-      "loss": 1.9410280227661132,
-      "mean_token_accuracy": 0.6608247576281429,
-      "num_tokens": 675213.0,
-      "step": 120
-    },
-    {
-      "entropy": 1.5429053504019976,
-      "epoch": 0.04326213107593752,
-      "grad_norm": 5.90625,
-      "learning_rate": 4.997902330258374e-06,
-      "loss": 1.9232656478881835,
-      "mean_token_accuracy": 0.6734512994065881,
-      "num_tokens": 729143.0,
-      "step": 130
-    },
-    {
-      "entropy": 1.5630079340189695,
-      "epoch": 0.0465899873125481,
-      "grad_norm": 4.3125,
-      "learning_rate": 4.996653304361499e-06,
-      "loss": 1.944386863708496,
-      "mean_token_accuracy": 0.6605244895443321,
-      "num_tokens": 786615.0,
-      "step": 140
-    },
-    {
-      "entropy": 1.5707480294629932,
-      "epoch": 0.04991784354915867,
-      "grad_norm": 3.8125,
-      "learning_rate": 4.995114093649645e-06,
-      "loss": 1.8894290924072266,
-      "mean_token_accuracy": 0.653051363863051,
-      "num_tokens": 845125.0,
-      "step": 150
-    },
-    {
-      "entropy": 1.5258742582052947,
-      "epoch": 0.053245699785769256,
-      "grad_norm": 3.65625,
-      "learning_rate": 4.993284877024531e-06,
-      "loss": 1.8971254348754882,
-      "mean_token_accuracy": 0.664631326124072,
-      "num_tokens": 902170.0,
-      "step": 160
-    },
-    {
-      "entropy": 1.5339138861745596,
-      "epoch": 0.05657355602237983,
-      "grad_norm": 3.578125,
-      "learning_rate": 4.99116586709513e-06,
-      "loss": 1.8392156600952148,
-      "mean_token_accuracy": 0.6655616400763392,
-      "num_tokens": 958941.0,
-      "step": 170
-    },
-    {
-      "entropy": 1.5186992075294257,
-      "epoch": 0.059901412258990414,
-      "grad_norm": 4.40625,
-      "learning_rate": 4.988757310152946e-06,
-      "loss": 1.8561361312866211,
-      "mean_token_accuracy": 0.6680668929591775,
-      "num_tokens": 1014457.0,
-      "step": 180
-    },
-    {
-      "entropy": 1.5594698129221798,
-      "epoch": 0.063229268495601,
-      "grad_norm": 3.34375,
-      "learning_rate": 4.986059486143398e-06,
-      "loss": 1.859320068359375,
-      "mean_token_accuracy": 0.6527148365974427,
-      "num_tokens": 1073981.0,
-      "step": 190
-    },
-    {
-      "entropy": 1.531857279315591,
-      "epoch": 0.06655712473221156,
-      "grad_norm": 3.640625,
-      "learning_rate": 4.9830727086332735e-06,
-      "loss": 1.856361198425293,
-      "mean_token_accuracy": 0.6642919164150953,
-      "num_tokens": 1131765.0,
-      "step": 200
-    },
-    {
-      "entropy": 1.4927255801856518,
-      "epoch": 0.06988498096882215,
-      "grad_norm": 3.703125,
-      "learning_rate": 4.979797324774291e-06,
-      "loss": 1.8407997131347655,
-      "mean_token_accuracy": 0.6682391256093979,
-      "num_tokens": 1188376.0,
-      "step": 210
-    },
-    {
-      "entropy": 1.631840031221509,
-      "epoch": 0.07321283720543273,
-      "grad_norm": 3.640625,
-      "learning_rate": 4.976233715262742e-06,
-      "loss": 1.90419921875,
-      "mean_token_accuracy": 0.6405316749587655,
-      "num_tokens": 1251178.0,
-      "step": 220
-    },
-    {
-      "entropy": 1.532007560133934,
-      "epoch": 0.0765406934420433,
-      "grad_norm": 3.546875,
-      "learning_rate": 4.9723822942952515e-06,
-      "loss": 1.856277084350586,
-      "mean_token_accuracy": 0.6626312166452408,
-      "num_tokens": 1306871.0,
-      "step": 230
-    },
-    {
-      "entropy": 1.4832018595188856,
-      "epoch": 0.07986854967865388,
-      "grad_norm": 3.0,
-      "learning_rate": 4.968243509520627e-06,
-      "loss": 1.8073028564453124,
-      "mean_token_accuracy": 0.6722651876509189,
-      "num_tokens": 1362512.0,
-      "step": 240
-    },
-    {
-      "entropy": 1.5231953041628004,
-      "epoch": 0.08319640591526446,
-      "grad_norm": 3.625,
-      "learning_rate": 4.963817841987838e-06,
-      "loss": 1.8772123336791993,
-      "mean_token_accuracy": 0.662684210576117,
-      "num_tokens": 1419454.0,
-      "step": 250
-    },
-    {
-      "entropy": 1.4576873349025845,
-      "epoch": 0.08652426215187504,
-      "grad_norm": 3.375,
-      "learning_rate": 4.959105806090096e-06,
-      "loss": 1.7649114608764649,
-      "mean_token_accuracy": 0.6785721253603697,
-      "num_tokens": 1472716.0,
-      "step": 260
-    },
-    {
-      "entropy": 1.4679617136716843,
-      "epoch": 0.08985211838848561,
-      "grad_norm": 3.4375,
-      "learning_rate": 4.954107949505072e-06,
-      "loss": 1.7993288040161133,
-      "mean_token_accuracy": 0.6775210529565812,
-      "num_tokens": 1528035.0,
-      "step": 270
-    },
-    {
-      "entropy": 1.6027428967878223,
-      "epoch": 0.0931799746250962,
-      "grad_norm": 3.265625,
-      "learning_rate": 4.948824853131237e-06,
-      "loss": 1.8703588485717773,
-      "mean_token_accuracy": 0.6443315070122481,
-      "num_tokens": 1590272.0,
-      "step": 280
-    },
-    {
-      "entropy": 1.4947082299739123,
-      "epoch": 0.09650783086170678,
-      "grad_norm": 3.484375,
-      "learning_rate": 4.943257131020346e-06,
-      "loss": 1.8419960021972657,
-      "mean_token_accuracy": 0.6725484007969499,
-      "num_tokens": 1645949.0,
-      "step": 290
-    },
-    {
-      "entropy": 1.5246242409572006,
-      "epoch": 0.09983568709831735,
-      "grad_norm": 3.140625,
-      "learning_rate": 4.937405430306067e-06,
-      "loss": 1.831895637512207,
-      "mean_token_accuracy": 0.6614987364038825,
-      "num_tokens": 1703424.0,
-      "step": 300
-    },
-    {
-      "entropy": 1.5506725419312715,
-      "epoch": 0.10316354333492793,
-      "grad_norm": 3.265625,
-      "learning_rate": 4.931270431128766e-06,
-      "loss": 1.874525260925293,
-      "mean_token_accuracy": 0.6693461928516626,
-      "num_tokens": 1759368.0,
-      "step": 310
-    },
-    {
-      "entropy": 1.4530342649668455,
-      "epoch": 0.10649139957153851,
-      "grad_norm": 3.53125,
-      "learning_rate": 4.92485284655645e-06,
-      "loss": 1.7907138824462892,
-      "mean_token_accuracy": 0.6832739936187864,
-      "num_tokens": 1813490.0,
-      "step": 320
-    },
-    {
-      "entropy": 1.5579677358269692,
-      "epoch": 0.1098192558081491,
-      "grad_norm": 3.578125,
-      "learning_rate": 4.918153422501895e-06,
-      "loss": 1.8589799880981446,
-      "mean_token_accuracy": 0.6599713157862425,
-      "num_tokens": 1873095.0,
-      "step": 330
-    },
-    {
-      "entropy": 1.5668061392381787,
-      "epoch": 0.11314711204475966,
-      "grad_norm": 3.609375,
-      "learning_rate": 4.911172937635942e-06,
-      "loss": 1.8748260498046876,
-      "mean_token_accuracy": 0.6570447353646159,
-      "num_tokens": 1933848.0,
-      "step": 340
-    },
-    {
-      "entropy": 1.5254148514941335,
-      "epoch": 0.11647496828137024,
-      "grad_norm": 3.359375,
-      "learning_rate": 4.903912203296995e-06,
-      "loss": 1.8022354125976563,
-      "mean_token_accuracy": 0.6677953096106648,
-      "num_tokens": 1991126.0,
-      "step": 350
-    },
-    {
-      "entropy": 1.4694697640836238,
-      "epoch": 0.11980282451798083,
-      "grad_norm": 3.359375,
-      "learning_rate": 4.896372063396722e-06,
-      "loss": 1.83734130859375,
-      "mean_token_accuracy": 0.681332571990788,
-      "num_tokens": 2045167.0,
-      "step": 360
-    },
-    {
-      "entropy": 1.4768266826868057,
-      "epoch": 0.1231306807545914,
-      "grad_norm": 3.390625,
-      "learning_rate": 4.888553394321961e-06,
-      "loss": 1.7910270690917969,
-      "mean_token_accuracy": 0.6767197269946337,
-      "num_tokens": 2100517.0,
-      "step": 370
-    },
-    {
-      "entropy": 1.5699415881186725,
-      "epoch": 0.126458536991202,
-      "grad_norm": 3.375,
-      "learning_rate": 4.880457104832868e-06,
-      "loss": 1.8176877975463868,
-      "mean_token_accuracy": 0.6539639614522457,
-      "num_tokens": 2159392.0,
-      "step": 380
-    },
-    {
-      "entropy": 1.52086035143584,
-      "epoch": 0.12978639322781255,
-      "grad_norm": 3.5,
-      "learning_rate": 4.872084135957282e-06,
-      "loss": 1.8439929962158204,
-      "mean_token_accuracy": 0.6692373549565673,
-      "num_tokens": 2217162.0,
-      "step": 390
-    },
-    {
-      "entropy": 1.4943883692845703,
-      "epoch": 0.13311424946442313,
-      "grad_norm": 3.671875,
-      "learning_rate": 4.863435460881356e-06,
-      "loss": 1.796240234375,
-      "mean_token_accuracy": 0.669551943615079,
-      "num_tokens": 2274353.0,
-      "step": 400
-    },
-    {
-      "entropy": 1.4666842954233288,
-      "epoch": 0.1364421057010337,
-      "grad_norm": 3.796875,
-      "learning_rate": 4.8545120848364426e-06,
-      "loss": 1.7682180404663086,
-      "mean_token_accuracy": 0.6762362584471703,
-      "num_tokens": 2329451.0,
-      "step": 410
-    },
-    {
-      "entropy": 1.5560785047709942,
-      "epoch": 0.1397699619376443,
-      "grad_norm": 3.09375,
-      "learning_rate": 4.845315044982255e-06,
-      "loss": 1.8618741989135743,
-      "mean_token_accuracy": 0.656427138671279,
-      "num_tokens": 2389129.0,
-      "step": 420
-    },
-    {
-      "entropy": 1.5334357468411326,
-      "epoch": 0.14309781817425488,
-      "grad_norm": 3.03125,
-      "learning_rate": 4.83584541028632e-06,
-      "loss": 1.8374927520751954,
-      "mean_token_accuracy": 0.663763827458024,
-      "num_tokens": 2448344.0,
-      "step": 430
-    },
-    {
-      "entropy": 1.5113085351884366,
-      "epoch": 0.14642567441086546,
-      "grad_norm": 3.84375,
-      "learning_rate": 4.826104281399732e-06,
-      "loss": 1.838669204711914,
-      "mean_token_accuracy": 0.6693438524380326,
-      "num_tokens": 2506162.0,
-      "step": 440
-    },
-    {
-      "entropy": 1.5036728654056788,
-      "epoch": 0.14975353064747604,
-      "grad_norm": 3.46875,
-      "learning_rate": 4.816092790529226e-06,
-      "loss": 1.8022287368774415,
-      "mean_token_accuracy": 0.6683000991120934,
-      "num_tokens": 2562576.0,
-      "step": 450
-    },
-    {
-      "entropy": 1.4728989064693452,
-      "epoch": 0.1530813868840866,
-      "grad_norm": 3.515625,
-      "learning_rate": 4.8058121013055785e-06,
-      "loss": 1.7823028564453125,
-      "mean_token_accuracy": 0.6793223405256867,
-      "num_tokens": 2617107.0,
-      "step": 460
-    },
-    {
-      "entropy": 1.4097753472626209,
-      "epoch": 0.15640924312069718,
-      "grad_norm": 3.53125,
-      "learning_rate": 4.795263408648364e-06,
-      "loss": 1.7742498397827149,
-      "mean_token_accuracy": 0.6969904739409685,
-      "num_tokens": 2669898.0,
-      "step": 470
-    },
-    {
-      "entropy": 1.5106059784069656,
-      "epoch": 0.15973709935730776,
-      "grad_norm": 3.25,
-      "learning_rate": 4.784447938627064e-06,
-      "loss": 1.7913530349731446,
-      "mean_token_accuracy": 0.6729067243635655,
-      "num_tokens": 2726029.0,
-      "step": 480
-    },
-    {
-      "entropy": 1.5330951320007444,
-      "epoch": 0.16306495559391834,
-      "grad_norm": 3.90625,
-      "learning_rate": 4.773366948318567e-06,
-      "loss": 1.8522315979003907,
-      "mean_token_accuracy": 0.6655845906585455,
-      "num_tokens": 2780960.0,
-      "step": 490
-    },
-    {
-      "entropy": 1.5599053224548698,
-      "epoch": 0.16639281183052892,
-      "grad_norm": 3.453125,
-      "learning_rate": 4.762021725661059e-06,
-      "loss": 1.876106071472168,
-      "mean_token_accuracy": 0.6559333061799407,
-      "num_tokens": 2841349.0,
-      "step": 500
-    },
-    {
-      "entropy": 1.4369502196088433,
-      "epoch": 0.1697206680671395,
-      "grad_norm": 3.78125,
-      "learning_rate": 4.75041358930432e-06,
-      "loss": 1.7831584930419921,
-      "mean_token_accuracy": 0.6914990266785026,
-      "num_tokens": 2893986.0,
-      "step": 510
-    },
-    {
-      "entropy": 1.4670223101973534,
-      "epoch": 0.1730485243037501,
-      "grad_norm": 3.859375,
-      "learning_rate": 4.738543888456467e-06,
-      "loss": 1.8110008239746094,
-      "mean_token_accuracy": 0.6840576194226742,
-      "num_tokens": 2945823.0,
-      "step": 520
-    },
-    {
-      "entropy": 1.4602944780141116,
-      "epoch": 0.17637638054036064,
-      "grad_norm": 3.5625,
-      "learning_rate": 4.726414002727133e-06,
-      "loss": 1.7574054718017578,
-      "mean_token_accuracy": 0.6761805353686213,
-      "num_tokens": 3001053.0,
-      "step": 530
-    },
-    {
-      "entropy": 1.586913832463324,
-      "epoch": 0.17970423677697123,
-      "grad_norm": 3.265625,
-      "learning_rate": 4.7140253419671105e-06,
-      "loss": 1.8853599548339843,
-      "mean_token_accuracy": 0.6572996182367206,
-      "num_tokens": 3060779.0,
-      "step": 540
-    },
-    {
-      "entropy": 1.5160053435713052,
-      "epoch": 0.1830320930135818,
-      "grad_norm": 3.5625,
-      "learning_rate": 4.701379346104494e-06,
-      "loss": 1.840207290649414,
-      "mean_token_accuracy": 0.6684120044112205,
-      "num_tokens": 3118357.0,
-      "step": 550
-    },
-    {
-      "entropy": 1.5852246707305313,
-      "epoch": 0.1863599492501924,
-      "grad_norm": 4.03125,
-      "learning_rate": 4.688477484977309e-06,
-      "loss": 1.8544384002685548,
-      "mean_token_accuracy": 0.654945139028132,
-      "num_tokens": 3179648.0,
-      "step": 560
-    },
-    {
-      "entropy": 1.5925893921405077,
-      "epoch": 0.18968780548680297,
-      "grad_norm": 2.9375,
-      "learning_rate": 4.675321258162684e-06,
-      "loss": 1.8783863067626954,
-      "mean_token_accuracy": 0.6494444390758872,
-      "num_tokens": 3241203.0,
-      "step": 570
-    },
-    {
-      "entropy": 1.4771136017516255,
-      "epoch": 0.19301566172341356,
-      "grad_norm": 3.921875,
-      "learning_rate": 4.6619121948025436e-06,
-      "loss": 1.7917003631591797,
-      "mean_token_accuracy": 0.6753362514078617,
-      "num_tokens": 3296607.0,
-      "step": 580
-    },
-    {
-      "entropy": 1.5505024868994952,
-      "epoch": 0.19634351796002414,
-      "grad_norm": 3.421875,
-      "learning_rate": 4.6482518534258845e-06,
-      "loss": 1.8494434356689453,
-      "mean_token_accuracy": 0.657556539028883,
-      "num_tokens": 3356398.0,
-      "step": 590
-    },
-    {
-      "entropy": 1.530809846147895,
-      "epoch": 0.1996713741966347,
-      "grad_norm": 3.390625,
-      "learning_rate": 4.634341821767628e-06,
-      "loss": 1.8060928344726563,
-      "mean_token_accuracy": 0.6659811200574041,
-      "num_tokens": 3414650.0,
-      "step": 600
-    },
-    {
-      "entropy": 1.522915854305029,
-      "epoch": 0.20299923043324528,
-      "grad_norm": 4.09375,
-      "learning_rate": 4.620183716584075e-06,
-      "loss": 1.8677490234375,
-      "mean_token_accuracy": 0.6639480855315923,
-      "num_tokens": 3472636.0,
-      "step": 610
-    },
-    {
-      "entropy": 1.458723592199385,
-      "epoch": 0.20632708666985586,
-      "grad_norm": 3.359375,
-      "learning_rate": 4.6057791834649936e-06,
-      "loss": 1.814854621887207,
-      "mean_token_accuracy": 0.6869227834045887,
-      "num_tokens": 3526641.0,
-      "step": 620
-    },
-    {
-      "entropy": 1.5554879356175662,
-      "epoch": 0.20965494290646644,
-      "grad_norm": 3.46875,
-      "learning_rate": 4.591129896642354e-06,
-      "loss": 1.8581478118896484,
-      "mean_token_accuracy": 0.6633507860824466,
-      "num_tokens": 3585211.0,
-      "step": 630
-    },
-    {
-      "entropy": 1.3848019087687136,
-      "epoch": 0.21298279914307702,
-      "grad_norm": 3.90625,
-      "learning_rate": 4.57623755879573e-06,
-      "loss": 1.7439926147460938,
-      "mean_token_accuracy": 0.6958894932642579,
-      "num_tokens": 3637641.0,
-      "step": 640
-    },
-    {
-      "entropy": 1.4623519806191325,
-      "epoch": 0.2163106553796876,
-      "grad_norm": 3.71875,
-      "learning_rate": 4.561103900854401e-06,
-      "loss": 1.7499933242797852,
-      "mean_token_accuracy": 0.6818006206303835,
-      "num_tokens": 3693156.0,
-      "step": 650
-    },
-    {
-      "entropy": 1.5304298089817165,
-      "epoch": 0.2196385116162982,
-      "grad_norm": 3.640625,
-      "learning_rate": 4.545730681796164e-06,
-      "loss": 1.834868812561035,
-      "mean_token_accuracy": 0.6672181274741888,
-      "num_tokens": 3751468.0,
-      "step": 660
-    },
-    {
-      "entropy": 1.5507907839491963,
-      "epoch": 0.22296636785290874,
-      "grad_norm": 3.34375,
-      "learning_rate": 4.5301196884428884e-06,
-      "loss": 1.8539545059204101,
-      "mean_token_accuracy": 0.6580703053623438,
-      "num_tokens": 3812358.0,
-      "step": 670
-    },
-    {
-      "entropy": 1.5054455982521175,
-      "epoch": 0.22629422408951932,
-      "grad_norm": 3.53125,
-      "learning_rate": 4.514272735252839e-06,
-      "loss": 1.8418838500976562,
-      "mean_token_accuracy": 0.6747614100575448,
-      "num_tokens": 3868669.0,
-      "step": 680
-    },
-    {
-      "entropy": 1.4915354089811443,
-      "epoch": 0.2296220803261299,
-      "grad_norm": 3.234375,
-      "learning_rate": 4.498191664109777e-06,
-      "loss": 1.7721128463745117,
-      "mean_token_accuracy": 0.6739375120028853,
-      "num_tokens": 3925971.0,
-      "step": 690
-    },
-    {
-      "entropy": 1.4589189939200877,
-      "epoch": 0.2329499365627405,
-      "grad_norm": 3.390625,
-      "learning_rate": 4.481878344108881e-06,
-      "loss": 1.7456041336059571,
-      "mean_token_accuracy": 0.6781141830608248,
-      "num_tokens": 3980546.0,
-      "step": 700
-    },
-    {
-      "entropy": 1.5345530206337572,
-      "epoch": 0.23627779279935107,
-      "grad_norm": 3.609375,
-      "learning_rate": 4.465334671339502e-06,
-      "loss": 1.8225982666015625,
-      "mean_token_accuracy": 0.6616290437057615,
-      "num_tokens": 4038422.0,
-      "step": 710
-    },
-    {
-      "entropy": 1.5462797056883573,
-      "epoch": 0.23960564903596165,
-      "grad_norm": 3.5625,
-      "learning_rate": 4.448562568664785e-06,
-      "loss": 1.8371921539306642,
-      "mean_token_accuracy": 0.663363009877503,
-      "num_tokens": 4096941.0,
-      "step": 720
-    },
-    {
-      "entropy": 1.5453467000275851,
-      "epoch": 0.24293350527257224,
-      "grad_norm": 3.46875,
-      "learning_rate": 4.431563985498171e-06,
-      "loss": 1.8565343856811523,
-      "mean_token_accuracy": 0.657871343754232,
-      "num_tokens": 4156911.0,
-      "step": 730
-    },
-    {
-      "entropy": 1.4919695736840368,
-      "epoch": 0.2462613615091828,
-      "grad_norm": 2.984375,
-      "learning_rate": 4.414340897576822e-06,
-      "loss": 1.7564289093017578,
-      "mean_token_accuracy": 0.6680593078956008,
-      "num_tokens": 4213079.0,
-      "step": 740
-    },
-    {
-      "entropy": 1.4652381472289562,
-      "epoch": 0.24958921774579337,
-      "grad_norm": 3.59375,
-      "learning_rate": 4.396895306731978e-06,
-      "loss": 1.8269855499267578,
-      "mean_token_accuracy": 0.6806315137073398,
-      "num_tokens": 4268858.0,
-      "step": 750
-    },
-    {
-      "entropy": 1.4714900465682148,
-      "epoch": 0.252917073982404,
-      "grad_norm": 3.546875,
-      "learning_rate": 4.379229240656288e-06,
-      "loss": 1.7527101516723633,
-      "mean_token_accuracy": 0.6823275567963719,
-      "num_tokens": 4324009.0,
-      "step": 760
-    },
-    {
-      "entropy": 1.4752345664426685,
-      "epoch": 0.25624493021901457,
-      "grad_norm": 4.0,
-      "learning_rate": 4.361344752668129e-06,
-      "loss": 1.7995849609375,
-      "mean_token_accuracy": 0.6790751110762357,
-      "num_tokens": 4379486.0,
-      "step": 770
-    },
-    {
-      "entropy": 1.4920156214386224,
-      "epoch": 0.2595727864556251,
-      "grad_norm": 3.390625,
-      "learning_rate": 4.343243921472958e-06,
-      "loss": 1.805205726623535,
-      "mean_token_accuracy": 0.6717268094420433,
-      "num_tokens": 4438258.0,
-      "step": 780
-    },
-    {
-      "entropy": 1.5178041452541948,
-      "epoch": 0.2629006426922357,
-      "grad_norm": 3.453125,
-      "learning_rate": 4.324928850921692e-06,
-      "loss": 1.8258188247680665,
-      "mean_token_accuracy": 0.664860924333334,
-      "num_tokens": 4495033.0,
-      "step": 790
-    },
-    {
-      "entropy": 1.5401784598827362,
-      "epoch": 0.26622849892884626,
-      "grad_norm": 3.234375,
-      "learning_rate": 4.3064016697661896e-06,
-      "loss": 1.8189167022705077,
-      "mean_token_accuracy": 0.6631952647119761,
-      "num_tokens": 4553223.0,
-      "step": 800
-    },
-    {
-      "entropy": 1.536975803039968,
-      "epoch": 0.26955635516545684,
-      "grad_norm": 3.6875,
-      "learning_rate": 4.2876645314118205e-06,
-      "loss": 1.797938346862793,
-      "mean_token_accuracy": 0.660793448984623,
-      "num_tokens": 4610392.0,
-      "step": 810
-    },
-    {
-      "entropy": 1.4629426643252372,
-      "epoch": 0.2728842114020674,
-      "grad_norm": 3.890625,
-      "learning_rate": 4.268719613667182e-06,
-      "loss": 1.7804573059082032,
-      "mean_token_accuracy": 0.6752110937610268,
-      "num_tokens": 4666654.0,
-      "step": 820
-    },
-    {
-      "entropy": 1.491467403434217,
-      "epoch": 0.276212067638678,
-      "grad_norm": 3.609375,
-      "learning_rate": 4.249569118490968e-06,
-      "loss": 1.7742938995361328,
-      "mean_token_accuracy": 0.6696773633360863,
-      "num_tokens": 4723217.0,
-      "step": 830
-    },
-    {
-      "entropy": 1.5438310438767076,
-      "epoch": 0.2795399238752886,
-      "grad_norm": 3.453125,
-      "learning_rate": 4.230215271736039e-06,
-      "loss": 1.8140148162841796,
-      "mean_token_accuracy": 0.6613235106691718,
-      "num_tokens": 4781200.0,
-      "step": 840
-    },
-    {
-      "entropy": 1.406342525035143,
-      "epoch": 0.28286778011189917,
-      "grad_norm": 4.09375,
-      "learning_rate": 4.210660322890713e-06,
-      "loss": 1.7505395889282227,
-      "mean_token_accuracy": 0.6878415673971177,
-      "num_tokens": 4834372.0,
-      "step": 850
-    },
-    {
-      "entropy": 1.4922395968809723,
-      "epoch": 0.28619563634850975,
-      "grad_norm": 3.84375,
-      "learning_rate": 4.190906544817305e-06,
-      "loss": 1.8420244216918946,
-      "mean_token_accuracy": 0.6758656786754728,
-      "num_tokens": 4890814.0,
-      "step": 860
-    },
-    {
-      "entropy": 1.477423189021647,
-      "epoch": 0.28952349258512033,
-      "grad_norm": 4.0625,
-      "learning_rate": 4.170956233487958e-06,
-      "loss": 1.784296989440918,
-      "mean_token_accuracy": 0.6722941329702735,
-      "num_tokens": 4945443.0,
-      "step": 870
-    },
-    {
-      "entropy": 1.4779939003288747,
-      "epoch": 0.2928513488217309,
-      "grad_norm": 3.4375,
-      "learning_rate": 4.15081170771778e-06,
-      "loss": 1.7739248275756836,
-      "mean_token_accuracy": 0.6801445078104734,
-      "num_tokens": 4998132.0,
-      "step": 880
-    },
-    {
-      "entropy": 1.538309420272708,
-      "epoch": 0.2961792050583415,
-      "grad_norm": 3.640625,
-      "learning_rate": 4.130475308895329e-06,
-      "loss": 1.8317113876342774,
-      "mean_token_accuracy": 0.6629494095221162,
-      "num_tokens": 5055840.0,
-      "step": 890
-    },
-    {
-      "entropy": 1.50099892988801,
-      "epoch": 0.2995070612949521,
-      "grad_norm": 3.671875,
-      "learning_rate": 4.10994940071048e-06,
-      "loss": 1.8030107498168946,
-      "mean_token_accuracy": 0.6740143038332462,
-      "num_tokens": 5111330.0,
-      "step": 900
-    },
-    {
-      "entropy": 1.5412938056513668,
-      "epoch": 0.30283491753156266,
-      "grad_norm": 3.484375,
-      "learning_rate": 4.089236368879685e-06,
-      "loss": 1.8853994369506837,
-      "mean_token_accuracy": 0.6653259627521038,
-      "num_tokens": 5169915.0,
-      "step": 910
-    },
-    {
-      "entropy": 1.367060486972332,
-      "epoch": 0.3061627737681732,
-      "grad_norm": 3.390625,
-      "learning_rate": 4.06833862086869e-06,
-      "loss": 1.7047807693481445,
-      "mean_token_accuracy": 0.7047108177095651,
-      "num_tokens": 5219404.0,
-      "step": 920
-    },
-    {
-      "entropy": 1.5187823886051774,
-      "epoch": 0.3094906300047838,
-      "grad_norm": 4.03125,
-      "learning_rate": 4.047258585612715e-06,
-      "loss": 1.8053022384643556,
-      "mean_token_accuracy": 0.668337308242917,
-      "num_tokens": 5276764.0,
-      "step": 930
-    },
-    {
-      "entropy": 1.5128734022378922,
-      "epoch": 0.31281848624139436,
-      "grad_norm": 3.90625,
-      "learning_rate": 4.025998713234137e-06,
-      "loss": 1.8357311248779298,
-      "mean_token_accuracy": 0.666442839615047,
-      "num_tokens": 5334208.0,
-      "step": 940
-    },
-    {
-      "entropy": 1.4276204142719506,
-      "epoch": 0.31614634247800494,
-      "grad_norm": 3.46875,
-      "learning_rate": 4.004561474757715e-06,
-      "loss": 1.7479360580444336,
-      "mean_token_accuracy": 0.6900471523404121,
-      "num_tokens": 5386242.0,
-      "step": 950
-    },
-    {
-      "entropy": 1.5229905935004353,
-      "epoch": 0.3194741987146155,
-      "grad_norm": 3.28125,
-      "learning_rate": 3.982949361823388e-06,
-      "loss": 1.8292566299438477,
-      "mean_token_accuracy": 0.6686824513599277,
-      "num_tokens": 5443228.0,
-      "step": 960
-    },
-    {
-      "entropy": 1.552778111770749,
-      "epoch": 0.3228020549512261,
-      "grad_norm": 3.453125,
-      "learning_rate": 3.961164886396666e-06,
-      "loss": 1.8724512100219726,
-      "mean_token_accuracy": 0.6608447909355164,
-      "num_tokens": 5504605.0,
-      "step": 970
-    },
-    {
-      "entropy": 1.5585516208782793,
-      "epoch": 0.3261299111878367,
-      "grad_norm": 3.34375,
-      "learning_rate": 3.939210580476669e-06,
-      "loss": 1.8408451080322266,
-      "mean_token_accuracy": 0.6585392042994499,
-      "num_tokens": 5563476.0,
-      "step": 980
-    },
-    {
-      "entropy": 1.472256763651967,
-      "epoch": 0.32945776742444727,
-      "grad_norm": 3.78125,
-      "learning_rate": 3.9170889958018346e-06,
-      "loss": 1.7644929885864258,
-      "mean_token_accuracy": 0.6759116293862462,
-      "num_tokens": 5618010.0,
-      "step": 990
-    },
-    {
-      "entropy": 1.5166746955364943,
-      "epoch": 0.33278562366105785,
-      "grad_norm": 3.09375,
-      "learning_rate": 3.894802703553329e-06,
-      "loss": 1.857872200012207,
-      "mean_token_accuracy": 0.6661853093653918,
-      "num_tokens": 5675293.0,
-      "step": 1000
-    },
-    {
-      "entropy": 1.5077380370348692,
-      "epoch": 0.33611347989766843,
-      "grad_norm": 2.921875,
-      "learning_rate": 3.872354294056202e-06,
-      "loss": 1.804924774169922,
-      "mean_token_accuracy": 0.6691981546580792,
-      "num_tokens": 5733357.0,
-      "step": 1010
-    },
-    {
-      "entropy": 1.5556912183761598,
-      "epoch": 0.339441336134279,
-      "grad_norm": 3.65625,
-      "learning_rate": 3.849746376478313e-06,
-      "loss": 1.8279289245605468,
-      "mean_token_accuracy": 0.6591873183846474,
-      "num_tokens": 5791310.0,
-      "step": 1020
-    },
-    {
-      "entropy": 1.5258273880928754,
-      "epoch": 0.3427691923708896,
-      "grad_norm": 3.78125,
-      "learning_rate": 3.826981578527067e-06,
-      "loss": 1.7992376327514648,
-      "mean_token_accuracy": 0.6683254396542907,
-      "num_tokens": 5846617.0,
-      "step": 1030
-    },
-    {
-      "entropy": 1.4530404249206186,
-      "epoch": 0.3460970486075002,
-      "grad_norm": 3.09375,
-      "learning_rate": 3.8040625461440015e-06,
-      "loss": 1.762941551208496,
-      "mean_token_accuracy": 0.6784038580954075,
-      "num_tokens": 5902593.0,
-      "step": 1040
-    },
-    {
-      "entropy": 1.4768676215782761,
-      "epoch": 0.34942490484411076,
-      "grad_norm": 3.234375,
-      "learning_rate": 3.7809919431972485e-06,
-      "loss": 1.7962345123291015,
-      "mean_token_accuracy": 0.6791946424171329,
-      "num_tokens": 5957173.0,
-      "step": 1050
-    },
-    {
-      "entropy": 1.426030796766281,
-      "epoch": 0.3527527610807213,
-      "grad_norm": 3.21875,
-      "learning_rate": 3.7577724511719116e-06,
-      "loss": 1.7332475662231446,
-      "mean_token_accuracy": 0.6893144998699426,
-      "num_tokens": 6010075.0,
-      "step": 1060
-    },
-    {
-      "entropy": 1.5724808547645808,
-      "epoch": 0.35608061731733187,
-      "grad_norm": 3.609375,
-      "learning_rate": 3.7344067688584024e-06,
-      "loss": 1.8324731826782226,
-      "mean_token_accuracy": 0.6561689108610154,
-      "num_tokens": 6070660.0,
-      "step": 1070
-    },
-    {
-      "entropy": 1.4366191050037742,
-      "epoch": 0.35940847355394245,
-      "grad_norm": 3.28125,
-      "learning_rate": 3.7108976120387587e-06,
-      "loss": 1.7667678833007812,
-      "mean_token_accuracy": 0.6842338047921658,
-      "num_tokens": 6124119.0,
-      "step": 1080
-    },
-    {
-      "entropy": 1.472322289273143,
-      "epoch": 0.36273632979055304,
-      "grad_norm": 3.3125,
-      "learning_rate": 3.6872477131709916e-06,
-      "loss": 1.7943756103515625,
-      "mean_token_accuracy": 0.6746624225750566,
-      "num_tokens": 6178914.0,
-      "step": 1090
-    },
-    {
-      "entropy": 1.5059184875339269,
-      "epoch": 0.3660641860271636,
-      "grad_norm": 3.359375,
-      "learning_rate": 3.663459821071492e-06,
-      "loss": 1.8035322189331056,
-      "mean_token_accuracy": 0.6714792117476464,
-      "num_tokens": 6236218.0,
-      "step": 1100
-    },
-    {
-      "entropy": 1.5011418825015426,
-      "epoch": 0.3693920422637742,
-      "grad_norm": 3.6875,
-      "learning_rate": 3.6395367005955397e-06,
-      "loss": 1.815826988220215,
-      "mean_token_accuracy": 0.6701009083539248,
-      "num_tokens": 6294142.0,
-      "step": 1110
-    },
-    {
-      "entropy": 1.506100300140679,
-      "epoch": 0.3727198985003848,
-      "grad_norm": 3.296875,
-      "learning_rate": 3.615481132315942e-06,
-      "loss": 1.8021814346313476,
-      "mean_token_accuracy": 0.6715581325814128,
-      "num_tokens": 6350301.0,
-      "step": 1120
-    },
-    {
-      "entropy": 1.5138352997601032,
-      "epoch": 0.37604775473699537,
-      "grad_norm": 3.671875,
-      "learning_rate": 3.591295912199852e-06,
-      "loss": 1.8525350570678711,
-      "mean_token_accuracy": 0.6697064289823175,
-      "num_tokens": 6406063.0,
-      "step": 1130
-    },
-    {
-      "entropy": 1.4703194240108133,
-      "epoch": 0.37937561097360595,
-      "grad_norm": 3.890625,
-      "learning_rate": 3.5669838512837928e-06,
-      "loss": 1.8076473236083985,
-      "mean_token_accuracy": 0.67458051238209,
-      "num_tokens": 6461809.0,
-      "step": 1140
-    },
-    {
-      "entropy": 1.4724275497719646,
-      "epoch": 0.38270346721021653,
-      "grad_norm": 3.46875,
-      "learning_rate": 3.542547775346933e-06,
-      "loss": 1.779327964782715,
-      "mean_token_accuracy": 0.679173220321536,
-      "num_tokens": 6518131.0,
-      "step": 1150
-    },
-    {
-      "entropy": 1.46819082275033,
-      "epoch": 0.3860313234468271,
-      "grad_norm": 3.28125,
-      "learning_rate": 3.5179905245826472e-06,
-      "loss": 1.7740880966186523,
-      "mean_token_accuracy": 0.6782984038814902,
-      "num_tokens": 6572801.0,
-      "step": 1160
-    },
-    {
-      "entropy": 1.4641494527459145,
-      "epoch": 0.3893591796834377,
-      "grad_norm": 3.65625,
-      "learning_rate": 3.493314953268403e-06,
-      "loss": 1.8116106033325194,
-      "mean_token_accuracy": 0.6800907859578729,
-      "num_tokens": 6628153.0,
-      "step": 1170
-    },
-    {
-      "entropy": 1.515749300457537,
-      "epoch": 0.3926870359200483,
-      "grad_norm": 3.21875,
-      "learning_rate": 3.468523929434008e-06,
-      "loss": 1.849051284790039,
-      "mean_token_accuracy": 0.6724825643002987,
-      "num_tokens": 6684775.0,
-      "step": 1180
-    },
-    {
-      "entropy": 1.4409945305436849,
-      "epoch": 0.39601489215665886,
-      "grad_norm": 4.09375,
-      "learning_rate": 3.4436203345282604e-06,
-      "loss": 1.7617074966430664,
-      "mean_token_accuracy": 0.6824791096150875,
-      "num_tokens": 6739059.0,
-      "step": 1190
-    },
-    {
-      "entropy": 1.552850889414549,
-      "epoch": 0.3993427483932694,
-      "grad_norm": 3.015625,
-      "learning_rate": 3.4186070630840406e-06,
-      "loss": 1.8339218139648437,
-      "mean_token_accuracy": 0.6552865024656057,
-      "num_tokens": 6798373.0,
-      "step": 1200
-    },
-    {
-      "entropy": 1.591016298532486,
-      "epoch": 0.40267060462987997,
-      "grad_norm": 3.015625,
-      "learning_rate": 3.3934870223818805e-06,
-      "loss": 1.8947530746459962,
-      "mean_token_accuracy": 0.653427678719163,
-      "num_tokens": 6859384.0,
-      "step": 1210
-    },
-    {
-      "entropy": 1.538698287308216,
-      "epoch": 0.40599846086649055,
-      "grad_norm": 3.515625,
-      "learning_rate": 3.3682631321120507e-06,
-      "loss": 1.8175443649291991,
-      "mean_token_accuracy": 0.6615260642021894,
-      "num_tokens": 6916445.0,
-      "step": 1220
-    },
-    {
-      "entropy": 1.5045099165290594,
-      "epoch": 0.40932631710310113,
-      "grad_norm": 3.609375,
-      "learning_rate": 3.3429383240352094e-06,
-      "loss": 1.8298114776611327,
-      "mean_token_accuracy": 0.6732751475647092,
-      "num_tokens": 6973165.0,
-      "step": 1230
-    },
-    {
-      "entropy": 1.4149478625506162,
-      "epoch": 0.4126541733397117,
-      "grad_norm": 3.59375,
-      "learning_rate": 3.31751554164164e-06,
-      "loss": 1.7360855102539063,
-      "mean_token_accuracy": 0.6919101564213633,
-      "num_tokens": 7025023.0,
-      "step": 1240
-    },
-    {
-      "entropy": 1.5094445595517754,
-      "epoch": 0.4159820295763223,
-      "grad_norm": 3.421875,
-      "learning_rate": 3.2919977398091358e-06,
-      "loss": 1.8317970275878905,
-      "mean_token_accuracy": 0.670648005977273,
-      "num_tokens": 7081622.0,
-      "step": 1250
-    },
-    {
-      "entropy": 1.4042024794965982,
-      "epoch": 0.4193098858129329,
-      "grad_norm": 3.96875,
-      "learning_rate": 3.2663878844595538e-06,
-      "loss": 1.7541933059692383,
-      "mean_token_accuracy": 0.6935789803043008,
-      "num_tokens": 7134028.0,
-      "step": 1260
-    },
-    {
-      "entropy": 1.4798556976020336,
-      "epoch": 0.42263774204954346,
-      "grad_norm": 3.515625,
-      "learning_rate": 3.2406889522140854e-06,
-      "loss": 1.7925809860229491,
-      "mean_token_accuracy": 0.6742974830791354,
-      "num_tokens": 7189458.0,
-      "step": 1270
-    },
-    {
-      "entropy": 1.530309046059847,
-      "epoch": 0.42596559828615405,
-      "grad_norm": 3.21875,
-      "learning_rate": 3.214903930047289e-06,
-      "loss": 1.7773765563964843,
-      "mean_token_accuracy": 0.6654345192015171,
-      "num_tokens": 7247590.0,
-      "step": 1280
-    },
-    {
-      "entropy": 1.4915742203593254,
-      "epoch": 0.42929345452276463,
-      "grad_norm": 4.09375,
-      "learning_rate": 3.1890358149399118e-06,
-      "loss": 1.8188709259033202,
-      "mean_token_accuracy": 0.6743956178426742,
-      "num_tokens": 7304322.0,
-      "step": 1290
-    },
-    {
-      "entropy": 1.4977670971304178,
-      "epoch": 0.4326213107593752,
-      "grad_norm": 3.78125,
-      "learning_rate": 3.1630876135305544e-06,
-      "loss": 1.8084901809692382,
-      "mean_token_accuracy": 0.6743098696693778,
-      "num_tokens": 7360953.0,
-      "step": 1300
-    },
-    {
-      "entropy": 1.4443225625902414,
-      "epoch": 0.4359491669959858,
-      "grad_norm": 3.34375,
-      "learning_rate": 3.1370623417662083e-06,
-      "loss": 1.8250289916992188,
-      "mean_token_accuracy": 0.68078558947891,
-      "num_tokens": 7415596.0,
-      "step": 1310
-    },
-    {
-      "entropy": 1.5016329048201442,
-      "epoch": 0.4392770232325964,
-      "grad_norm": 3.40625,
-      "learning_rate": 3.110963024551718e-06,
-      "loss": 1.815121841430664,
-      "mean_token_accuracy": 0.6725350238382817,
-      "num_tokens": 7471664.0,
-      "step": 1320
-    },
-    {
-      "entropy": 1.4518400821834803,
-      "epoch": 0.44260487946920696,
-      "grad_norm": 4.21875,
-      "learning_rate": 3.084792695398194e-06,
-      "loss": 1.7869695663452148,
-      "mean_token_accuracy": 0.6813382845371961,
-      "num_tokens": 7525747.0,
-      "step": 1330
-    },
-    {
-      "entropy": 1.5350855968892574,
-      "epoch": 0.4459327357058175,
-      "grad_norm": 3.25,
-      "learning_rate": 3.0585543960704282e-06,
-      "loss": 1.8438173294067384,
-      "mean_token_accuracy": 0.6637213431298733,
-      "num_tokens": 7585347.0,
-      "step": 1340
-    },
-    {
-      "entropy": 1.5509626347571612,
-      "epoch": 0.44926059194242807,
-      "grad_norm": 3.828125,
-      "learning_rate": 3.032251176233357e-06,
-      "loss": 1.8521547317504883,
-      "mean_token_accuracy": 0.6574134444817901,
-      "num_tokens": 7645260.0,
-      "step": 1350
-    },
-    {
-      "entropy": 1.5006514800712467,
-      "epoch": 0.45258844817903865,
-      "grad_norm": 3.28125,
-      "learning_rate": 3.005886093097593e-06,
-      "loss": 1.8141725540161133,
-      "mean_token_accuracy": 0.6761237462982536,
-      "num_tokens": 7702510.0,
-      "step": 1360
-    },
-    {
-      "entropy": 1.4634888933971524,
-      "epoch": 0.45591630441564923,
-      "grad_norm": 3.9375,
-      "learning_rate": 2.9794622110640927e-06,
-      "loss": 1.793300247192383,
-      "mean_token_accuracy": 0.6758374128490686,
-      "num_tokens": 7758905.0,
-      "step": 1370
-    },
-    {
-      "entropy": 1.4552174312993884,
-      "epoch": 0.4592441606522598,
-      "grad_norm": 3.0,
-      "learning_rate": 2.9529826013679785e-06,
-      "loss": 1.785028839111328,
-      "mean_token_accuracy": 0.6792163219302892,
-      "num_tokens": 7816865.0,
-      "step": 1380
-    },
-    {
-      "entropy": 1.4852726243436336,
-      "epoch": 0.4625720168888704,
-      "grad_norm": 3.5,
-      "learning_rate": 2.9264503417215747e-06,
-      "loss": 1.7959661483764648,
-      "mean_token_accuracy": 0.675879941880703,
-      "num_tokens": 7872611.0,
-      "step": 1390
-    },
-    {
-      "entropy": 1.509228605404496,
-      "epoch": 0.465899873125481,
-      "grad_norm": 3.46875,
-      "learning_rate": 2.8998685159566807e-06,
-      "loss": 1.8583616256713866,
-      "mean_token_accuracy": 0.6697035066783428,
-      "num_tokens": 7929995.0,
-      "step": 1400
-    },
-    {
-      "entropy": 1.4310267018154263,
-      "epoch": 0.46922772936209156,
-      "grad_norm": 3.375,
-      "learning_rate": 2.8732402136661448e-06,
-      "loss": 1.7277301788330077,
-      "mean_token_accuracy": 0.686691053956747,
-      "num_tokens": 7982764.0,
-      "step": 1410
-    },
-    {
-      "entropy": 1.5269542466849089,
-      "epoch": 0.47255558559870214,
-      "grad_norm": 3.53125,
-      "learning_rate": 2.8465685298447583e-06,
-      "loss": 1.8329839706420898,
-      "mean_token_accuracy": 0.6662309978157281,
-      "num_tokens": 8042070.0,
-      "step": 1420
-    },
-    {
-      "entropy": 1.468349908106029,
-      "epoch": 0.4758834418353127,
-      "grad_norm": 3.40625,
-      "learning_rate": 2.819856564529527e-06,
-      "loss": 1.7808021545410155,
-      "mean_token_accuracy": 0.6772725872695446,
-      "num_tokens": 8097634.0,
-      "step": 1430
-    },
-    {
-      "entropy": 1.465778378210962,
-      "epoch": 0.4792112980719233,
-      "grad_norm": 3.59375,
-      "learning_rate": 2.7931074224393573e-06,
-      "loss": 1.7484277725219726,
-      "mean_token_accuracy": 0.675658754259348,
-      "num_tokens": 8152889.0,
-      "step": 1440
-    },
-    {
-      "entropy": 1.4955711901187896,
-      "epoch": 0.4825391543085339,
-      "grad_norm": 3.1875,
-      "learning_rate": 2.76632421261419e-06,
-      "loss": 1.8247089385986328,
-      "mean_token_accuracy": 0.6732167150825262,
-      "num_tokens": 8208905.0,
-      "step": 1450
-    },
-    {
-      "entropy": 1.3527502533048392,
-      "epoch": 0.4858670105451445,
-      "grad_norm": 4.03125,
-      "learning_rate": 2.739510048053648e-06,
-      "loss": 1.735288429260254,
-      "mean_token_accuracy": 0.7055882178246975,
-      "num_tokens": 8258707.0,
-      "step": 1460
-    },
-    {
-      "entropy": 1.5091867126524448,
-      "epoch": 0.48919486678175506,
-      "grad_norm": 3.546875,
-      "learning_rate": 2.7126680453552036e-06,
-      "loss": 1.8230413436889648,
-      "mean_token_accuracy": 0.6701420545578003,
-      "num_tokens": 8316648.0,
-      "step": 1470
-    },
-    {
-      "entropy": 1.4676661742851138,
-      "epoch": 0.4925227230183656,
-      "grad_norm": 3.625,
-      "learning_rate": 2.6858013243519442e-06,
-      "loss": 1.769273567199707,
-      "mean_token_accuracy": 0.6794083835557103,
-      "num_tokens": 8371509.0,
-      "step": 1480
-    },
-    {
-      "entropy": 1.4840746331959962,
-      "epoch": 0.49585057925497616,
-      "grad_norm": 3.40625,
-      "learning_rate": 2.658913007749954e-06,
-      "loss": 1.8088008880615234,
-      "mean_token_accuracy": 0.6781815193593502,
-      "num_tokens": 8426946.0,
-      "step": 1490
-    },
-    {
-      "entropy": 1.5195502968505026,
-      "epoch": 0.49917843549158675,
-      "grad_norm": 3.5,
-      "learning_rate": 2.6320062207653646e-06,
-      "loss": 1.8746976852416992,
-      "mean_token_accuracy": 0.6683929346501827,
-      "num_tokens": 8485026.0,
-      "step": 1500
-    },
-    {
-      "entropy": 1.4764012772589923,
-      "epoch": 0.5025062917281974,
-      "grad_norm": 3.640625,
-      "learning_rate": 2.6050840907611073e-06,
-      "loss": 1.8286975860595702,
-      "mean_token_accuracy": 0.6774128306657076,
-      "num_tokens": 8543665.0,
-      "step": 1510
-    },
-    {
-      "entropy": 1.4877220762893557,
-      "epoch": 0.505834147964808,
-      "grad_norm": 3.3125,
-      "learning_rate": 2.5781497468834297e-06,
-      "loss": 1.7994909286499023,
-      "mean_token_accuracy": 0.6716964548453689,
-      "num_tokens": 8600486.0,
-      "step": 1520
-    },
-    {
-      "entropy": 1.4299600241705774,
-      "epoch": 0.5091620042014185,
-      "grad_norm": 3.5,
-      "learning_rate": 2.5512063196981906e-06,
-      "loss": 1.7645320892333984,
-      "mean_token_accuracy": 0.685977460630238,
-      "num_tokens": 8655398.0,
-      "step": 1530
-    },
-    {
-      "entropy": 1.5359071258455514,
-      "epoch": 0.5124898604380291,
-      "grad_norm": 3.109375,
-      "learning_rate": 2.5242569408269953e-06,
-      "loss": 1.8088735580444335,
-      "mean_token_accuracy": 0.6595347670838236,
-      "num_tokens": 8715507.0,
-      "step": 1540
-    },
-    {
-      "entropy": 1.4962305579334498,
-      "epoch": 0.5158177166746396,
-      "grad_norm": 3.546875,
-      "learning_rate": 2.497304742583213e-06,
-      "loss": 1.7858535766601562,
-      "mean_token_accuracy": 0.6688470123335719,
-      "num_tokens": 8772361.0,
-      "step": 1550
-    },
-    {
-      "entropy": 1.4240371275693178,
-      "epoch": 0.5191455729112502,
-      "grad_norm": 3.796875,
-      "learning_rate": 2.470352857607906e-06,
-      "loss": 1.7937036514282227,
-      "mean_token_accuracy": 0.6922440087422729,
-      "num_tokens": 8825262.0,
-      "step": 1560
-    },
-    {
-      "entropy": 1.428672723658383,
-      "epoch": 0.5224734291478608,
-      "grad_norm": 3.296875,
-      "learning_rate": 2.4434044185057264e-06,
-      "loss": 1.7313251495361328,
-      "mean_token_accuracy": 0.6897899683564901,
-      "num_tokens": 8878456.0,
-      "step": 1570
-    },
-    {
-      "entropy": 1.5070601791143416,
-      "epoch": 0.5258012853844714,
-      "grad_norm": 3.109375,
-      "learning_rate": 2.4164625574808145e-06,
-      "loss": 1.794417953491211,
-      "mean_token_accuracy": 0.6715059170499444,
-      "num_tokens": 8935456.0,
-      "step": 1580
-    },
-    {
-      "entropy": 1.4930913293734194,
-      "epoch": 0.5291291416210819,
-      "grad_norm": 3.34375,
-      "learning_rate": 2.3895304059727413e-06,
-      "loss": 1.8089893341064454,
-      "mean_token_accuracy": 0.6704512108117342,
-      "num_tokens": 8989152.0,
-      "step": 1590
-    },
-    {
-      "entropy": 1.4421082593500614,
-      "epoch": 0.5324569978576925,
-      "grad_norm": 3.359375,
-      "learning_rate": 2.3626110942925477e-06,
-      "loss": 1.773001480102539,
-      "mean_token_accuracy": 0.6869578551501035,
-      "num_tokens": 9044016.0,
-      "step": 1600
-    },
-    {
-      "entropy": 1.532307936809957,
-      "epoch": 0.5357848540943031,
-      "grad_norm": 3.6875,
-      "learning_rate": 2.3357077512589077e-06,
-      "loss": 1.8285242080688477,
-      "mean_token_accuracy": 0.6633320618420839,
-      "num_tokens": 9101768.0,
-      "step": 1610
-    },
-    {
-      "entropy": 1.4023213343694807,
-      "epoch": 0.5391127103309137,
-      "grad_norm": 4.125,
-      "learning_rate": 2.3088235038344664e-06,
-      "loss": 1.7209089279174805,
-      "mean_token_accuracy": 0.6950622322037816,
-      "num_tokens": 9154600.0,
-      "step": 1620
-    },
-    {
-      "entropy": 1.5615595361217856,
-      "epoch": 0.5424405665675243,
-      "grad_norm": 3.296875,
-      "learning_rate": 2.281961476762395e-06,
-      "loss": 1.883418083190918,
-      "mean_token_accuracy": 0.6599239958450198,
-      "num_tokens": 9215597.0,
-      "step": 1630
-    },
-    {
-      "entropy": 1.527036171220243,
-      "epoch": 0.5457684228041348,
-      "grad_norm": 3.34375,
-      "learning_rate": 2.255124792203203e-06,
-      "loss": 1.7833805084228516,
-      "mean_token_accuracy": 0.6661577256396413,
-      "num_tokens": 9273312.0,
-      "step": 1640
-    },
-    {
-      "entropy": 1.4240582948550582,
-      "epoch": 0.5490962790407454,
-      "grad_norm": 3.71875,
-      "learning_rate": 2.2283165693718565e-06,
-      "loss": 1.7378274917602539,
-      "mean_token_accuracy": 0.688914524205029,
-      "num_tokens": 9325847.0,
-      "step": 1650
-    },
-    {
-      "entropy": 1.3499843146651984,
-      "epoch": 0.552424135277356,
-      "grad_norm": 4.0625,
-      "learning_rate": 2.2015399241752215e-06,
-      "loss": 1.6975704193115235,
-      "mean_token_accuracy": 0.7067850837484002,
-      "num_tokens": 9375599.0,
-      "step": 1660
-    },
-    {
-      "entropy": 1.4400643074885011,
-      "epoch": 0.5557519915139666,
-      "grad_norm": 3.484375,
-      "learning_rate": 2.174797968849916e-06,
-      "loss": 1.7711650848388671,
-      "mean_token_accuracy": 0.6860870625823736,
-      "num_tokens": 9430213.0,
-      "step": 1670
-    },
-    {
-      "entropy": 1.4675764391198753,
-      "epoch": 0.5590798477505772,
-      "grad_norm": 3.34375,
-      "learning_rate": 2.148093811600569e-06,
-      "loss": 1.7572071075439453,
-      "mean_token_accuracy": 0.6749137572944164,
-      "num_tokens": 9485743.0,
-      "step": 1680
-    },
-    {
-      "entropy": 1.43440181016922,
-      "epoch": 0.5624077039871878,
-      "grad_norm": 3.9375,
-      "learning_rate": 2.1214305562385592e-06,
-      "loss": 1.7365966796875,
-      "mean_token_accuracy": 0.6852966235950589,
-      "num_tokens": 9542766.0,
-      "step": 1690
-    },
-    {
-      "entropy": 1.5254338281229138,
-      "epoch": 0.5657355602237983,
-      "grad_norm": 4.03125,
-      "learning_rate": 2.094811301821254e-06,
-      "loss": 1.815847396850586,
-      "mean_token_accuracy": 0.6700579663738608,
-      "num_tokens": 9602472.0,
-      "step": 1700
-    },
-    {
-      "entropy": 1.5060730464756489,
-      "epoch": 0.5690634164604089,
-      "grad_norm": 4.0625,
-      "learning_rate": 2.0682391422918167e-06,
-      "loss": 1.823427963256836,
-      "mean_token_accuracy": 0.6679550934582948,
-      "num_tokens": 9659605.0,
-      "step": 1710
-    },
-    {
-      "entropy": 1.5248936207965016,
-      "epoch": 0.5723912726970195,
-      "grad_norm": 3.4375,
-      "learning_rate": 2.0417171661195922e-06,
-      "loss": 1.8489822387695312,
-      "mean_token_accuracy": 0.6648134855553508,
-      "num_tokens": 9716525.0,
-      "step": 1720
-    },
-    {
-      "entropy": 1.5259105894714593,
-      "epoch": 0.5757191289336301,
-      "grad_norm": 3.734375,
-      "learning_rate": 2.0152484559411395e-06,
-      "loss": 1.8146631240844726,
-      "mean_token_accuracy": 0.6613813102245331,
-      "num_tokens": 9775917.0,
-      "step": 1730
-    },
-    {
-      "entropy": 1.4618713861331343,
-      "epoch": 0.5790469851702407,
-      "grad_norm": 3.234375,
-      "learning_rate": 1.9888360882019343e-06,
-      "loss": 1.758311653137207,
-      "mean_token_accuracy": 0.6800999123603105,
-      "num_tokens": 9831615.0,
-      "step": 1740
-    },
-    {
-      "entropy": 1.4584601072594523,
-      "epoch": 0.5823748414068513,
-      "grad_norm": 3.875,
-      "learning_rate": 1.962483132798799e-06,
-      "loss": 1.7423946380615234,
-      "mean_token_accuracy": 0.6758447799831628,
-      "num_tokens": 9889422.0,
-      "step": 1750
-    },
-    {
-      "entropy": 1.4273712508380414,
-      "epoch": 0.5857026976434618,
-      "grad_norm": 3.390625,
-      "learning_rate": 1.936192652723091e-06,
-      "loss": 1.7512628555297851,
-      "mean_token_accuracy": 0.6875329326838255,
-      "num_tokens": 9943113.0,
-      "step": 1760
-    },
-    {
-      "entropy": 1.4706622283905744,
-      "epoch": 0.5890305538800724,
-      "grad_norm": 3.296875,
-      "learning_rate": 1.9099677037046846e-06,
-      "loss": 1.7784095764160157,
-      "mean_token_accuracy": 0.6782323146238923,
-      "num_tokens": 9997979.0,
-      "step": 1770
-    },
-    {
-      "entropy": 1.5421956520527602,
-      "epoch": 0.592358410116683,
-      "grad_norm": 3.46875,
-      "learning_rate": 1.883811333856816e-06,
-      "loss": 1.8497350692749024,
-      "mean_token_accuracy": 0.6586372330784798,
-      "num_tokens": 10056742.0,
-      "step": 1780
-    },
-    {
-      "entropy": 1.5302059898152947,
-      "epoch": 0.5956862663532936,
-      "grad_norm": 3.359375,
-      "learning_rate": 1.8577265833217956e-06,
-      "loss": 1.8037921905517578,
-      "mean_token_accuracy": 0.6654811218380928,
-      "num_tokens": 10114501.0,
-      "step": 1790
-    },
-    {
-      "entropy": 1.4876639338210225,
-      "epoch": 0.5990141225899042,
-      "grad_norm": 3.265625,
-      "learning_rate": 1.8317164839176555e-06,
-      "loss": 1.7971521377563477,
-      "mean_token_accuracy": 0.6732841899618507,
-      "num_tokens": 10172296.0,
-      "step": 1800
-    },
-    {
-      "entropy": 1.5145339325070382,
-      "epoch": 0.6023419788265147,
-      "grad_norm": 3.46875,
-      "learning_rate": 1.8057840587857617e-06,
-      "loss": 1.815719223022461,
-      "mean_token_accuracy": 0.6730715375393629,
-      "num_tokens": 10227690.0,
-      "step": 1810
-    },
-    {
-      "entropy": 1.5149301951751113,
-      "epoch": 0.6056698350631253,
-      "grad_norm": 3.453125,
-      "learning_rate": 1.7799323220394383e-06,
-      "loss": 1.8125064849853516,
-      "mean_token_accuracy": 0.6658022409304977,
-      "num_tokens": 10285146.0,
-      "step": 1820
-    },
-    {
-      "entropy": 1.4355439454317094,
-      "epoch": 0.6089976912997358,
-      "grad_norm": 4.09375,
-      "learning_rate": 1.7541642784136368e-06,
-      "loss": 1.7578384399414062,
-      "mean_token_accuracy": 0.6822795016691089,
-      "num_tokens": 10340188.0,
-      "step": 1830
-    },
-    {
-      "entropy": 1.5701565401628614,
-      "epoch": 0.6123255475363464,
-      "grad_norm": 4.0625,
-      "learning_rate": 1.728482922915698e-06,
-      "loss": 1.8548479080200195,
-      "mean_token_accuracy": 0.6528071105480194,
-      "num_tokens": 10400455.0,
-      "step": 1840
-    },
-    {
-      "entropy": 1.5451415011659264,
-      "epoch": 0.615653403772957,
-      "grad_norm": 3.578125,
-      "learning_rate": 1.70289124047724e-06,
-      "loss": 1.832956886291504,
-      "mean_token_accuracy": 0.6592088233679533,
-      "num_tokens": 10457516.0,
-      "step": 1850
-    },
-    {
-      "entropy": 1.487533039972186,
-      "epoch": 0.6189812600095675,
-      "grad_norm": 3.84375,
-      "learning_rate": 1.6773922056072278e-06,
-      "loss": 1.7887454986572267,
-      "mean_token_accuracy": 0.6713353207334876,
-      "num_tokens": 10513956.0,
-      "step": 1860
-    },
-    {
-      "entropy": 1.4386489083990455,
-      "epoch": 0.6223091162461781,
-      "grad_norm": 3.53125,
-      "learning_rate": 1.6519887820462435e-06,
-      "loss": 1.7683986663818358,
-      "mean_token_accuracy": 0.6874082546681166,
-      "num_tokens": 10566931.0,
-      "step": 1870
-    },
-    {
-      "entropy": 1.5980463478714229,
-      "epoch": 0.6256369724827887,
-      "grad_norm": 3.328125,
-      "learning_rate": 1.6266839224220123e-06,
-      "loss": 1.8513069152832031,
-      "mean_token_accuracy": 0.6485659413039684,
-      "num_tokens": 10628414.0,
-      "step": 1880
-    },
-    {
-      "entropy": 1.5000085776671768,
-      "epoch": 0.6289648287193993,
-      "grad_norm": 3.640625,
-      "learning_rate": 1.6014805679062185e-06,
-      "loss": 1.770261573791504,
-      "mean_token_accuracy": 0.6704936826601624,
-      "num_tokens": 10684850.0,
-      "step": 1890
-    },
-    {
-      "entropy": 1.4550607884302735,
-      "epoch": 0.6322926849560099,
-      "grad_norm": 3.640625,
-      "learning_rate": 1.5763816478726585e-06,
-      "loss": 1.7998729705810548,
-      "mean_token_accuracy": 0.6798409396782518,
-      "num_tokens": 10741272.0,
-      "step": 1900
-    },
-    {
-      "entropy": 1.5604901734739542,
-      "epoch": 0.6356205411926205,
-      "grad_norm": 3.34375,
-      "learning_rate": 1.551390079556759e-06,
-      "loss": 1.8369234085083008,
-      "mean_token_accuracy": 0.6565213907510042,
-      "num_tokens": 10800575.0,
-      "step": 1910
-    },
-    {
-      "entropy": 1.4974081246182323,
-      "epoch": 0.638948397429231,
-      "grad_norm": 3.453125,
-      "learning_rate": 1.526508767716507e-06,
-      "loss": 1.7877006530761719,
-      "mean_token_accuracy": 0.6657007589936257,
-      "num_tokens": 10859226.0,
-      "step": 1920
-    },
-    {
-      "entropy": 1.4927168628200889,
-      "epoch": 0.6422762536658416,
-      "grad_norm": 3.4375,
-      "learning_rate": 1.5017406042948323e-06,
-      "loss": 1.7678947448730469,
-      "mean_token_accuracy": 0.6713365180417895,
-      "num_tokens": 10916627.0,
-      "step": 1930
-    },
-    {
-      "entropy": 1.4446895811706781,
-      "epoch": 0.6456041099024522,
-      "grad_norm": 3.25,
-      "learning_rate": 1.4770884680834797e-06,
-      "loss": 1.7734203338623047,
-      "mean_token_accuracy": 0.6822393707931041,
-      "num_tokens": 10971013.0,
-      "step": 1940
-    },
-    {
-      "entropy": 1.4686868280172347,
-      "epoch": 0.6489319661390628,
-      "grad_norm": 3.4375,
-      "learning_rate": 1.4525552243884062e-06,
-      "loss": 1.8040889739990233,
-      "mean_token_accuracy": 0.679110954143107,
-      "num_tokens": 11026128.0,
-      "step": 1950
-    },
-    {
-      "entropy": 1.546417790837586,
-      "epoch": 0.6522598223756734,
-      "grad_norm": 3.125,
-      "learning_rate": 1.428143724696748e-06,
-      "loss": 1.8216423034667968,
-      "mean_token_accuracy": 0.6596982540562749,
-      "num_tokens": 11086080.0,
-      "step": 1960
-    },
-    {
-      "entropy": 1.4961379690095782,
-      "epoch": 0.655587678612284,
-      "grad_norm": 3.65625,
-      "learning_rate": 1.4038568063453942e-06,
-      "loss": 1.8080915451049804,
-      "mean_token_accuracy": 0.6699457567185163,
-      "num_tokens": 11141382.0,
-      "step": 1970
-    },
-    {
-      "entropy": 1.543052814155817,
-      "epoch": 0.6589155348488945,
-      "grad_norm": 3.828125,
-      "learning_rate": 1.3796972921912071e-06,
-      "loss": 1.840565299987793,
-      "mean_token_accuracy": 0.66066266130656,
-      "num_tokens": 11201156.0,
-      "step": 1980
-    },
-    {
-      "entropy": 1.4301580367609859,
-      "epoch": 0.6622433910855051,
-      "grad_norm": 3.515625,
-      "learning_rate": 1.3556679902829172e-06,
-      "loss": 1.7641998291015626,
-      "mean_token_accuracy": 0.6893540635704994,
-      "num_tokens": 11254496.0,
-      "step": 1990
-    },
-    {
-      "entropy": 1.409593917056918,
-      "epoch": 0.6655712473221157,
-      "grad_norm": 4.40625,
-      "learning_rate": 1.3317716935347497e-06,
-      "loss": 1.7531818389892577,
-      "mean_token_accuracy": 0.6899214440956711,
-      "num_tokens": 11306517.0,
-      "step": 2000
-    },
-    {
-      "entropy": 1.5069324739277363,
-      "epoch": 0.6688991035587263,
-      "grad_norm": 3.4375,
-      "learning_rate": 1.3080111794018024e-06,
-      "loss": 1.8224695205688477,
-      "mean_token_accuracy": 0.6681538235396147,
-      "num_tokens": 11365379.0,
-      "step": 2010
-    },
-    {
-      "entropy": 1.5197775404900313,
-      "epoch": 0.6722269597953369,
-      "grad_norm": 3.53125,
-      "learning_rate": 1.2843892095572292e-06,
-      "loss": 1.8044471740722656,
-      "mean_token_accuracy": 0.6651798382401466,
-      "num_tokens": 11423634.0,
-      "step": 2020
-    },
-    {
-      "entropy": 1.4911304451525211,
-      "epoch": 0.6755548160319474,
-      "grad_norm": 3.0,
-      "learning_rate": 1.260908529571241e-06,
-      "loss": 1.7956329345703126,
-      "mean_token_accuracy": 0.6722222389653325,
-      "num_tokens": 11480435.0,
-      "step": 2030
-    },
-    {
-      "entropy": 1.5530314367264508,
-      "epoch": 0.678882672268558,
-      "grad_norm": 3.46875,
-      "learning_rate": 1.2375718685920024e-06,
-      "loss": 1.8273714065551758,
-      "mean_token_accuracy": 0.6560450242832303,
-      "num_tokens": 11539058.0,
-      "step": 2040
-    },
-    {
-      "entropy": 1.4723984507843852,
-      "epoch": 0.6822105285051686,
-      "grad_norm": 3.65625,
-      "learning_rate": 1.2143819390284141e-06,
-      "loss": 1.771543312072754,
-      "mean_token_accuracy": 0.6758286790922284,
-      "num_tokens": 11593265.0,
-      "step": 2050
-    },
-    {
-      "entropy": 1.5247776431962847,
-      "epoch": 0.6855383847417792,
-      "grad_norm": 3.296875,
-      "learning_rate": 1.1913414362348582e-06,
-      "loss": 1.7962211608886718,
-      "mean_token_accuracy": 0.6597121929749846,
-      "num_tokens": 11652327.0,
-      "step": 2060
-    },
-    {
-      "entropy": 1.4976553913205861,
-      "epoch": 0.6888662409783898,
-      "grad_norm": 3.296875,
-      "learning_rate": 1.1684530381979122e-06,
-      "loss": 1.8075973510742187,
-      "mean_token_accuracy": 0.6744270976632833,
-      "num_tokens": 11707212.0,
-      "step": 2070
-    },
-    {
-      "entropy": 1.5088494287803769,
-      "epoch": 0.6921940972150004,
-      "grad_norm": 3.296875,
-      "learning_rate": 1.145719405225091e-06,
-      "loss": 1.8094753265380858,
-      "mean_token_accuracy": 0.6734291119500995,
-      "num_tokens": 11763595.0,
-      "step": 2080
-    },
-    {
-      "entropy": 1.443124557286501,
-      "epoch": 0.6955219534516109,
-      "grad_norm": 3.21875,
-      "learning_rate": 1.1231431796356429e-06,
-      "loss": 1.7646524429321289,
-      "mean_token_accuracy": 0.6808732116594911,
-      "num_tokens": 11818713.0,
-      "step": 2090
-    },
-    {
-      "entropy": 1.43332000374794,
-      "epoch": 0.6988498096882215,
-      "grad_norm": 3.65625,
-      "learning_rate": 1.1007269854534287e-06,
-      "loss": 1.736590576171875,
-      "mean_token_accuracy": 0.6846986025571823,
-      "num_tokens": 11873101.0,
-      "step": 2100
-    },
-    {
-      "entropy": 1.4546937309205532,
-      "epoch": 0.702177665924832,
-      "grad_norm": 3.484375,
-      "learning_rate": 1.078473428101937e-06,
-      "loss": 1.754458236694336,
-      "mean_token_accuracy": 0.681413502804935,
-      "num_tokens": 11928452.0,
-      "step": 2110
-    },
-    {
-      "entropy": 1.5165076784789562,
-      "epoch": 0.7055055221614426,
-      "grad_norm": 3.859375,
-      "learning_rate": 1.0563850941014539e-06,
-      "loss": 1.853016471862793,
-      "mean_token_accuracy": 0.6662718730047346,
-      "num_tokens": 11987860.0,
-      "step": 2120
-    },
-    {
-      "entropy": 1.3975598519667982,
-      "epoch": 0.7088333783980532,
-      "grad_norm": 3.4375,
-      "learning_rate": 1.0344645507684395e-06,
-      "loss": 1.722806167602539,
-      "mean_token_accuracy": 0.6960443116724491,
-      "num_tokens": 12041289.0,
-      "step": 2130
-    },
-    {
-      "entropy": 1.5318202249705792,
-      "epoch": 0.7121612346346637,
-      "grad_norm": 3.34375,
-      "learning_rate": 1.0127143459171183e-06,
-      "loss": 1.7773944854736328,
-      "mean_token_accuracy": 0.6636665435507894,
-      "num_tokens": 12101524.0,
-      "step": 2140
-    },
-    {
-      "entropy": 1.5015214225277305,
-      "epoch": 0.7154890908712743,
-      "grad_norm": 3.296875,
-      "learning_rate": 9.911370075633606e-07,
-      "loss": 1.8127532958984376,
-      "mean_token_accuracy": 0.6707638284191489,
-      "num_tokens": 12158413.0,
-      "step": 2150
-    },
-    {
-      "entropy": 1.4262309791520238,
-      "epoch": 0.7188169471078849,
-      "grad_norm": 4.09375,
-      "learning_rate": 9.697350436308428e-07,
-      "loss": 1.7615140914916991,
-      "mean_token_accuracy": 0.6933442341163755,
-      "num_tokens": 12211468.0,
-      "step": 2160
-    },
-    {
-      "entropy": 1.5151672471314668,
-      "epoch": 0.7221448033444955,
-      "grad_norm": 3.328125,
-      "learning_rate": 9.485109416595603e-07,
-      "loss": 1.8634859085083009,
-      "mean_token_accuracy": 0.6674028748646379,
-      "num_tokens": 12268291.0,
-      "step": 2170
-    },
-    {
-      "entropy": 1.468200252763927,
-      "epoch": 0.7254726595811061,
-      "grad_norm": 3.140625,
-      "learning_rate": 9.27467168516695e-07,
-      "loss": 1.8100305557250977,
-      "mean_token_accuracy": 0.6783682005479932,
-      "num_tokens": 12324982.0,
-      "step": 2180
-    },
-    {
-      "entropy": 1.4821064189076423,
-      "epoch": 0.7288005158177167,
-      "grad_norm": 3.1875,
-      "learning_rate": 9.066061701098957e-07,
-      "loss": 1.7380363464355468,
-      "mean_token_accuracy": 0.6706728080287576,
-      "num_tokens": 12382329.0,
-      "step": 2190
-    },
-    {
-      "entropy": 1.466934666968882,
-      "epoch": 0.7321283720543272,
-      "grad_norm": 3.953125,
-      "learning_rate": 8.85930371102994e-07,
-      "loss": 1.7481739044189453,
-      "mean_token_accuracy": 0.6736409882083535,
-      "num_tokens": 12436981.0,
-      "step": 2200
-    },
-    {
-      "entropy": 1.5480544183403253,
-      "epoch": 0.7354562282909378,
-      "grad_norm": 3.65625,
-      "learning_rate": 8.65442174634182e-07,
-      "loss": 1.85827579498291,
-      "mean_token_accuracy": 0.6623124727979303,
-      "num_tokens": 12495304.0,
-      "step": 2210
-    },
-    {
-      "entropy": 1.4503806740045548,
-      "epoch": 0.7387840845275484,
-      "grad_norm": 3.75,
-      "learning_rate": 8.451439620366988e-07,
-      "loss": 1.7461252212524414,
-      "mean_token_accuracy": 0.679668728262186,
-      "num_tokens": 12550484.0,
-      "step": 2220
-    },
-    {
-      "entropy": 1.5361512675881386,
-      "epoch": 0.742111940764159,
-      "grad_norm": 3.65625,
-      "learning_rate": 8.250380925620496e-07,
-      "loss": 1.791541862487793,
-      "mean_token_accuracy": 0.661879819445312,
-      "num_tokens": 12610616.0,
-      "step": 2230
-    },
-    {
-      "entropy": 1.4404333807528018,
-      "epoch": 0.7454397970007696,
-      "grad_norm": 3.59375,
-      "learning_rate": 8.051269031057926e-07,
-      "loss": 1.7786466598510742,
-      "mean_token_accuracy": 0.6859370598569512,
-      "num_tokens": 12663688.0,
-      "step": 2240
-    },
-    {
-      "entropy": 1.5164645900949836,
-      "epoch": 0.7487676532373801,
-      "grad_norm": 3.5625,
-      "learning_rate": 7.854127079359183e-07,
-      "loss": 1.8197664260864257,
-      "mean_token_accuracy": 0.6693000530824065,
-      "num_tokens": 12721514.0,
-      "step": 2250
-    },
-    {
-      "entropy": 1.4675652343779801,
-      "epoch": 0.7520955094739907,
-      "grad_norm": 3.5625,
-      "learning_rate": 7.658977984238666e-07,
-      "loss": 1.7725656509399415,
-      "mean_token_accuracy": 0.6813359869644046,
-      "num_tokens": 12776789.0,
-      "step": 2260
-    },
-    {
-      "entropy": 1.509461098536849,
-      "epoch": 0.7554233657106013,
-      "grad_norm": 3.59375,
-      "learning_rate": 7.465844427782029e-07,
-      "loss": 1.7925626754760742,
-      "mean_token_accuracy": 0.6684816736727953,
-      "num_tokens": 12833157.0,
-      "step": 2270
-    },
-    {
-      "entropy": 1.4822339963167905,
-      "epoch": 0.7587512219472119,
-      "grad_norm": 3.140625,
-      "learning_rate": 7.274748857809808e-07,
-      "loss": 1.7800407409667969,
-      "mean_token_accuracy": 0.6704392759129405,
-      "num_tokens": 12891778.0,
-      "step": 2280
-    },
-    {
-      "entropy": 1.4534372324123979,
-      "epoch": 0.7620790781838225,
-      "grad_norm": 3.625,
-      "learning_rate": 7.085713485268364e-07,
-      "loss": 1.751553726196289,
-      "mean_token_accuracy": 0.679329103603959,
-      "num_tokens": 12949362.0,
-      "step": 2290
-    },
-    {
-      "entropy": 1.4593978630378843,
-      "epoch": 0.7654069344204331,
-      "grad_norm": 3.4375,
-      "learning_rate": 6.898760281648289e-07,
-      "loss": 1.8134092330932616,
-      "mean_token_accuracy": 0.6820162219926715,
-      "num_tokens": 13004104.0,
-      "step": 2300
-    },
-    {
-      "entropy": 1.519648071564734,
-      "epoch": 0.7687347906570436,
-      "grad_norm": 3.234375,
-      "learning_rate": 6.713910976430701e-07,
-      "loss": 1.8339576721191406,
-      "mean_token_accuracy": 0.6691967220976949,
-      "num_tokens": 13064698.0,
-      "step": 2310
-    },
-    {
-      "entropy": 1.4073020175099373,
-      "epoch": 0.7720626468936542,
-      "grad_norm": 3.375,
-      "learning_rate": 6.5311870545616e-07,
-      "loss": 1.7583913803100586,
-      "mean_token_accuracy": 0.6953523855656385,
-      "num_tokens": 13117531.0,
-      "step": 2320
-    },
-    {
-      "entropy": 1.507370169647038,
-      "epoch": 0.7753905031302648,
-      "grad_norm": 3.578125,
-      "learning_rate": 6.350609753954695e-07,
-      "loss": 1.8335344314575195,
-      "mean_token_accuracy": 0.6711913945153356,
-      "num_tokens": 13174183.0,
-      "step": 2330
-    },
-    {
-      "entropy": 1.4708391016349196,
-      "epoch": 0.7787183593668754,
-      "grad_norm": 3.671875,
-      "learning_rate": 6.172200063022946e-07,
-      "loss": 1.751777458190918,
-      "mean_token_accuracy": 0.6720907106995583,
-      "num_tokens": 13230535.0,
-      "step": 2340
-    },
-    {
-      "entropy": 1.4793819611892105,
-      "epoch": 0.782046215603486,
-      "grad_norm": 3.546875,
-      "learning_rate": 5.99597871823909e-07,
-      "loss": 1.8233556747436523,
-      "mean_token_accuracy": 0.6735257873311639,
-      "num_tokens": 13287454.0,
-      "step": 2350
-    },
-    {
-      "entropy": 1.5198968678712845,
-      "epoch": 0.7853740718400966,
-      "grad_norm": 3.328125,
-      "learning_rate": 5.821966201725427e-07,
-      "loss": 1.8299474716186523,
-      "mean_token_accuracy": 0.6705495104193687,
-      "num_tokens": 13343504.0,
-      "step": 2360
-    },
-    {
-      "entropy": 1.5564576109871269,
-      "epoch": 0.7887019280767071,
-      "grad_norm": 3.296875,
-      "learning_rate": 5.6501827388732e-07,
-      "loss": 1.8394550323486327,
-      "mean_token_accuracy": 0.6564194856211543,
-      "num_tokens": 13403107.0,
-      "step": 2370
-    },
-    {
-      "entropy": 1.4308181822299957,
-      "epoch": 0.7920297843133177,
-      "grad_norm": 3.671875,
-      "learning_rate": 5.480648295991847e-07,
-      "loss": 1.811262321472168,
-      "mean_token_accuracy": 0.6887904206290841,
-      "num_tokens": 13456531.0,
-      "step": 2380
-    },
-    {
-      "entropy": 1.5030910313129424,
-      "epoch": 0.7953576405499282,
-      "grad_norm": 3.15625,
-      "learning_rate": 5.313382577988263e-07,
-      "loss": 1.8046056747436523,
-      "mean_token_accuracy": 0.6701665591448546,
-      "num_tokens": 13513742.0,
-      "step": 2390
-    },
-    {
-      "entropy": 1.5541024476289749,
-      "epoch": 0.7986854967865388,
-      "grad_norm": 3.265625,
-      "learning_rate": 5.148405026076553e-07,
-      "loss": 1.8463783264160156,
-      "mean_token_accuracy": 0.6587805470451713,
-      "num_tokens": 13573921.0,
-      "step": 2400
-    },
-    {
-      "entropy": 1.5251292526721953,
-      "epoch": 0.8020133530231494,
-      "grad_norm": 3.8125,
-      "learning_rate": 4.985734815518367e-07,
-      "loss": 1.8205596923828125,
-      "mean_token_accuracy": 0.662408541701734,
-      "num_tokens": 13631596.0,
-      "step": 2410
-    },
-    {
-      "entropy": 1.4595861924812197,
-      "epoch": 0.8053412092597599,
-      "grad_norm": 3.328125,
-      "learning_rate": 4.825390853394196e-07,
-      "loss": 1.7567846298217773,
-      "mean_token_accuracy": 0.6789529614150525,
-      "num_tokens": 13687335.0,
-      "step": 2420
-    },
-    {
-      "entropy": 1.359525348804891,
-      "epoch": 0.8086690654963705,
-      "grad_norm": 3.671875,
-      "learning_rate": 4.6673917764057776e-07,
-      "loss": 1.6626811981201173,
-      "mean_token_accuracy": 0.7015537656843662,
-      "num_tokens": 13737424.0,
-      "step": 2430
-    },
-    {
-      "entropy": 1.4667940527200698,
-      "epoch": 0.8119969217329811,
-      "grad_norm": 3.46875,
-      "learning_rate": 4.5117559487099726e-07,
-      "loss": 1.7650856018066405,
-      "mean_token_accuracy": 0.6796885745599865,
-      "num_tokens": 13792250.0,
-      "step": 2440
-    },
-    {
-      "entropy": 1.4889131482690572,
-      "epoch": 0.8153247779695917,
-      "grad_norm": 3.4375,
-      "learning_rate": 4.358501459784317e-07,
-      "loss": 1.807198715209961,
-      "mean_token_accuracy": 0.67678750064224,
-      "num_tokens": 13848474.0,
-      "step": 2450
-    },
-    {
-      "entropy": 1.4557686105370522,
-      "epoch": 0.8186526342062023,
-      "grad_norm": 3.640625,
-      "learning_rate": 4.2076461223244936e-07,
-      "loss": 1.7964117050170898,
-      "mean_token_accuracy": 0.6834126306697726,
-      "num_tokens": 13902795.0,
-      "step": 2460
-    },
-    {
-      "entropy": 1.445817507430911,
-      "epoch": 0.8219804904428129,
-      "grad_norm": 3.390625,
-      "learning_rate": 4.0592074701739454e-07,
-      "loss": 1.7610958099365235,
-      "mean_token_accuracy": 0.6819076800718904,
-      "num_tokens": 13957785.0,
-      "step": 2470
-    },
-    {
-      "entropy": 1.5069481041282415,
-      "epoch": 0.8253083466794234,
-      "grad_norm": 3.5,
-      "learning_rate": 3.913202756285947e-07,
-      "loss": 1.8082542419433594,
-      "mean_token_accuracy": 0.6689409431070089,
-      "num_tokens": 14015241.0,
-      "step": 2480
-    },
-    {
-      "entropy": 1.3713191958144306,
-      "epoch": 0.828636202916034,
-      "grad_norm": 3.859375,
-      "learning_rate": 3.7696489507183134e-07,
-      "loss": 1.7333429336547852,
-      "mean_token_accuracy": 0.7018359253183007,
-      "num_tokens": 14064998.0,
-      "step": 2490
-    },
-    {
-      "entropy": 1.5192375615239144,
-      "epoch": 0.8319640591526446,
-      "grad_norm": 3.09375,
-      "learning_rate": 3.628562738660937e-07,
-      "loss": 1.7873783111572266,
-      "mean_token_accuracy": 0.6620162755250931,
-      "num_tokens": 14122159.0,
-      "step": 2500
-    },
-    {
-      "entropy": 1.4952821891754866,
-      "epoch": 0.8352919153892552,
-      "grad_norm": 3.546875,
-      "learning_rate": 3.489960518496521e-07,
-      "loss": 1.7901464462280274,
-      "mean_token_accuracy": 0.6704302851110697,
-      "num_tokens": 14177608.0,
-      "step": 2510
-    },
-    {
-      "entropy": 1.46396619156003,
-      "epoch": 0.8386197716258658,
-      "grad_norm": 3.578125,
-      "learning_rate": 3.3538583998945633e-07,
-      "loss": 1.7857032775878907,
-      "mean_token_accuracy": 0.678037466853857,
-      "num_tokens": 14233450.0,
-      "step": 2520
-    },
-    {
-      "entropy": 1.530667070671916,
-      "epoch": 0.8419476278624763,
-      "grad_norm": 4.4375,
-      "learning_rate": 3.2202722019389813e-07,
-      "loss": 1.8058586120605469,
-      "mean_token_accuracy": 0.6656222017481923,
-      "num_tokens": 14292176.0,
-      "step": 2530
-    },
-    {
-      "entropy": 1.5300720006227493,
-      "epoch": 0.8452754840990869,
-      "grad_norm": 4.34375,
-      "learning_rate": 3.0892174512894305e-07,
-      "loss": 1.8195119857788087,
-      "mean_token_accuracy": 0.6630910068750382,
-      "num_tokens": 14349684.0,
-      "step": 2540
-    },
-    {
-      "entropy": 1.5402492625638842,
-      "epoch": 0.8486033403356975,
-      "grad_norm": 3.859375,
-      "learning_rate": 2.9607093803766527e-07,
-      "loss": 1.8140392303466797,
-      "mean_token_accuracy": 0.6606448484584689,
-      "num_tokens": 14407203.0,
-      "step": 2550
-    },
-    {
-      "entropy": 1.4784167483448982,
-      "epoch": 0.8519311965723081,
-      "grad_norm": 3.46875,
-      "learning_rate": 2.8347629256320506e-07,
-      "loss": 1.7870410919189452,
-      "mean_token_accuracy": 0.6776479806751012,
-      "num_tokens": 14462897.0,
-      "step": 2560
-    },
-    {
-      "entropy": 1.5170332927256822,
-      "epoch": 0.8552590528089187,
-      "grad_norm": 3.65625,
-      "learning_rate": 2.711392725751583e-07,
-      "loss": 1.7928014755249024,
-      "mean_token_accuracy": 0.6670875972136855,
-      "num_tokens": 14519359.0,
-      "step": 2570
-    },
-    {
-      "entropy": 1.486384341493249,
-      "epoch": 0.8585869090455293,
-      "grad_norm": 3.5,
-      "learning_rate": 2.5906131199943676e-07,
-      "loss": 1.8128595352172852,
-      "mean_token_accuracy": 0.672566282376647,
-      "num_tokens": 14576677.0,
-      "step": 2580
-    },
-    {
-      "entropy": 1.4284999679774046,
-      "epoch": 0.8619147652821398,
-      "grad_norm": 3.5625,
-      "learning_rate": 2.472438146515982e-07,
-      "loss": 1.741770362854004,
-      "mean_token_accuracy": 0.6889012634754181,
-      "num_tokens": 14629187.0,
-      "step": 2590
-    },
-    {
-      "entropy": 1.5108024176210164,
-      "epoch": 0.8652426215187504,
-      "grad_norm": 3.3125,
-      "learning_rate": 2.3568815407368743e-07,
-      "loss": 1.7998302459716797,
-      "mean_token_accuracy": 0.6648341674357653,
-      "num_tokens": 14686862.0,
-      "step": 2600
-    },
-    {
-      "entropy": 1.4616851940751077,
-      "epoch": 0.868570477755361,
-      "grad_norm": 3.546875,
-      "learning_rate": 2.2439567337458473e-07,
-      "loss": 1.7760169982910157,
-      "mean_token_accuracy": 0.6819986145943403,
-      "num_tokens": 14741837.0,
-      "step": 2610
-    },
-    {
-      "entropy": 1.4263469209894537,
-      "epoch": 0.8718983339919716,
-      "grad_norm": 3.3125,
-      "learning_rate": 2.133676850739022e-07,
-      "loss": 1.7606956481933593,
-      "mean_token_accuracy": 0.6853868184611202,
-      "num_tokens": 14795649.0,
-      "step": 2620
-    },
-    {
-      "entropy": 1.4555895825847984,
-      "epoch": 0.8752261902285822,
-      "grad_norm": 3.5,
-      "learning_rate": 2.026054709494235e-07,
-      "loss": 1.769582176208496,
-      "mean_token_accuracy": 0.6829280275851488,
-      "num_tokens": 14851619.0,
-      "step": 2630
-    },
-    {
-      "entropy": 1.5214006565511227,
-      "epoch": 0.8785540464651928,
-      "grad_norm": 3.359375,
-      "learning_rate": 1.9211028188813126e-07,
-      "loss": 1.783592414855957,
-      "mean_token_accuracy": 0.661458052508533,
-      "num_tokens": 14909529.0,
-      "step": 2640
-    },
-    {
-      "entropy": 1.5377874106168747,
-      "epoch": 0.8818819027018033,
-      "grad_norm": 3.703125,
-      "learning_rate": 1.8188333774081106e-07,
-      "loss": 1.887862205505371,
-      "mean_token_accuracy": 0.6658550573512911,
-      "num_tokens": 14968625.0,
-      "step": 2650
-    },
-    {
-      "entropy": 1.4760029539465904,
-      "epoch": 0.8852097589384139,
-      "grad_norm": 3.875,
-      "learning_rate": 1.7192582718027096e-07,
-      "loss": 1.8009464263916015,
-      "mean_token_accuracy": 0.6779217705130577,
-      "num_tokens": 15023778.0,
-      "step": 2660
-    },
-    {
-      "entropy": 1.416380785778165,
-      "epoch": 0.8885376151750244,
-      "grad_norm": 3.859375,
-      "learning_rate": 1.6223890756318388e-07,
-      "loss": 1.7390851974487305,
-      "mean_token_accuracy": 0.6859621239826084,
-      "num_tokens": 15077436.0,
-      "step": 2670
-    },
-    {
-      "entropy": 1.4251615216955542,
-      "epoch": 0.891865471411635,
-      "grad_norm": 3.828125,
-      "learning_rate": 1.528237047955658e-07,
-      "loss": 1.7280912399291992,
-      "mean_token_accuracy": 0.6857763158157468,
-      "num_tokens": 15131479.0,
-      "step": 2680
-    },
-    {
-      "entropy": 1.4459022121503948,
-      "epoch": 0.8951933276482456,
-      "grad_norm": 3.3125,
-      "learning_rate": 1.4368131320191518e-07,
-      "loss": 1.7651968002319336,
-      "mean_token_accuracy": 0.6837036889046431,
-      "num_tokens": 15186064.0,
-      "step": 2690
-    },
-    {
-      "entropy": 1.5040705800056458,
-      "epoch": 0.8985211838848561,
-      "grad_norm": 3.84375,
-      "learning_rate": 1.348127953980169e-07,
-      "loss": 1.8105247497558594,
-      "mean_token_accuracy": 0.6697355670854449,
-      "num_tokens": 15244401.0,
-      "step": 2700
-    },
-    {
-      "entropy": 1.507199539989233,
-      "epoch": 0.9018490401214667,
-      "grad_norm": 3.15625,
-      "learning_rate": 1.2621918216743856e-07,
-      "loss": 1.7864213943481446,
-      "mean_token_accuracy": 0.6679751843214035,
-      "num_tokens": 15303435.0,
-      "step": 2710
-    },
-    {
-      "entropy": 1.4260680416598916,
-      "epoch": 0.9051768963580773,
-      "grad_norm": 4.84375,
-      "learning_rate": 1.1790147234171961e-07,
-      "loss": 1.770793342590332,
-      "mean_token_accuracy": 0.6893512340262532,
-      "num_tokens": 15358227.0,
-      "step": 2720
-    },
-    {
-      "entropy": 1.5067538300529122,
-      "epoch": 0.9085047525946879,
-      "grad_norm": 3.53125,
-      "learning_rate": 1.0986063268428076e-07,
-      "loss": 1.8035863876342773,
-      "mean_token_accuracy": 0.6674127342179418,
-      "num_tokens": 15416989.0,
-      "step": 2730
-    },
-    {
-      "entropy": 1.422794833034277,
-      "epoch": 0.9118326088312985,
-      "grad_norm": 3.796875,
-      "learning_rate": 1.0209759777805445e-07,
-      "loss": 1.7294546127319337,
-      "mean_token_accuracy": 0.6865978138521314,
-      "num_tokens": 15470829.0,
-      "step": 2740
-    },
-    {
-      "entropy": 1.38873875644058,
-      "epoch": 0.915160465067909,
-      "grad_norm": 3.140625,
-      "learning_rate": 9.461326991686143e-08,
-      "loss": 1.7182132720947265,
-      "mean_token_accuracy": 0.6963811507448554,
-      "num_tokens": 15524650.0,
-      "step": 2750
-    },
-    {
-      "entropy": 1.4680787263438106,
-      "epoch": 0.9184883213045196,
-      "grad_norm": 4.21875,
-      "learning_rate": 8.740851900053609e-08,
-      "loss": 1.8178903579711914,
-      "mean_token_accuracy": 0.6810888020321727,
-      "num_tokens": 15580034.0,
-      "step": 2760
-    },
-    {
-      "entropy": 1.5453741947188973,
-      "epoch": 0.9218161775411302,
-      "grad_norm": 3.109375,
-      "learning_rate": 8.04841824338179e-08,
-      "loss": 1.8256441116333009,
-      "mean_token_accuracy": 0.6548603283241391,
-      "num_tokens": 15642604.0,
-      "step": 2770
-    },
-    {
-      "entropy": 1.5282332230359317,
-      "epoch": 0.9251440337777408,
-      "grad_norm": 3.515625,
-      "learning_rate": 7.38410650290225e-08,
-      "loss": 1.8326656341552734,
-      "mean_token_accuracy": 0.6604917479678989,
-      "num_tokens": 15701356.0,
-      "step": 2780
-    },
-    {
-      "entropy": 1.3951038680970669,
-      "epoch": 0.9284718900143514,
-      "grad_norm": 6.125,
-      "learning_rate": 6.747993891249583e-08,
-      "loss": 1.6723697662353516,
-      "mean_token_accuracy": 0.6912660021334887,
-      "num_tokens": 15753812.0,
-      "step": 2790
-    },
-    {
-      "entropy": 1.4850552605465055,
-      "epoch": 0.931799746250962,
-      "grad_norm": 3.328125,
-      "learning_rate": 6.140154343487415e-08,
-      "loss": 1.7583208084106445,
-      "mean_token_accuracy": 0.6724124224856496,
-      "num_tokens": 15810988.0,
-      "step": 2800
-    },
-    {
-      "entropy": 1.3817319422960281,
-      "epoch": 0.9351276024875725,
-      "grad_norm": 4.15625,
-      "learning_rate": 5.560658508514394e-08,
-      "loss": 1.736399269104004,
-      "mean_token_accuracy": 0.6983843455091119,
-      "num_tokens": 15863426.0,
-      "step": 2810
-    },
-    {
-      "entropy": 1.4817607009783387,
-      "epoch": 0.9384554587241831,
-      "grad_norm": 4.09375,
-      "learning_rate": 5.009573740853313e-08,
-      "loss": 1.7787746429443358,
-      "mean_token_accuracy": 0.6699172344058753,
-      "num_tokens": 15921527.0,
-      "step": 2820
-    },
-    {
-      "entropy": 1.5170931756496429,
-      "epoch": 0.9417833149607937,
-      "grad_norm": 3.453125,
-      "learning_rate": 4.4869640928221216e-08,
-      "loss": 1.8043737411499023,
-      "mean_token_accuracy": 0.6683488871902228,
-      "num_tokens": 15978982.0,
-      "step": 2830
-    },
-    {
-      "entropy": 1.4666070902720094,
-      "epoch": 0.9451111711974043,
-      "grad_norm": 3.515625,
-      "learning_rate": 3.9928903070893255e-08,
-      "loss": 1.7780164718627929,
-      "mean_token_accuracy": 0.6797637686133384,
-      "num_tokens": 16033998.0,
-      "step": 2840
-    },
-    {
-      "entropy": 1.4543495135381819,
-      "epoch": 0.9484390274340149,
-      "grad_norm": 3.640625,
-      "learning_rate": 3.5274098096138855e-08,
-      "loss": 1.771834945678711,
-      "mean_token_accuracy": 0.6792650388553738,
-      "num_tokens": 16090592.0,
-      "step": 2850
-    },
-    {
-      "entropy": 1.4376623326912523,
-      "epoch": 0.9517668836706255,
-      "grad_norm": 3.78125,
-      "learning_rate": 3.090576702970499e-08,
-      "loss": 1.778950309753418,
-      "mean_token_accuracy": 0.685476124100387,
-      "num_tokens": 16143533.0,
-      "step": 2860
-    },
-    {
-      "entropy": 1.4532674498856069,
-      "epoch": 0.955094739907236,
-      "grad_norm": 3.34375,
-      "learning_rate": 2.682441760061494e-08,
-      "loss": 1.7271738052368164,
-      "mean_token_accuracy": 0.6771795397624374,
-      "num_tokens": 16198611.0,
-      "step": 2870
-    },
-    {
-      "entropy": 1.479434742219746,
-      "epoch": 0.9584225961438466,
-      "grad_norm": 3.671875,
-      "learning_rate": 2.3030524182152682e-08,
-      "loss": 1.771022415161133,
-      "mean_token_accuracy": 0.6740621255710721,
-      "num_tokens": 16255401.0,
-      "step": 2880
-    },
-    {
-      "entropy": 1.4560583192855119,
-      "epoch": 0.9617504523804572,
-      "grad_norm": 3.609375,
-      "learning_rate": 1.9524527736729805e-08,
-      "loss": 1.8014892578125,
-      "mean_token_accuracy": 0.6842205019667744,
-      "num_tokens": 16310435.0,
-      "step": 2890
-    },
-    {
-      "entropy": 1.4272220829501747,
-      "epoch": 0.9650783086170678,
-      "grad_norm": 3.703125,
-      "learning_rate": 1.630683576462927e-08,
-      "loss": 1.7681388854980469,
-      "mean_token_accuracy": 0.6838855624198914,
-      "num_tokens": 16365036.0,
-      "step": 2900
-    },
-    {
-      "entropy": 1.417688966728747,
-      "epoch": 0.9684061648536784,
-      "grad_norm": 3.234375,
-      "learning_rate": 1.3377822256644968e-08,
-      "loss": 1.7678020477294922,
-      "mean_token_accuracy": 0.6901428224518895,
-      "num_tokens": 16417240.0,
-      "step": 2910
-    },
-    {
-      "entropy": 1.4878744868561626,
-      "epoch": 0.971734021090289,
-      "grad_norm": 3.71875,
-      "learning_rate": 1.0737827650610099e-08,
-      "loss": 1.8392343521118164,
-      "mean_token_accuracy": 0.6748305330052972,
-      "num_tokens": 16474742.0,
-      "step": 2920
-    },
-    {
-      "entropy": 1.457695894315839,
-      "epoch": 0.9750618773268995,
-      "grad_norm": 3.234375,
-      "learning_rate": 8.387158791830497e-09,
-      "loss": 1.7888792037963868,
-      "mean_token_accuracy": 0.682488145492971,
-      "num_tokens": 16529245.0,
-      "step": 2930
-    },
-    {
-      "entropy": 1.4639849903061986,
-      "epoch": 0.9783897335635101,
-      "grad_norm": 3.4375,
-      "learning_rate": 6.326088897419258e-09,
-      "loss": 1.777825927734375,
-      "mean_token_accuracy": 0.6783558579161764,
-      "num_tokens": 16586436.0,
-      "step": 2940
-    },
-    {
-      "entropy": 1.4944805596023798,
-      "epoch": 0.9817175898001206,
-      "grad_norm": 3.53125,
-      "learning_rate": 4.554857524540768e-09,
-      "loss": 1.7782327651977539,
-      "mean_token_accuracy": 0.6676720147952437,
-      "num_tokens": 16643922.0,
-      "step": 2950
-    },
-    {
-      "entropy": 1.506484674848616,
-      "epoch": 0.9850454460367312,
-      "grad_norm": 3.890625,
-      "learning_rate": 3.0736705425671308e-09,
-      "loss": 1.7977352142333984,
-      "mean_token_accuracy": 0.6666848212480545,
-      "num_tokens": 16702267.0,
-      "step": 2960
-    },
-    {
-      "entropy": 1.4802768064662815,
-      "epoch": 0.9883733022733417,
-      "grad_norm": 3.78125,
-      "learning_rate": 1.882700109150093e-09,
-      "loss": 1.7765169143676758,
-      "mean_token_accuracy": 0.6719037894159555,
-      "num_tokens": 16759574.0,
-      "step": 2970
-    },
-    {
-      "entropy": 1.5233498934656382,
-      "epoch": 0.9917011585099523,
-      "grad_norm": 3.953125,
-      "learning_rate": 9.8208465021149e-10,
-      "loss": 1.8176689147949219,
-      "mean_token_accuracy": 0.6648949354887008,
-      "num_tokens": 16817165.0,
-      "step": 2980
-    },
-    {
-      "entropy": 1.481272677332163,
-      "epoch": 0.9950290147465629,
-      "grad_norm": 3.5625,
-      "learning_rate": 3.7192884385389706e-10,
-      "loss": 1.7591985702514648,
-      "mean_token_accuracy": 0.6735810926184058,
-      "num_tokens": 16872324.0,
-      "step": 2990
-    },
-    {
-      "entropy": 1.4698033582419157,
-      "epoch": 0.9983568709831735,
-      "grad_norm": 3.375,
-      "learning_rate": 5.230360819341585e-11,
-      "loss": 1.8078372955322266,
-      "mean_token_accuracy": 0.6768567437306047,
-      "num_tokens": 16929560.0,
-      "step": 3000
-    }
-  ],
-  "logging_steps": 10,
-  "max_steps": 3005,
-  "num_input_tokens_seen": 0,
-  "num_train_epochs": 1,
-  "save_steps": 500,
-  "stateful_callbacks": {
-    "TrainerControl": {
-      "args": {
-        "should_epoch_stop": false,
-        "should_evaluate": false,
-        "should_log": false,
-        "should_save": true,
-        "should_training_stop": true
-      },
-      "attributes": {}
-    }
-  },
-  "total_flos": 1.4337475997678592e+17,
-  "train_batch_size": 1,
-  "trial_name": null,
-  "trial_params": null
-}

checkpoint-3005/training_args.bin DELETED Viewed

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:17b9beb14ecfe12d9f9d1aaa211f12af32890a1644d822d745bed48747a3178b
-size 5777