Text Generation
Transformers
Safetensors
qwen2
Generated from Trainer
trl
grpo
conversational
text-generation-inference
Instructions to use hector-gr/RLCR-frontier-math with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hector-gr/RLCR-frontier-math with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="hector-gr/RLCR-frontier-math") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("hector-gr/RLCR-frontier-math") model = AutoModelForCausalLM.from_pretrained("hector-gr/RLCR-frontier-math", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use hector-gr/RLCR-frontier-math with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "hector-gr/RLCR-frontier-math" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/RLCR-frontier-math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/hector-gr/RLCR-frontier-math
- SGLang
How to use hector-gr/RLCR-frontier-math with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "hector-gr/RLCR-frontier-math" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/RLCR-frontier-math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "hector-gr/RLCR-frontier-math" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/RLCR-frontier-math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use hector-gr/RLCR-frontier-math with Docker Model Runner:
docker model run hf.co/hector-gr/RLCR-frontier-math
Download trainer_state.json from hector-gr/RLCR-frontier-math: direct link, hf CLI and curl.
- Browser
- Download file 63.5 kB
-
https://huggingface.co/hector-gr/RLCR-frontier-math/resolve/main/trainer_state.json
- Command line
-
hf download hf://hector-gr/RLCR-frontier-math/trainer_state.json
-
curl -L -o trainer_state.json https://huggingface.co/hector-gr/RLCR-frontier-math/resolve/main/trainer_state.json
63.5 kB
{
"best_metric": null,
"best_model_checkpoint": null,
"epoch": 0.49919376007799904,
"eval_steps": 50,
"global_step": 208,
"is_hyper_param_search": false,
"is_local_process_zero": true,
"is_world_process_zero": true,
"log_history": [
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.022743055555555558,
"completions/max_length": 4040.6,
"completions/max_terminated_length": 4040.6,
"completions/mean_length": 510.892626953125,
"completions/mean_terminated_length": 522.7780090332031,
"completions/min_length": 0.0,
"completions/min_terminated_length": 6.8,
"epoch": 0.011999850001874977,
"grad_norm": 0.005889138672500849,
"learning_rate": 5.952380952380953e-07,
"loss": 0.0095,
"num_tokens": 8999691.0,
"reward": 0.7351018190383911,
"reward_std": 0.725162398815155,
"rewards/accuracy_reward": 0.2590277761220932,
"rewards/brier_reward": 0.3106847941875458,
"rewards/format_reward": 0.5899305582046509,
"rewards/frontier_aurc_reward": 0.3106847941875458,
"rewards/frontier_coverage_1": 0.3106847941875458,
"rewards/frontier_coverage_10": 0.3106847941875458,
"rewards/frontier_coverage_15": 0.3106847941875458,
"rewards/frontier_coverage_20": 0.3106847941875458,
"rewards/frontier_coverage_25": 0.3106847941875458,
"rewards/frontier_coverage_5": 0.3106847941875458,
"step": 5
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.01649305555555556,
"completions/max_length": 4001.4,
"completions/max_terminated_length": 4001.4,
"completions/mean_length": 474.1986083984375,
"completions/mean_terminated_length": 482.28477172851564,
"completions/min_length": 0.0,
"completions/min_terminated_length": 30.8,
"epoch": 0.023999700003749954,
"grad_norm": 0.005823688581585884,
"learning_rate": 1.1904761904761906e-06,
"loss": 0.0005,
"num_tokens": 17545179.0,
"reward": 0.8662121772766114,
"reward_std": 0.6877874732017517,
"rewards/accuracy_reward": 0.2932291626930237,
"rewards/brier_reward": 0.3596000373363495,
"rewards/format_reward": 0.7201388955116272,
"rewards/frontier_aurc_reward": 0.3596000373363495,
"rewards/frontier_coverage_1": 0.3596000373363495,
"rewards/frontier_coverage_10": 0.3596000373363495,
"rewards/frontier_coverage_15": 0.3596000373363495,
"rewards/frontier_coverage_20": 0.3596000373363495,
"rewards/frontier_coverage_25": 0.3596000373363495,
"rewards/frontier_coverage_5": 0.3596000373363495,
"step": 10
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.009375,
"completions/max_length": 3710.6,
"completions/max_terminated_length": 3710.6,
"completions/mean_length": 433.11754150390624,
"completions/mean_terminated_length": 437.21246948242185,
"completions/min_length": 0.0,
"completions/min_terminated_length": 66.0,
"epoch": 0.03599955000562493,
"grad_norm": 0.005002312827855349,
"learning_rate": 1.7857142857142859e-06,
"loss": -0.0212,
"num_tokens": 25636677.0,
"reward": 1.0900377035140991,
"reward_std": 0.5645716547966003,
"rewards/accuracy_reward": 0.3383680522441864,
"rewards/brier_reward": 0.4504141747951508,
"rewards/format_reward": 0.9410590291023254,
"rewards/frontier_aurc_reward": 0.4504141747951508,
"rewards/frontier_coverage_1": 0.4504141747951508,
"rewards/frontier_coverage_10": 0.4504141747951508,
"rewards/frontier_coverage_15": 0.4504141747951508,
"rewards/frontier_coverage_20": 0.4504141747951508,
"rewards/frontier_coverage_25": 0.4504141747951508,
"rewards/frontier_coverage_5": 0.4504141747951508,
"step": 15
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.009548611111111095,
"completions/max_length": 3564.0,
"completions/max_terminated_length": 3564.0,
"completions/mean_length": 472.31806030273435,
"completions/mean_terminated_length": 476.92942504882814,
"completions/min_length": 0.0,
"completions/min_terminated_length": 91.4,
"epoch": 0.04799940000749991,
"grad_norm": 0.0009594348375685513,
"learning_rate": 2.380952380952381e-06,
"loss": -0.0102,
"num_tokens": 34191477.0,
"reward": 1.1244295120239258,
"reward_std": 0.3705786347389221,
"rewards/accuracy_reward": 0.44097223281860354,
"rewards/brier_reward": 0.5901519417762756,
"rewards/format_reward": 0.9860243082046509,
"rewards/frontier_aurc_reward": 0.1845359332859516,
"rewards/frontier_coverage_1": 0.23968117898330094,
"rewards/frontier_coverage_10": 0.23968117898330094,
"rewards/frontier_coverage_15": 0.23968117898330094,
"rewards/frontier_coverage_20": 0.23968117898330094,
"rewards/frontier_coverage_25": 0.23968117898330094,
"rewards/frontier_coverage_5": 0.23968117898330094,
"step": 20
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.011024305555555558,
"completions/max_length": 3814.6,
"completions/max_terminated_length": 3814.6,
"completions/mean_length": 515.2892333984375,
"completions/mean_terminated_length": 521.0623657226563,
"completions/min_length": 0.0,
"completions/min_terminated_length": 101.0,
"epoch": 0.05999925000937488,
"grad_norm": 0.0005871884641237557,
"learning_rate": 2.9761904761904763e-06,
"loss": -0.0103,
"num_tokens": 43252057.0,
"reward": 1.1103782415390016,
"reward_std": 0.21970458328723907,
"rewards/accuracy_reward": 0.5546007037162781,
"rewards/brier_reward": 0.7086271166801452,
"rewards/format_reward": 0.9869791746139527,
"rewards/frontier_aurc_reward": -0.057991190999746325,
"rewards/frontier_coverage_1": -0.02470763949677348,
"rewards/frontier_coverage_10": -0.02470763949677348,
"rewards/frontier_coverage_15": -0.02470763949677348,
"rewards/frontier_coverage_20": -0.02470763949677348,
"rewards/frontier_coverage_25": -0.02470763949677348,
"rewards/frontier_coverage_5": -0.02470763949677348,
"step": 25
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.013715277777777767,
"completions/max_length": 3851.0,
"completions/max_terminated_length": 3851.0,
"completions/mean_length": 584.6976684570312,
"completions/mean_terminated_length": 592.8659790039062,
"completions/min_length": 0.0,
"completions/min_terminated_length": 134.2,
"epoch": 0.07199910001124986,
"grad_norm": 0.0003692097670864314,
"learning_rate": 3.5714285714285718e-06,
"loss": -0.0129,
"num_tokens": 53097694.0,
"reward": 1.1326131582260133,
"reward_std": 0.16843081414699554,
"rewards/accuracy_reward": 0.5986979126930236,
"rewards/brier_reward": 0.7471577286720276,
"rewards/format_reward": 0.9848958373069763,
"rewards/frontier_aurc_reward": -0.0548021212220192,
"rewards/frontier_coverage_1": -0.0673428863286972,
"rewards/frontier_coverage_10": -0.0673428863286972,
"rewards/frontier_coverage_15": -0.0673428863286972,
"rewards/frontier_coverage_20": -0.0673428863286972,
"rewards/frontier_coverage_25": -0.0673428863286972,
"rewards/frontier_coverage_5": -0.0673428863286972,
"step": 30
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.018836805555555558,
"completions/max_length": 3889.2,
"completions/max_terminated_length": 3889.2,
"completions/mean_length": 600.4510620117187,
"completions/mean_terminated_length": 612.0839721679688,
"completions/min_length": 0.0,
"completions/min_terminated_length": 162.8,
"epoch": 0.08399895001312484,
"grad_norm": 0.0003851756628137082,
"learning_rate": 4.166666666666667e-06,
"loss": -0.0189,
"num_tokens": 63092330.0,
"reward": 1.142646861076355,
"reward_std": 0.15561008751392363,
"rewards/accuracy_reward": 0.6313368082046509,
"rewards/brier_reward": 0.7595993876457214,
"rewards/format_reward": 0.9797742962837219,
"rewards/frontier_aurc_reward": -0.04966666474938393,
"rewards/frontier_coverage_1": -0.09141493141651154,
"rewards/frontier_coverage_10": -0.09141493141651154,
"rewards/frontier_coverage_15": -0.09141493141651154,
"rewards/frontier_coverage_20": -0.09141493141651154,
"rewards/frontier_coverage_25": -0.09141493141651154,
"rewards/frontier_coverage_5": -0.09141493141651154,
"step": 35
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.01484375,
"completions/max_length": 3659.8,
"completions/max_terminated_length": 3659.8,
"completions/mean_length": 619.5973022460937,
"completions/mean_terminated_length": 628.9518432617188,
"completions/min_length": 0.0,
"completions/min_terminated_length": 174.8,
"epoch": 0.09599880001499982,
"grad_norm": 0.00045195635175332427,
"learning_rate": 4.761904761904762e-06,
"loss": -0.0171,
"num_tokens": 73349611.0,
"reward": 1.1597545862197876,
"reward_std": 0.15942782461643218,
"rewards/accuracy_reward": 0.6471354126930237,
"rewards/brier_reward": 0.7686648011207581,
"rewards/format_reward": 0.9844618082046509,
"rewards/frontier_aurc_reward": -0.04486135542392731,
"rewards/frontier_coverage_1": -0.08677257299423217,
"rewards/frontier_coverage_10": -0.08677257299423217,
"rewards/frontier_coverage_15": -0.08677257299423217,
"rewards/frontier_coverage_20": -0.08677257299423217,
"rewards/frontier_coverage_25": -0.08677257299423217,
"rewards/frontier_coverage_5": -0.08677257299423217,
"step": 40
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.01649305555555556,
"completions/max_length": 3682.2,
"completions/max_terminated_length": 3682.2,
"completions/mean_length": 641.230810546875,
"completions/mean_terminated_length": 652.0267211914063,
"completions/min_length": 0.0,
"completions/min_terminated_length": 204.8,
"epoch": 0.1079986500168748,
"grad_norm": 0.00045349443098530173,
"learning_rate": 4.909638554216868e-06,
"loss": -0.018,
"num_tokens": 83871854.0,
"reward": 1.1661339282989502,
"reward_std": 0.17443443536758424,
"rewards/accuracy_reward": 0.643055546283722,
"rewards/brier_reward": 0.763503897190094,
"rewards/format_reward": 0.9828125,
"rewards/frontier_aurc_reward": -0.03968145102262497,
"rewards/frontier_coverage_1": -0.060034720972180364,
"rewards/frontier_coverage_10": -0.060034720972180364,
"rewards/frontier_coverage_15": -0.060034720972180364,
"rewards/frontier_coverage_20": -0.060034720972180364,
"rewards/frontier_coverage_25": -0.060034720972180364,
"rewards/frontier_coverage_5": -0.060034720972180364,
"step": 45
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.012934027777777768,
"completions/max_length": 3546.0,
"completions/max_terminated_length": 3546.0,
"completions/mean_length": 661.9184814453125,
"completions/mean_terminated_length": 670.5972778320313,
"completions/min_length": 0.0,
"completions/min_terminated_length": 198.6,
"epoch": 0.11999850001874976,
"grad_norm": 0.0004998245276510715,
"learning_rate": 4.759036144578314e-06,
"loss": -0.0145,
"num_tokens": 94594755.0,
"reward": 1.1634884595870971,
"reward_std": 0.17470200955867768,
"rewards/accuracy_reward": 0.6353298664093018,
"rewards/brier_reward": 0.7527400135993958,
"rewards/format_reward": 0.9869791746139527,
"rewards/frontier_aurc_reward": -0.037785654515028,
"rewards/frontier_coverage_1": -0.04980902746319771,
"rewards/frontier_coverage_10": -0.04980902746319771,
"rewards/frontier_coverage_15": -0.04980902746319771,
"rewards/frontier_coverage_20": -0.04980902746319771,
"rewards/frontier_coverage_25": -0.04980902746319771,
"rewards/frontier_coverage_5": -0.04980902746319771,
"step": 50
},
{
"epoch": 0.11999850001874976,
"eval_completions/clipped_ratio": 0.01128472222222221,
"eval_completions/max_length": 1798.0,
"eval_completions/max_terminated_length": 1798.0,
"eval_completions/mean_length": 662.8635660807291,
"eval_completions/mean_terminated_length": 670.3329569498698,
"eval_completions/min_length": 107.33333333333333,
"eval_completions/min_terminated_length": 257.3333333333333,
"eval_loss": 0.0,
"eval_num_tokens": 94594755.0,
"eval_reward": 1.1788331270217896,
"eval_reward_std": 0.3053928663333257,
"eval_rewards/accuracy_reward": 0.6588541666666666,
"eval_rewards/brier_reward": 0.7654253244400024,
"eval_rewards/format_reward": 0.987847218910853,
"eval_rewards/frontier_aurc_reward": -0.03554386148850123,
"eval_rewards/frontier_coverage_1": -0.05763888948907455,
"eval_rewards/frontier_coverage_10": -0.05763888948907455,
"eval_rewards/frontier_coverage_15": -0.05763888948907455,
"eval_rewards/frontier_coverage_20": -0.05763888948907455,
"eval_rewards/frontier_coverage_25": -0.05763888948907455,
"eval_rewards/frontier_coverage_5": -0.05763888948907455,
"eval_runtime": 290.0417,
"eval_samples_per_second": 3.448,
"eval_steps_per_second": 0.021,
"step": 50
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.014409722222222254,
"completions/max_length": 3508.0,
"completions/max_terminated_length": 3508.0,
"completions/mean_length": 707.3890625,
"completions/mean_terminated_length": 717.869970703125,
"completions/min_length": 0.0,
"completions/min_terminated_length": 178.2,
"epoch": 0.13199835002062474,
"grad_norm": 0.0004837003943976015,
"learning_rate": 4.60843373493976e-06,
"loss": -0.0157,
"num_tokens": 105824453.0,
"reward": 1.1699814558029176,
"reward_std": 0.17824766933918,
"rewards/accuracy_reward": 0.64296875,
"rewards/brier_reward": 0.7596558570861817,
"rewards/format_reward": 0.9855034589767456,
"rewards/frontier_aurc_reward": -0.03552067577838898,
"rewards/frontier_coverage_1": -0.050295138359069826,
"rewards/frontier_coverage_10": -0.050295138359069826,
"rewards/frontier_coverage_15": -0.050295138359069826,
"rewards/frontier_coverage_20": -0.050295138359069826,
"rewards/frontier_coverage_25": -0.050295138359069826,
"rewards/frontier_coverage_5": -0.050295138359069826,
"step": 55
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.015798611111111138,
"completions/max_length": 3667.8,
"completions/max_terminated_length": 3667.8,
"completions/mean_length": 739.84150390625,
"completions/mean_terminated_length": 751.838671875,
"completions/min_length": 0.0,
"completions/min_terminated_length": 223.8,
"epoch": 0.14399820002249972,
"grad_norm": 0.000441178388427943,
"learning_rate": 4.457831325301205e-06,
"loss": -0.0194,
"num_tokens": 117444003.0,
"reward": 1.1630918502807617,
"reward_std": 0.17888002395629882,
"rewards/accuracy_reward": 0.6155381917953491,
"rewards/brier_reward": 0.7562003016471863,
"rewards/format_reward": 0.9842013835906982,
"rewards/frontier_aurc_reward": -0.03496500179171562,
"rewards/frontier_coverage_1": -0.028901910595595836,
"rewards/frontier_coverage_10": -0.028901910595595836,
"rewards/frontier_coverage_15": -0.028901910595595836,
"rewards/frontier_coverage_20": -0.028901910595595836,
"rewards/frontier_coverage_25": -0.028901910595595836,
"rewards/frontier_coverage_5": -0.028901910595595836,
"step": 60
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.014322916666666652,
"completions/max_length": 3344.6,
"completions/max_terminated_length": 3344.6,
"completions/mean_length": 725.77734375,
"completions/mean_terminated_length": 736.3476684570312,
"completions/min_length": 0.0,
"completions/min_terminated_length": 214.6,
"epoch": 0.1559980500243747,
"grad_norm": 0.0004953753086738288,
"learning_rate": 4.307228915662651e-06,
"loss": -0.0162,
"num_tokens": 128899006.0,
"reward": 1.1944715976715088,
"reward_std": 0.1739921510219574,
"rewards/accuracy_reward": 0.6505208373069763,
"rewards/brier_reward": 0.7868575096130371,
"rewards/format_reward": 0.9855902791023254,
"rewards/frontier_aurc_reward": -0.02906487099826336,
"rewards/frontier_coverage_1": -0.03486805595457554,
"rewards/frontier_coverage_10": -0.03486805595457554,
"rewards/frontier_coverage_15": -0.03486805595457554,
"rewards/frontier_coverage_20": -0.03486805595457554,
"rewards/frontier_coverage_25": -0.03486805595457554,
"rewards/frontier_coverage_5": -0.03486805595457554,
"step": 65
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.013802083333333326,
"completions/max_length": 2856.2,
"completions/max_terminated_length": 2856.2,
"completions/mean_length": 717.7317749023438,
"completions/mean_terminated_length": 727.8273193359375,
"completions/min_length": 0.0,
"completions/min_terminated_length": 198.8,
"epoch": 0.16799790002624967,
"grad_norm": 0.0005154712707735598,
"learning_rate": 4.156626506024097e-06,
"loss": -0.0182,
"num_tokens": 140245420.0,
"reward": 1.1966476678848266,
"reward_std": 0.184171262383461,
"rewards/accuracy_reward": 0.6315972328186035,
"rewards/brier_reward": 0.7812270045280456,
"rewards/format_reward": 0.9861979246139526,
"rewards/frontier_aurc_reward": -0.025832943990826605,
"rewards/frontier_coverage_1": -0.0023784732446074488,
"rewards/frontier_coverage_10": -0.0023784732446074488,
"rewards/frontier_coverage_15": -0.0023784732446074488,
"rewards/frontier_coverage_20": -0.0023784732446074488,
"rewards/frontier_coverage_25": -0.0023784732446074488,
"rewards/frontier_coverage_5": -0.0023784732446074488,
"step": 70
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.010503472222222232,
"completions/max_length": 3227.8,
"completions/max_terminated_length": 3227.8,
"completions/mean_length": 707.8462768554688,
"completions/mean_terminated_length": 715.4318603515625,
"completions/min_length": 0.0,
"completions/min_terminated_length": 208.6,
"epoch": 0.17999775002812465,
"grad_norm": 0.0005772764561697841,
"learning_rate": 4.006024096385543e-06,
"loss": -0.0123,
"num_tokens": 151464705.0,
"reward": 1.2308397054672242,
"reward_std": 0.18623916208744049,
"rewards/accuracy_reward": 0.6825520753860473,
"rewards/brier_reward": 0.8032798886299133,
"rewards/format_reward": 0.9894965291023254,
"rewards/frontier_aurc_reward": -0.0188261728733778,
"rewards/frontier_coverage_1": -0.012792536243796348,
"rewards/frontier_coverage_10": -0.012792536243796348,
"rewards/frontier_coverage_15": -0.012792536243796348,
"rewards/frontier_coverage_20": -0.012792536243796348,
"rewards/frontier_coverage_25": -0.012792536243796348,
"rewards/frontier_coverage_5": -0.012792536243796348,
"step": 75
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.01206597222222221,
"completions/max_length": 3769.6,
"completions/max_terminated_length": 3769.6,
"completions/mean_length": 757.0249267578125,
"completions/mean_terminated_length": 766.343994140625,
"completions/min_length": 0.0,
"completions/min_terminated_length": 228.4,
"epoch": 0.19199760002999963,
"grad_norm": 0.0006477182032540441,
"learning_rate": 3.855421686746989e-06,
"loss": -0.0172,
"num_tokens": 163238912.0,
"reward": 1.2179627895355225,
"reward_std": 0.20686974227428437,
"rewards/accuracy_reward": 0.647569453716278,
"rewards/brier_reward": 0.7879575967788697,
"rewards/format_reward": 0.9879340291023254,
"rewards/frontier_aurc_reward": -0.020410439744591714,
"rewards/frontier_coverage_1": 0.017949651886010543,
"rewards/frontier_coverage_10": 0.017949651886010543,
"rewards/frontier_coverage_15": 0.017949651886010543,
"rewards/frontier_coverage_20": 0.017949651886010543,
"rewards/frontier_coverage_25": 0.017949651886010543,
"rewards/frontier_coverage_5": 0.017949651886010543,
"step": 80
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.00859375,
"completions/max_length": 3865.4,
"completions/max_terminated_length": 3865.4,
"completions/mean_length": 755.281787109375,
"completions/mean_terminated_length": 761.8419311523437,
"completions/min_length": 0.0,
"completions/min_terminated_length": 251.8,
"epoch": 0.2039974500318746,
"grad_norm": 0.0007033683941699564,
"learning_rate": 3.7048192771084342e-06,
"loss": -0.0121,
"num_tokens": 175026958.0,
"reward": 1.2426313161849976,
"reward_std": 0.20636551976203918,
"rewards/accuracy_reward": 0.6682291626930237,
"rewards/brier_reward": 0.7987733721733093,
"rewards/format_reward": 0.991319453716278,
"rewards/frontier_aurc_reward": -0.012340086698532104,
"rewards/frontier_coverage_1": 0.03349999748170376,
"rewards/frontier_coverage_10": 0.03349999748170376,
"rewards/frontier_coverage_15": 0.03349999748170376,
"rewards/frontier_coverage_20": 0.03349999748170376,
"rewards/frontier_coverage_25": 0.03349999748170376,
"rewards/frontier_coverage_5": 0.03349999748170376,
"step": 85
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.009895833333333303,
"completions/max_length": 2590.8,
"completions/max_terminated_length": 2590.8,
"completions/mean_length": 726.2506103515625,
"completions/mean_terminated_length": 733.5089721679688,
"completions/min_length": 0.0,
"completions/min_terminated_length": 234.8,
"epoch": 0.2159973000337496,
"grad_norm": 0.0007190480828285217,
"learning_rate": 3.5542168674698798e-06,
"loss": -0.014,
"num_tokens": 186462037.0,
"reward": 1.2519195795059204,
"reward_std": 0.20773771405220032,
"rewards/accuracy_reward": 0.6611111164093018,
"rewards/brier_reward": 0.8078829288482666,
"rewards/format_reward": 0.9899305582046509,
"rewards/frontier_aurc_reward": -0.01144675500690937,
"rewards/frontier_coverage_1": 0.0543289914727211,
"rewards/frontier_coverage_10": 0.0543289914727211,
"rewards/frontier_coverage_15": 0.0543289914727211,
"rewards/frontier_coverage_20": 0.0543289914727211,
"rewards/frontier_coverage_25": 0.0543289914727211,
"rewards/frontier_coverage_5": 0.0543289914727211,
"step": 90
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.008680555555555535,
"completions/max_length": 2696.2,
"completions/max_terminated_length": 2696.2,
"completions/mean_length": 697.3243041992188,
"completions/mean_terminated_length": 703.4289916992187,
"completions/min_length": 0.0,
"completions/min_terminated_length": 191.8,
"epoch": 0.22799715003562457,
"grad_norm": 0.0007180027896538377,
"learning_rate": 3.4036144578313257e-06,
"loss": -0.0104,
"num_tokens": 197586893.0,
"reward": 1.2571552991867065,
"reward_std": 0.20568722784519194,
"rewards/accuracy_reward": 0.6302951335906982,
"rewards/brier_reward": 0.8152652025222779,
"rewards/format_reward": 0.9913194417953491,
"rewards/frontier_aurc_reward": -0.013813909329473972,
"rewards/frontier_coverage_1": 0.09267447590827942,
"rewards/frontier_coverage_10": 0.09267447590827942,
"rewards/frontier_coverage_15": 0.09267447590827942,
"rewards/frontier_coverage_20": 0.09267447590827942,
"rewards/frontier_coverage_25": 0.09267447590827942,
"rewards/frontier_coverage_5": 0.09267447590827942,
"step": 95
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0050347222222222095,
"completions/max_length": 3043.0,
"completions/max_terminated_length": 3043.0,
"completions/mean_length": 660.3557373046875,
"completions/mean_terminated_length": 663.6663452148438,
"completions/min_length": 0.0,
"completions/min_terminated_length": 196.8,
"epoch": 0.23999700003749952,
"grad_norm": 0.0007035965099930763,
"learning_rate": 3.2530120481927713e-06,
"loss": -0.0069,
"num_tokens": 208293263.0,
"reward": 1.293380045890808,
"reward_std": 0.20104076564311982,
"rewards/accuracy_reward": 0.61171875,
"rewards/brier_reward": 0.8441121816635132,
"rewards/format_reward": 0.9949652671813964,
"rewards/frontier_aurc_reward": -0.011699357628822326,
"rewards/frontier_coverage_1": 0.16053591966629027,
"rewards/frontier_coverage_10": 0.16053591966629027,
"rewards/frontier_coverage_15": 0.16053591966629027,
"rewards/frontier_coverage_20": 0.16053591966629027,
"rewards/frontier_coverage_25": 0.16114789843559266,
"rewards/frontier_coverage_5": 0.16053591966629027,
"step": 100
},
{
"epoch": 0.23999700003749952,
"eval_completions/clipped_ratio": 0.004340277777777772,
"eval_completions/max_length": 1796.8333333333333,
"eval_completions/max_terminated_length": 1796.8333333333333,
"eval_completions/mean_length": 625.5163370768229,
"eval_completions/mean_terminated_length": 628.2371012369791,
"eval_completions/min_length": 123.16666666666667,
"eval_completions/min_terminated_length": 218.83333333333334,
"eval_loss": 0.0,
"eval_num_tokens": 208293263.0,
"eval_reward": 1.2958853443463643,
"eval_reward_std": 0.3372383564710617,
"eval_rewards/accuracy_reward": 0.5668402910232544,
"eval_rewards/brier_reward": 0.840250680843989,
"eval_rewards/format_reward": 0.9939236044883728,
"eval_rewards/frontier_aurc_reward": -0.014694347822417816,
"eval_rewards/frontier_coverage_1": 0.22447916865348816,
"eval_rewards/frontier_coverage_10": 0.22447916865348816,
"eval_rewards/frontier_coverage_15": 0.22447916865348816,
"eval_rewards/frontier_coverage_20": 0.22447916865348816,
"eval_rewards/frontier_coverage_25": 0.22812499354283014,
"eval_rewards/frontier_coverage_5": 0.22447916865348816,
"eval_runtime": 258.0713,
"eval_samples_per_second": 3.875,
"eval_steps_per_second": 0.023,
"step": 100
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.002430555555555558,
"completions/max_length": 2970.2,
"completions/max_terminated_length": 2970.2,
"completions/mean_length": 602.419873046875,
"completions/mean_terminated_length": 603.8918823242187,
"completions/min_length": 30.4,
"completions/min_terminated_length": 161.0,
"epoch": 0.2519968500393745,
"grad_norm": 0.0008806795813143253,
"learning_rate": 3.1024096385542172e-06,
"loss": -0.0037,
"num_tokens": 218310004.0,
"reward": 1.3100762844085694,
"reward_std": 0.19428215622901918,
"rewards/accuracy_reward": 0.5513888835906983,
"rewards/brier_reward": 0.8520191311836243,
"rewards/format_reward": 0.997569453716278,
"rewards/frontier_aurc_reward": -0.01624637758359313,
"rewards/frontier_coverage_1": 0.2581909716129303,
"rewards/frontier_coverage_10": 0.2581909716129303,
"rewards/frontier_coverage_15": 0.2581909716129303,
"rewards/frontier_coverage_20": 0.2581909716129303,
"rewards/frontier_coverage_25": 0.26013107001781466,
"rewards/frontier_coverage_5": 0.2581909716129303,
"step": 105
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0002604166666666741,
"completions/max_length": 1787.0,
"completions/max_terminated_length": 1787.0,
"completions/mean_length": 566.1722290039063,
"completions/mean_terminated_length": 566.3225463867187,
"completions/min_length": 70.6,
"completions/min_terminated_length": 177.0,
"epoch": 0.2639967000412495,
"grad_norm": 0.00077627639984712,
"learning_rate": 2.9518072289156627e-06,
"loss": -0.0012,
"num_tokens": 227940756.0,
"reward": 1.3691948652267456,
"reward_std": 0.15197983980178834,
"rewards/accuracy_reward": 0.5227430522441864,
"rewards/brier_reward": 0.9113756537437439,
"rewards/format_reward": 0.9996527671813965,
"rewards/frontier_aurc_reward": -0.011038295924663544,
"rewards/frontier_coverage_1": 0.3568064272403717,
"rewards/frontier_coverage_10": 0.3568064272403717,
"rewards/frontier_coverage_15": 0.3568064272403717,
"rewards/frontier_coverage_20": 0.3568064272403717,
"rewards/frontier_coverage_25": 0.36018748879432677,
"rewards/frontier_coverage_5": 0.3568064272403717,
"step": 110
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.00017361111111111605,
"completions/max_length": 1882.2,
"completions/max_terminated_length": 1882.2,
"completions/mean_length": 521.3574035644531,
"completions/mean_terminated_length": 521.4490173339843,
"completions/min_length": 87.2,
"completions/min_terminated_length": 155.0,
"epoch": 0.27599655004312446,
"grad_norm": 0.0004159161471761763,
"learning_rate": 2.8012048192771087e-06,
"loss": -0.0005,
"num_tokens": 237025993.0,
"reward": 1.3876139879226685,
"reward_std": 0.11337690055370331,
"rewards/accuracy_reward": 0.2541666731238365,
"rewards/brier_reward": 0.9493119835853576,
"rewards/format_reward": 0.9998263835906982,
"rewards/frontier_aurc_reward": -0.026482035219669343,
"rewards/frontier_coverage_1": 0.6781017780303955,
"rewards/frontier_coverage_10": 0.6781017780303955,
"rewards/frontier_coverage_15": 0.6781017780303955,
"rewards/frontier_coverage_20": 0.6781017780303955,
"rewards/frontier_coverage_25": 0.6410357832908631,
"rewards/frontier_coverage_5": 0.6781017780303955,
"step": 115
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0003472222222222321,
"completions/max_length": 1527.0,
"completions/max_terminated_length": 1527.0,
"completions/mean_length": 479.53533935546875,
"completions/mean_terminated_length": 479.69696655273435,
"completions/min_length": 89.6,
"completions/min_terminated_length": 150.2,
"epoch": 0.28799640004499943,
"grad_norm": 0.0002834384795278311,
"learning_rate": 2.6506024096385547e-06,
"loss": -0.0001,
"num_tokens": 245632096.0,
"reward": 1.415544605255127,
"reward_std": 0.03439807705581188,
"rewards/accuracy_reward": 0.05911458320915699,
"rewards/brier_reward": 0.991649055480957,
"rewards/format_reward": 0.9996527791023254,
"rewards/frontier_aurc_reward": -0.0368982445448637,
"rewards/frontier_coverage_1": 0.928013014793396,
"rewards/frontier_coverage_10": 0.928013014793396,
"rewards/frontier_coverage_15": 0.928013014793396,
"rewards/frontier_coverage_20": 0.928013014793396,
"rewards/frontier_coverage_25": 0.863729190826416,
"rewards/frontier_coverage_5": 0.928013014793396,
"step": 120
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 8.680555555555803e-05,
"completions/max_length": 1358.8,
"completions/max_terminated_length": 1358.8,
"completions/mean_length": 434.4568603515625,
"completions/mean_terminated_length": 434.49669189453124,
"completions/min_length": 114.6,
"completions/min_terminated_length": 147.0,
"epoch": 0.2999962500468744,
"grad_norm": 0.00010355796257499605,
"learning_rate": 2.5e-06,
"loss": -0.0001,
"num_tokens": 253754703.0,
"reward": 1.4191264152526855,
"reward_std": 0.006818058248609304,
"rewards/accuracy_reward": 0.009114583302289248,
"rewards/brier_reward": 0.9989409923553467,
"rewards/format_reward": 0.9999131917953491,
"rewards/frontier_aurc_reward": -0.04175037369132042,
"rewards/frontier_coverage_1": 0.9891067385673523,
"rewards/frontier_coverage_10": 0.9891067385673523,
"rewards/frontier_coverage_15": 0.9891067385673523,
"rewards/frontier_coverage_20": 0.9891067385673523,
"rewards/frontier_coverage_25": 0.910530424118042,
"rewards/frontier_coverage_5": 0.9891067385673523,
"step": 125
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 8.680555555555803e-05,
"completions/max_length": 1311.2,
"completions/max_terminated_length": 1311.2,
"completions/mean_length": 392.66709594726564,
"completions/mean_terminated_length": 392.70135498046875,
"completions/min_length": 115.8,
"completions/min_terminated_length": 145.2,
"epoch": 0.3119961000487494,
"grad_norm": 2.0409556782396976e-06,
"learning_rate": 2.349397590361446e-06,
"loss": -0.0002,
"num_tokens": 261403028.0,
"reward": 1.4193828344345092,
"reward_std": 0.002322225472016726,
"rewards/accuracy_reward": 0.0015625000291038305,
"rewards/brier_reward": 0.9996683716773986,
"rewards/format_reward": 0.9999131917953491,
"rewards/frontier_aurc_reward": -0.044092994183301926,
"rewards/frontier_coverage_1": 0.9979808807373047,
"rewards/frontier_coverage_10": 0.9979808807373047,
"rewards/frontier_coverage_15": 0.9979808807373047,
"rewards/frontier_coverage_20": 0.9979808807373047,
"rewards/frontier_coverage_25": 0.9198862910270691,
"rewards/frontier_coverage_5": 0.9979808807373047,
"step": 130
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 8.680555555555803e-05,
"completions/max_length": 1178.0,
"completions/max_terminated_length": 1178.0,
"completions/mean_length": 361.4263916015625,
"completions/mean_terminated_length": 361.4570068359375,
"completions/min_length": 111.0,
"completions/min_terminated_length": 139.0,
"epoch": 0.32399595005062437,
"grad_norm": 6.781884280826489e-07,
"learning_rate": 2.1987951807228917e-06,
"loss": -0.0001,
"num_tokens": 268659684.0,
"reward": 1.419327688217163,
"reward_std": 0.0009700188855276792,
"rewards/accuracy_reward": 0.0006076388992369175,
"rewards/brier_reward": 0.9999122738838195,
"rewards/format_reward": 0.9999131917953491,
"rewards/frontier_aurc_reward": -0.04585757106542587,
"rewards/frontier_coverage_1": 0.999276876449585,
"rewards/frontier_coverage_10": 0.999276876449585,
"rewards/frontier_coverage_15": 0.999276876449585,
"rewards/frontier_coverage_20": 0.999276876449585,
"rewards/frontier_coverage_25": 0.9193767547607422,
"rewards/frontier_coverage_5": 0.999276876449585,
"step": 135
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0,
"completions/max_length": 1222.6,
"completions/max_terminated_length": 1222.6,
"completions/mean_length": 353.6166687011719,
"completions/mean_terminated_length": 353.6166687011719,
"completions/min_length": 135.4,
"completions/min_terminated_length": 135.4,
"epoch": 0.33599580005249935,
"grad_norm": 6.293203114182688e-07,
"learning_rate": 2.0481927710843377e-06,
"loss": 0.0,
"num_tokens": 275837572.0,
"reward": 1.4175257444381715,
"reward_std": 8.126303111648525e-05,
"rewards/accuracy_reward": 0.0001736111124046147,
"rewards/brier_reward": 0.999999725818634,
"rewards/format_reward": 1.0,
"rewards/frontier_aurc_reward": -0.047466978430747986,
"rewards/frontier_coverage_1": 0.9998168230056763,
"rewards/frontier_coverage_10": 0.9998168230056763,
"rewards/frontier_coverage_15": 0.9998168230056763,
"rewards/frontier_coverage_20": 0.9881675124168396,
"rewards/frontier_coverage_25": 0.9065182089805603,
"rewards/frontier_coverage_5": 0.9998168230056763,
"step": 140
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 8.680555555555803e-05,
"completions/max_length": 1009.0,
"completions/max_terminated_length": 1009.0,
"completions/mean_length": 345.4958435058594,
"completions/mean_terminated_length": 345.52516479492186,
"completions/min_length": 112.6,
"completions/min_terminated_length": 141.0,
"epoch": 0.34799565005437433,
"grad_norm": 2.6995683583663777e-05,
"learning_rate": 1.8975903614457832e-06,
"loss": -0.0002,
"num_tokens": 282882292.0,
"reward": 1.4068356275558471,
"reward_std": 0.0014974135039171443,
"rewards/accuracy_reward": 0.000260416668606922,
"rewards/brier_reward": 0.9998260855674743,
"rewards/format_reward": 0.9998263835906982,
"rewards/frontier_aurc_reward": -0.04886244907975197,
"rewards/frontier_coverage_1": 0.9995555400848388,
"rewards/frontier_coverage_10": 0.9995555400848388,
"rewards/frontier_coverage_15": 0.9945555329322815,
"rewards/frontier_coverage_20": 0.922927987575531,
"rewards/frontier_coverage_25": 0.8313003420829773,
"rewards/frontier_coverage_5": 0.9995555400848388,
"step": 145
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.00017361111111111605,
"completions/max_length": 1237.0,
"completions/max_terminated_length": 1237.0,
"completions/mean_length": 355.6992248535156,
"completions/mean_terminated_length": 355.759912109375,
"completions/min_length": 87.0,
"completions/min_terminated_length": 142.8,
"epoch": 0.3599955000562493,
"grad_norm": 3.2444859243696555e-05,
"learning_rate": 1.7469879518072292e-06,
"loss": -0.0003,
"num_tokens": 290090283.0,
"reward": 1.3922984600067139,
"reward_std": 0.0013689107200073104,
"rewards/accuracy_reward": 0.0,
"rewards/brier_reward": 0.9998263597488404,
"rewards/format_reward": 0.9998263835906982,
"rewards/frontier_aurc_reward": -0.05022518485784531,
"rewards/frontier_coverage_1": 0.9998237729072571,
"rewards/frontier_coverage_10": 0.9991570949554444,
"rewards/frontier_coverage_15": 0.9551658391952514,
"rewards/frontier_coverage_20": 0.8598541617393494,
"rewards/frontier_coverage_25": 0.733209204673767,
"rewards/frontier_coverage_5": 0.9998237729072571,
"step": 150
},
{
"epoch": 0.3599955000562493,
"eval_completions/clipped_ratio": 0.0,
"eval_completions/max_length": 774.0,
"eval_completions/max_terminated_length": 774.0,
"eval_completions/mean_length": 339.916498819987,
"eval_completions/mean_terminated_length": 339.916498819987,
"eval_completions/min_length": 166.0,
"eval_completions/min_terminated_length": 166.0,
"eval_loss": 0.0,
"eval_num_tokens": 290090283.0,
"eval_reward": 1.3871149619420369,
"eval_reward_std": 7.270483119251973e-08,
"eval_rewards/accuracy_reward": 0.0,
"eval_rewards/brier_reward": 1.0,
"eval_rewards/format_reward": 1.0,
"eval_rewards/frontier_aurc_reward": -0.050999817748864494,
"eval_rewards/frontier_coverage_1": 1.0,
"eval_rewards/frontier_coverage_10": 0.99249999721845,
"eval_rewards/frontier_coverage_15": 0.9469444354375204,
"eval_rewards/frontier_coverage_20": 0.8333333134651184,
"eval_rewards/frontier_coverage_25": 0.699999988079071,
"eval_rewards/frontier_coverage_5": 1.0,
"eval_runtime": 72.0507,
"eval_samples_per_second": 13.879,
"eval_steps_per_second": 0.083,
"step": 150
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0,
"completions/max_length": 1081.0,
"completions/max_terminated_length": 1081.0,
"completions/mean_length": 342.2134521484375,
"completions/mean_terminated_length": 342.2134521484375,
"completions/min_length": 139.4,
"completions/min_terminated_length": 139.4,
"epoch": 0.3719953500581243,
"grad_norm": 4.4718333924720355e-07,
"learning_rate": 1.5963855421686747e-06,
"loss": 0.0,
"num_tokens": 297140294.0,
"reward": 1.385084629058838,
"reward_std": 0.0007667588843105477,
"rewards/accuracy_reward": 8.680555620230735e-05,
"rewards/brier_reward": 0.99991295337677,
"rewards/format_reward": 0.9999131917953491,
"rewards/frontier_aurc_reward": -0.05209794640541077,
"rewards/frontier_coverage_1": 0.9991510272026062,
"rewards/frontier_coverage_10": 0.9874878644943237,
"rewards/frontier_coverage_15": 0.9431684494018555,
"rewards/frontier_coverage_20": 0.8248611330986023,
"rewards/frontier_coverage_25": 0.6932292103767395,
"rewards/frontier_coverage_5": 0.9981509923934937,
"step": 155
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0,
"completions/max_length": 947.6,
"completions/max_terminated_length": 947.6,
"completions/mean_length": 342.5266540527344,
"completions/mean_terminated_length": 342.5266540527344,
"completions/min_length": 134.2,
"completions/min_terminated_length": 134.2,
"epoch": 0.38399520005999926,
"grad_norm": 2.0079804485817476e-10,
"learning_rate": 1.4457831325301204e-06,
"loss": -0.0,
"num_tokens": 304173497.0,
"reward": 1.3827007293701172,
"reward_std": 6.615957584452659e-05,
"rewards/accuracy_reward": 8.680555620230735e-05,
"rewards/brier_reward": 0.9999999523162841,
"rewards/format_reward": 1.0,
"rewards/frontier_aurc_reward": -0.054189642518758775,
"rewards/frontier_coverage_1": 0.9979105591773987,
"rewards/frontier_coverage_10": 0.9879105687141418,
"rewards/frontier_coverage_15": 0.9405807018280029,
"rewards/frontier_coverage_20": 0.823261308670044,
"rewards/frontier_coverage_25": 0.6666293740272522,
"rewards/frontier_coverage_5": 0.997243857383728,
"step": 160
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0,
"completions/max_length": 1039.4,
"completions/max_terminated_length": 1039.4,
"completions/mean_length": 332.5473937988281,
"completions/mean_terminated_length": 332.5473937988281,
"completions/min_length": 140.4,
"completions/min_terminated_length": 140.4,
"epoch": 0.39599505006187424,
"grad_norm": 1.5090553806551554e-10,
"learning_rate": 1.2951807228915664e-06,
"loss": 0.0,
"num_tokens": 311143515.0,
"reward": 1.381039547920227,
"reward_std": 0.00016765535435769863,
"rewards/accuracy_reward": 0.0001736111124046147,
"rewards/brier_reward": 0.9999999284744263,
"rewards/format_reward": 1.0,
"rewards/frontier_aurc_reward": -0.056350186467170715,
"rewards/frontier_coverage_1": 0.9974886775016785,
"rewards/frontier_coverage_10": 0.9898272752761841,
"rewards/frontier_coverage_15": 0.9398481130599976,
"rewards/frontier_coverage_20": 0.818224823474884,
"rewards/frontier_coverage_25": 0.6499436140060425,
"rewards/frontier_coverage_5": 0.9964903950691223,
"step": 165
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0,
"completions/max_length": 1003.6,
"completions/max_terminated_length": 1003.6,
"completions/mean_length": 339.8690185546875,
"completions/mean_terminated_length": 339.8690185546875,
"completions/min_length": 139.6,
"completions/min_terminated_length": 139.6,
"epoch": 0.4079949000637492,
"grad_norm": 1.991925097089009e-10,
"learning_rate": 1.1445783132530121e-06,
"loss": 0.0,
"num_tokens": 318147990.0,
"reward": 1.3802492141723632,
"reward_std": 4.262864834103653e-06,
"rewards/accuracy_reward": 0.0,
"rewards/brier_reward": 0.9999999761581421,
"rewards/format_reward": 1.0,
"rewards/frontier_aurc_reward": -0.058704220503568647,
"rewards/frontier_coverage_1": 0.9969982266426086,
"rewards/frontier_coverage_10": 0.9876649498939514,
"rewards/frontier_coverage_15": 0.9439982652664185,
"rewards/frontier_coverage_20": 0.8199982881546021,
"rewards/frontier_coverage_25": 0.6399983167648315,
"rewards/frontier_coverage_5": 0.9956649422645569,
"step": 170
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0,
"completions/max_length": 1048.6,
"completions/max_terminated_length": 1048.6,
"completions/mean_length": 341.6985290527344,
"completions/mean_terminated_length": 341.6985290527344,
"completions/min_length": 142.0,
"completions/min_terminated_length": 142.0,
"epoch": 0.4199947500656242,
"grad_norm": 2.3663744286750443e-05,
"learning_rate": 9.93975903614458e-07,
"loss": 0.0,
"num_tokens": 325192325.0,
"reward": 1.3785095691680909,
"reward_std": 0.0009700911837066783,
"rewards/accuracy_reward": 0.0003472222248092294,
"rewards/brier_reward": 0.9999115943908692,
"rewards/format_reward": 0.9999131917953491,
"rewards/frontier_aurc_reward": -0.0610056534409523,
"rewards/frontier_coverage_1": 0.9968766808509827,
"rewards/frontier_coverage_10": 0.9872161626815796,
"rewards/frontier_coverage_15": 0.9429158091545105,
"rewards/frontier_coverage_20": 0.8213359475135803,
"rewards/frontier_coverage_25": 0.6165008902549743,
"rewards/frontier_coverage_5": 0.9962100148200989,
"step": 175
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.00017361111111111605,
"completions/max_length": 993.0,
"completions/max_terminated_length": 993.0,
"completions/mean_length": 334.0981811523437,
"completions/mean_terminated_length": 334.15501708984374,
"completions/min_length": 83.8,
"completions/min_terminated_length": 141.0,
"epoch": 0.4319946000674992,
"grad_norm": 2.9501152312150225e-05,
"learning_rate": 8.433734939759036e-07,
"loss": -0.0003,
"num_tokens": 332141104.0,
"reward": 1.3708134651184083,
"reward_std": 0.0024533939658795134,
"rewards/accuracy_reward": 0.0001736111124046147,
"rewards/brier_reward": 0.9995822429656982,
"rewards/format_reward": 0.9997395753860474,
"rewards/frontier_aurc_reward": -0.06325942426919937,
"rewards/frontier_coverage_1": 0.9967308521270752,
"rewards/frontier_coverage_10": 0.9870703458786011,
"rewards/frontier_coverage_15": 0.9441032767295837,
"rewards/frontier_coverage_20": 0.7862335205078125,
"rewards/frontier_coverage_25": 0.5497231125831604,
"rewards/frontier_coverage_5": 0.9963975191116333,
"step": 180
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0,
"completions/max_length": 1000.8,
"completions/max_terminated_length": 1000.8,
"completions/mean_length": 337.2212707519531,
"completions/mean_terminated_length": 337.2212707519531,
"completions/min_length": 140.0,
"completions/min_terminated_length": 140.0,
"epoch": 0.44399445006937416,
"grad_norm": 1.91206308954861e-05,
"learning_rate": 6.927710843373495e-07,
"loss": 0.0,
"num_tokens": 339115941.0,
"reward": 1.362591028213501,
"reward_std": 0.0003789564379985677,
"rewards/accuracy_reward": 0.0003472222248092294,
"rewards/brier_reward": 0.9999996304512024,
"rewards/format_reward": 1.0,
"rewards/frontier_aurc_reward": -0.06550476551055909,
"rewards/frontier_coverage_1": 0.9969739079475403,
"rewards/frontier_coverage_10": 0.9876545786857605,
"rewards/frontier_coverage_15": 0.9476753830909729,
"rewards/frontier_coverage_20": 0.7564635753631592,
"rewards/frontier_coverage_25": 0.45663716793060305,
"rewards/frontier_coverage_5": 0.9959774136543273,
"step": 185
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 8.680555555555803e-05,
"completions/max_length": 1036.0,
"completions/max_terminated_length": 1036.0,
"completions/mean_length": 332.005908203125,
"completions/mean_terminated_length": 332.03541870117186,
"completions/min_length": 114.6,
"completions/min_terminated_length": 138.4,
"epoch": 0.45599430007124914,
"grad_norm": 7.611962473674794e-07,
"learning_rate": 5.421686746987952e-07,
"loss": -0.0002,
"num_tokens": 346023593.0,
"reward": 1.3553855419158936,
"reward_std": 0.002256077638046783,
"rewards/accuracy_reward": 0.0001736111124046147,
"rewards/brier_reward": 0.9997395157814026,
"rewards/format_reward": 0.9997395753860474,
"rewards/frontier_aurc_reward": -0.06776920258998871,
"rewards/frontier_coverage_1": 0.996229088306427,
"rewards/frontier_coverage_10": 0.9895642757415771,
"rewards/frontier_coverage_15": 0.9479201674461365,
"rewards/frontier_coverage_20": 0.7363819718360901,
"rewards/frontier_coverage_25": 0.3815989553928375,
"rewards/frontier_coverage_5": 0.9958957672119141,
"step": 190
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0,
"completions/max_length": 1096.2,
"completions/max_terminated_length": 1096.2,
"completions/mean_length": 343.455029296875,
"completions/mean_terminated_length": 343.455029296875,
"completions/min_length": 134.4,
"completions/min_terminated_length": 134.4,
"epoch": 0.46799415007312406,
"grad_norm": 1.7637200033782108e-10,
"learning_rate": 3.91566265060241e-07,
"loss": -0.0,
"num_tokens": 353061059.0,
"reward": 1.3460167407989503,
"reward_std": 0.0009337843641445964,
"rewards/accuracy_reward": 0.00026041667442768814,
"rewards/brier_reward": 0.9999126553535461,
"rewards/format_reward": 0.9999131917953491,
"rewards/frontier_aurc_reward": -0.07003692388534546,
"rewards/frontier_coverage_1": 0.9963098168373108,
"rewards/frontier_coverage_10": 0.9886466979980468,
"rewards/frontier_coverage_15": 0.9443359375,
"rewards/frontier_coverage_20": 0.6798906445503234,
"rewards/frontier_coverage_25": 0.3101119935512543,
"rewards/frontier_coverage_5": 0.9963098168373108,
"step": 195
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0,
"completions/max_length": 1046.6,
"completions/max_terminated_length": 1046.6,
"completions/mean_length": 333.68116455078126,
"completions/mean_terminated_length": 333.68116455078126,
"completions/min_length": 137.6,
"completions/min_terminated_length": 137.6,
"epoch": 0.47999400007499904,
"grad_norm": 5.642870632982522e-07,
"learning_rate": 2.409638554216868e-07,
"loss": -0.0,
"num_tokens": 359972874.0,
"reward": 1.3373116731643677,
"reward_std": 0.00012597611311280233,
"rewards/accuracy_reward": 8.680555620230735e-05,
"rewards/brier_reward": 0.9999999523162841,
"rewards/format_reward": 1.0,
"rewards/frontier_aurc_reward": -0.07235789746046066,
"rewards/frontier_coverage_1": 0.9975763559341431,
"rewards/frontier_coverage_10": 0.9865799188613892,
"rewards/frontier_coverage_15": 0.9439270853996277,
"rewards/frontier_coverage_20": 0.6216458439826965,
"rewards/frontier_coverage_25": 0.25003125369548795,
"rewards/frontier_coverage_5": 0.9962430000305176,
"step": 200
},
{
"epoch": 0.47999400007499904,
"eval_completions/clipped_ratio": 0.0,
"eval_completions/max_length": 851.3333333333334,
"eval_completions/max_terminated_length": 851.3333333333334,
"eval_completions/mean_length": 339.9757080078125,
"eval_completions/mean_terminated_length": 339.9757080078125,
"eval_completions/min_length": 156.16666666666666,
"eval_completions/min_terminated_length": 156.16666666666666,
"eval_loss": 0.0,
"eval_num_tokens": 359972874.0,
"eval_reward": 1.3312063217163086,
"eval_reward_std": 1.1017274677745566e-07,
"eval_rewards/accuracy_reward": 0.0,
"eval_rewards/brier_reward": 1.0,
"eval_rewards/format_reward": 1.0,
"eval_rewards/frontier_aurc_reward": -0.07375634461641312,
"eval_rewards/frontier_coverage_1": 0.9969444374243418,
"eval_rewards/frontier_coverage_10": 0.9875000417232513,
"eval_rewards/frontier_coverage_15": 0.9461111426353455,
"eval_rewards/frontier_coverage_20": 0.5694444378217062,
"eval_rewards/frontier_coverage_25": 0.21666668355464935,
"eval_rewards/frontier_coverage_5": 0.9958333671092987,
"eval_runtime": 73.0767,
"eval_samples_per_second": 13.684,
"eval_steps_per_second": 0.082,
"step": 200
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 8.680555555555803e-05,
"completions/max_length": 988.8,
"completions/max_terminated_length": 988.8,
"completions/mean_length": 336.18073120117185,
"completions/mean_terminated_length": 336.2108520507812,
"completions/min_length": 111.0,
"completions/min_terminated_length": 139.8,
"epoch": 0.491993850076874,
"grad_norm": 2.1033699781902726e-10,
"learning_rate": 9.036144578313253e-08,
"loss": -0.0001,
"num_tokens": 366911628.0,
"reward": 1.3294225931167603,
"reward_std": 0.0013062187141763104,
"rewards/accuracy_reward": 0.00034722223062999547,
"rewards/brier_reward": 0.9997871875762939,
"rewards/format_reward": 0.9999131917953491,
"rewards/frontier_aurc_reward": -0.07490274161100388,
"rewards/frontier_coverage_1": 0.9970815539360046,
"rewards/frontier_coverage_10": 0.9877586960792542,
"rewards/frontier_coverage_15": 0.9437821507453918,
"rewards/frontier_coverage_20": 0.548003488779068,
"rewards/frontier_coverage_25": 0.21662326157093048,
"rewards/frontier_coverage_5": 0.9950815677642822,
"step": 205
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.0,
"completions/max_length": 1061.3333333333333,
"completions/max_terminated_length": 1061.3333333333333,
"completions/mean_length": 335.6025797526042,
"completions/mean_terminated_length": 335.6025797526042,
"completions/min_length": 135.0,
"completions/min_terminated_length": 135.0,
"epoch": 0.49919376007799904,
"num_tokens": 371097361.0,
"reward": 1.3221978346506755,
"reward_std": 7.101231412824187e-06,
"rewards/accuracy_reward": 0.0,
"rewards/brier_reward": 1.0,
"rewards/format_reward": 1.0,
"rewards/frontier_aurc_reward": -0.07685168584187825,
"rewards/frontier_coverage_1": 0.9966637094815572,
"rewards/frontier_coverage_10": 0.9888860384623209,
"rewards/frontier_coverage_15": 0.9472193519274393,
"rewards/frontier_coverage_20": 0.4555526375770569,
"rewards/frontier_coverage_25": 0.20555266737937927,
"rewards/frontier_coverage_5": 0.995552639166514,
"step": 208,
"total_flos": 0.0,
"train_loss": -0.006311835187304335,
"train_runtime": 56803.1734,
"train_samples_per_second": 0.264,
"train_steps_per_second": 0.004
}
],
"logging_steps": 5,
"max_steps": 208,
"num_input_tokens_seen": 371097361,
"num_train_epochs": 1,
"save_steps": 60,
"stateful_callbacks": {
"TrainerControl": {
"args": {
"should_epoch_stop": false,
"should_evaluate": false,
"should_log": false,
"should_save": true,
"should_training_stop": true
},
"attributes": {}
}
},
"total_flos": 0.0,
"train_batch_size": 6,
"trial_name": null,
"trial_params": null
}
{
"best_metric": null,
"best_model_checkpoint": null,
"epoch": 0.49919376007799904,
"eval_steps": 50,
"global_step": 208,
"is_hyper_param_search": false,
"is_local_process_zero": true,
"is_world_process_zero": true,
"log_history": [
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.022743055555555558,
"completions/max_length": 4040.6,
"completions/max_terminated_length": 4040.6,
"completions/mean_length": 510.892626953125,
"completions/mean_terminated_length": 522.7780090332031,
"completions/min_length": 0.0,
"completions/min_terminated_length": 6.8,
"epoch": 0.011999850001874977,
"grad_norm": 0.005889138672500849,
"learning_rate": 5.952380952380953e-07,
"loss": 0.0095,
"num_tokens": 8999691.0,
"reward": 0.7351018190383911,
"reward_std": 0.725162398815155,
"rewards/accuracy_reward": 0.2590277761220932,
"rewards/brier_reward": 0.3106847941875458,
"rewards/format_reward": 0.5899305582046509,
"rewards/frontier_aurc_reward": 0.3106847941875458,
"rewards/frontier_coverage_1": 0.3106847941875458,
"rewards/frontier_coverage_10": 0.3106847941875458,
"rewards/frontier_coverage_15": 0.3106847941875458,
"rewards/frontier_coverage_20": 0.3106847941875458,
"rewards/frontier_coverage_25": 0.3106847941875458,
"rewards/frontier_coverage_5": 0.3106847941875458,
"step": 5
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.01649305555555556,
"completions/max_length": 4001.4,
"completions/max_terminated_length": 4001.4,
"completions/mean_length": 474.1986083984375,
"completions/mean_terminated_length": 482.28477172851564,
"completions/min_length": 0.0,
"completions/min_terminated_length": 30.8,
"epoch": 0.023999700003749954,
"grad_norm": 0.005823688581585884,
"learning_rate": 1.1904761904761906e-06,
"loss": 0.0005,
"num_tokens": 17545179.0,
"reward": 0.8662121772766114,
"reward_std": 0.6877874732017517,
"rewards/accuracy_reward": 0.2932291626930237,
"rewards/brier_reward": 0.3596000373363495,
"rewards/format_reward": 0.7201388955116272,
"rewards/frontier_aurc_reward": 0.3596000373363495,
"rewards/frontier_coverage_1": 0.3596000373363495,
"rewards/frontier_coverage_10": 0.3596000373363495,
"rewards/frontier_coverage_15": 0.3596000373363495,
"rewards/frontier_coverage_20": 0.3596000373363495,
"rewards/frontier_coverage_25": 0.3596000373363495,
"rewards/frontier_coverage_5": 0.3596000373363495,
"step": 10
},
{
"clip_ratio/high_max": 0.0,
"clip_ratio/high_mean": 0.0,
"clip_ratio/low_mean": 0.0,
"clip_ratio/low_min": 0.0,
"clip_ratio/region_mean": 0.0,
"completions/clipped_ratio": 0.009375,
"completions/max_length": 3710.6,
"completions/max_terminated_length": 3710.6,
"completions/mean_length": 433.11754150390624,
"completions/mean_terminated_length": 437.21246948242185,
"completions/min_length": 0.0,
"completions/min_terminated_length": 66.0,
"epoch": 0.03599955000562493,
"grad_norm": 0.005002312827855349,
"learning_rate": 1.7857142857142859e-06,
"loss": -0.0212,
"num_tokens": 25636677.0,
"reward": 1.0900377035140991,
"reward_std": 0.5645716547966003,
"rewards/accuracy_reward": 0.3383680522441864,
"rewards/brier_reward": 0.4504141747951508,
"rewards/format_reward": 0.9410590291023254,
"rewards/frontier_aurc_reward": 0.4504141747951508,
"rewards/frontier_coverage_1": 0.4504141747951508,
"rewards/frontier_coverage_10": 0.4504141747951508,