Add mixture-N (text67) logs for 01-12 and 14_bpe1_large, weights for 10-12 and 14
Browse filesWeights zip: best + terminal checkpoints, all rank dirs, stdout for the locally trained mixture-N runs not in yhc_v2_all9. Account names redacted.
- text67_10_12_14_weights_logs_20261014.zip +3 -0
- training_logs/README.md +14 -0
- training_logs/run_metrics/14_bpe1_large/args.yml +99 -0
- training_logs/run_metrics/14_bpe1_large/metrics.csv +40 -0
- training_logs/run_metrics_text67/01_bittoken_xl/args.yml +99 -0
- training_logs/run_metrics_text67/01_bittoken_xl/metrics.csv +40 -0
- training_logs/run_metrics_text67/02_bpe_xl/args.yml +99 -0
- training_logs/run_metrics_text67/02_bpe_xl/metrics.csv +40 -0
- training_logs/run_metrics_text67/03_fone_xl/args.yml +99 -0
- training_logs/run_metrics_text67/03_fone_xl/metrics.csv +40 -0
- training_logs/run_metrics_text67/04_bittoken_large/args.yml +99 -0
- training_logs/run_metrics_text67/04_bittoken_large/metrics.csv +40 -0
- training_logs/run_metrics_text67/05_bpe_large/args.yml +99 -0
- training_logs/run_metrics_text67/05_bpe_large/metrics.csv +40 -0
- training_logs/run_metrics_text67/06_fone_large/args.yml +99 -0
- training_logs/run_metrics_text67/06_fone_large/metrics.csv +40 -0
- training_logs/run_metrics_text67/07_bittoken_medium/args.yml +99 -0
- training_logs/run_metrics_text67/07_bittoken_medium/metrics.csv +40 -0
- training_logs/run_metrics_text67/08_bpe_medium/args.yml +99 -0
- training_logs/run_metrics_text67/08_bpe_medium/metrics.csv +40 -0
- training_logs/run_metrics_text67/09_fone_medium/args.yml +99 -0
- training_logs/run_metrics_text67/09_fone_medium/metrics.csv +40 -0
- training_logs/run_metrics_text67/10_bittoken_small/args.yml +99 -0
- training_logs/run_metrics_text67/10_bittoken_small/metrics.csv +40 -0
- training_logs/run_metrics_text67/11_bpe_small/args.yml +99 -0
- training_logs/run_metrics_text67/11_bpe_small/metrics.csv +40 -0
- training_logs/run_metrics_text67/12_fone_small/args.yml +99 -0
- training_logs/run_metrics_text67/12_fone_small/metrics.csv +40 -0
- training_logs/train_stdout/14_bpe1_large_train.log.gz +3 -0
- training_logs/train_stdout_text67/10_bittoken_small_train.log.gz +3 -0
- training_logs/train_stdout_text67/11_bpe_small_train.log.gz +3 -0
- training_logs/train_stdout_text67/12_fone_small_train.log.gz +3 -0
text67_10_12_14_weights_logs_20261014.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:36962b8186d013e07d328d80b5ba5e0499b9d83ca209e2df5d1d251bb31d7316
|
| 3 |
+
size 7495115770
|
training_logs/README.md
CHANGED
|
@@ -6,3 +6,17 @@ Numbering matches the checkpoint zips in the repo root.
|
|
| 6 |
- `run_metrics/<NN>_<method>_<size>/{args.yml,metrics.csv}` - args and per-eval metrics of the rank-0 (seed 42) run. Available for 02, 04-12.
|
| 7 |
- `train_stdout/<NN>_<method>_<size>_train.log.gz` - raw stdout/tqdm log (gzip). Available for the locally trained runs: 02, 04, 06, 07, 10, 11, 12.
|
| 8 |
05/08/09 were trained on a B200 node (no stdout log kept), 01/03 (xl bittoken / xl fone) were trained on a B200 node (no logs here at all).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
- `run_metrics/<NN>_<method>_<size>/{args.yml,metrics.csv}` - args and per-eval metrics of the rank-0 (seed 42) run. Available for 02, 04-12.
|
| 7 |
- `train_stdout/<NN>_<method>_<size>_train.log.gz` - raw stdout/tqdm log (gzip). Available for the locally trained runs: 02, 04, 06, 07, 10, 11, 12.
|
| 8 |
05/08/09 were trained on a B200 node (no stdout log kept), 01/03 (xl bittoken / xl fone) were trained on a B200 node (no logs here at all).
|
| 9 |
+
|
| 10 |
+
# Mixture N (text67) runs
|
| 11 |
+
|
| 12 |
+
Same recipe without curriculum or adaptive weighting: fixed sampling ratios (6/7 FineWeb, 1/7 arithmetic), unweighted loss.
|
| 13 |
+
NN follows the grid above: 01-03 xl, 04-06 large, 07-09 medium, 10-12 small (bittoken, bpe, fone).
|
| 14 |
+
|
| 15 |
+
- `run_metrics_text67/<NN>_<method>_<size>/{args.yml,metrics.csv}` - rank-0 (seed 42) args and per-eval metrics, 01-12.
|
| 16 |
+
- `run_metrics/{13_bpe1_xl,14_bpe1_large,15_bpe1_medium,16_bpe1_small}/` - BPE-1-digit runs (mixture N only).
|
| 17 |
+
- `train_stdout_text67/{10,11,12}_<method>_small_train.log.gz`, `train_stdout/{13,14,15,16}_bpe1_<size>_train.log.gz` - stdout logs.
|
| 18 |
+
- Weights and full run dirs (all ranks, best + terminal checkpoints, stdout): `yhc_v2_all9_weights_logs_wandb_20261008.zip` (01-09) and
|
| 19 |
+
`text67_10_12_14_weights_logs_20261014.zip` (10-12 and 14_large_bpe1).
|
| 20 |
+
- `train_stdout/03_fone_xl_aborted_attempt_train.log.gz` is an aborted mixture-M xl FoNE attempt, not the run behind `01_03_best_ckpts.zip`.
|
| 21 |
+
|
| 22 |
+
Account names in paths and URLs are redacted as `<user>`, `<wandb_user>`, `<itsc_user>`.
|
training_logs/run_metrics/14_bpe1_large/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: False
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 2
|
| 5 |
+
cache_base_path: /work8/<user>/byte_v2/math_bittoken/mix_text67/cache
|
| 6 |
+
combine_strategy: sum
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 12
|
| 22 |
+
difficulty_column: difficulty_sd
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 48
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 5088
|
| 28 |
+
eval_every_k_tokens: 62521344
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /work8/<user>/byte_v2/math_bittoken/mix_text67/configs/config_td1_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 203451
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_gpt2
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 1280
|
| 45 |
+
n_head: 20
|
| 46 |
+
n_layer: 36
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: bittoken
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: mse
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 20346
|
| 56 |
+
num_warmup_tokens: 250000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /work8/<user>/byte_v2/math_bittoken/mix_text67/configs/config_td1_text67.py --tqdm --verbose --seed 42 --run_name bpe1_gpt2_large_text67 --device_batch_size 12 --effective_batch_size 48 --num_workers 4 --n_layer 36 --n_head 20 --n_embd 1280 --context_length 1024 --train_token_budget 2500000000 --num_warmup_tokens 250000000 --eval_every_k_tokens 62521344 --max_eval_steps 2 --save_dir /work8/<user>/byte_v2/math_bittoken/mix_text67/trained/bpe1_gpt2_large --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group bpe1_gpt2_large_text67 --tie_word_embeddings --save_checkpoint_tokens 125042688 --tokenizer_dir /work8/<user>/byte_v2/math_bittoken/mix_text67/tokenizers/td1_gpt2 --model rope_gpt2 --dropout 0 --cache_base_path /work8/<user>/byte_v2/math_bittoken/mix_text67/cache --compile --difficulty_column difficulty_sd --train_set_paths /work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/mix_text67/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard --train_set_ratios 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_prompt efficient_prompt efficient_prompt efficient_prompt efficient_prompt efficient_prompt pretokenized --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Sat Oct 10 04:29:13 2026', 'git_root': '/work8/<user>/byte_v2/math_bittoken/_paper_repo', 'git_url': 'https://github.com/KreitnerL/BitTokens/tree/192363b53b04652aec75c4cdf84d1a330beb25d9', 'git_has_uncommitted_changes': True}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: bpe1_gpt2_large_text67
|
| 68 |
+
save_checkpoint_steps: 10176
|
| 69 |
+
save_checkpoint_tokens: 125042688
|
| 70 |
+
save_dir: /work8/<user>/byte_v2/math_bittoken/mix_text67/trained/bpe1_gpt2_large
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /work8/<user>/byte_v2/math_bittoken/mix_text67/tokenizers/td1_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td1_gpt2_59750704/96841076'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td1_gpt2_59750704/63050008'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td1_gpt2_59750704/44316829'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td1_gpt2_59750704/48579485'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td1_gpt2_59750704/63589654'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td1_gpt2_59750704/28473695'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td1_gpt2_59750704/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 2500000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'pretokenized']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: bpe1_gpt2_large_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics/14_bpe1_large/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
5088,62521344,1.5107573121786118,1.8722712993621826,1.762038454413414,0.8190860860049725,0.7347516864538193,2.5392455607652664,4.266682863235474,4.808270678955591,6.576719078568449,6.028293650544364,2.649480262432673,2.086511900967154,16.20225394018249,71.33894308966815,15.670067514474125,0.32860398275657476,0.03328443943600519,0.03873680007917352,0.7291666716337204,0.125,0.5041646175086498,0.7320386283641289,0.09773336350917816,0.125,0.0,0.0,0.8243175467505353,nan,nan,0.7219794944006329,0.13521024585497798,0.11720575194340199,5.18341444826351,0.019297877124894364,0.023114957663658186,0.018740308725087664,0.009194760657954007,0.000918898492301602,0.021275042702386057,5.09087263244503,594.6940424745937,0.20904745657691173,0.004997126321997158,85.68922532362978
|
| 3 |
+
10176,125042688,1.0058209151029587,1.5825687497854233,2.093796417117119,0.18664080380403902,0.7476693615317345,1.687627762556076,3.785349190235138,2.830641888559001,4.9382183924336704,9.490196688286723,1.2597853355911064,2.1169073794452538,8.48280757908838,44.06706313197262,10.455088627910966,0.4862211499857988,0.1250201252574849,0.11912385909795055,0.9583333432674408,0.09895833488553762,0.7582914307713509,0.9590397991875835,0.21902231872081757,0.2708333333333333,0.052083333333333336,0.020833333333333332,0.9738679931130415,nan,nan,0.8805062449537218,0.12221258674056419,0.10347076163937648,4.518335130932571,0.015054153331644018,0.02094023289604101,0.016683670956825943,0.005077337075308834,0.0007858715763983848,0.012112595261193215,4.447681294226983,49.244368082697285,0.32879994129293466,0.009998084214664627,85.73335212655608
|
| 4 |
+
15264,187564032,0.3751345071941614,1.649525836110115,2.2026022747159004,0.0799053093069233,0.09766797965858132,0.9990465492010117,3.6454018354415894,1.4716695070495676,5.2572240119752465,11.081798032767404,1.109557064913112,1.1051235896109355,3.6449501812297216,38.31273120429089,8.854721941690983,0.6787019291507634,0.1317826946368096,0.16110428095292387,0.9895833358168602,0.9348958488553762,0.8984841853380203,1.0,0.35967594385147095,0.5208333333333334,0.052083333333333336,0.052083333333333336,0.9901969157314549,nan,nan,0.9413024009748673,0.14461402828843953,0.1254413347148026,4.210461324272166,0.012153619493227755,0.019815291464524012,0.015608431747479978,0.003598475948497415,0.0006992525442826951,0.008839627329740111,4.149746647046047,37.636725805090656,0.3556887467374217,0.014999042107332095,85.72728336013589
|
| 5 |
+
20352,250085376,0.23615136463195086,1.4144557863473892,2.4380043745040894,0.055029850050686946,0.033700450949254446,0.5009346549923066,3.582855701446533,1.2997430574907236,4.180842185548029,14.343808798110947,1.068932158332849,1.0356203674980007,1.7477631951673662,36.003749577298514,8.525779905635204,0.8019297898816916,0.1925786632708449,0.1885613493375389,0.9895833358168602,0.9739583395421505,0.9301975518465042,1.0,0.4414929151535034,0.6875,0.08333333333333333,0.07291666666666667,0.9895953763940876,nan,nan,0.9680590132872263,0.12973987917090804,0.10952968682007243,4.029720348569582,0.009913962909263748,0.01912753933999664,0.014994206766099317,0.002848666358092277,0.00057035373053239,0.007130456716096212,3.9751351845198832,33.757169145083274,0.3681347846586562,0.020000000000000004,85.77453496186718
|
| 6 |
+
25440,312606720,0.166809247341007,1.387430801987648,2.198298379778862,0.00043113200274547125,0.01712268947812845,0.41946534962335136,3.4875431060791016,1.2356221869214246,4.068479194268111,11.052747619133804,1.0004313681856651,1.018158861485399,1.7076975495040123,32.71922468804524,7.543194495363379,0.9192700970682803,0.20603527864390558,0.2154357732723227,1.0,0.9869791697710752,0.9678373336791992,1.0,0.4784840941429138,0.8541666666666666,0.09375,0.09375,1.0,nan,nan,0.9831256226946911,0.15006431186338887,0.12538637077280632,3.5859552734654465,0.005670122336916576,0.01748166142072552,0.013515300921815354,0.0006889272366765703,0.00037226439555895584,0.0022620993045366165,3.5459649153405204,31.20829473370276,0.37645223924784726,0.019965729849129636,85.7971424528054
|
| 7 |
+
30528,375128064,0.22163628973066807,1.3450169414281845,2.5131354928016663,0.0007515863926528255,0.007755116559565067,0.18343891548283864,3.4197728037834167,1.2975765859626076,3.891700812474148,16.381829318051953,1.0007522020904365,1.0079459820165428,1.227884258050321,30.576773155095054,7.912066044820151,0.8679135226987559,0.23156133698012854,0.2641783308704876,1.0,0.9869791697710752,0.953079029917717,1.0,0.5255869030952454,0.78125,0.10416666666666667,0.13541666666666666,1.0,nan,nan,0.9760874287070086,0.1665029346816785,0.14525857428088784,3.45803075506346,0.003447298374591564,0.016839749303517525,0.012994002371732842,0.0005834425662768658,0.00023537040428381147,0.001957309298538717,3.4219736008078985,29.149247594224583,0.3836932276322875,0.019863180383909898,85.76746819566118
|
| 8 |
+
35616,437649408,0.14488372759660706,1.423810824751854,3.041365474462509,0.0006588036592063418,0.006079874641727656,0.08053515707069892,3.379262089729309,1.2162521833848279,4.18701504962281,28.27329000696284,1.000659470974949,1.0061888654294473,1.1012598759411374,29.35710488470839,9.44882433386063,0.9171975476901367,0.22501674981598468,0.25046802244576916,1.0,0.9869791697710752,0.9763888865709305,1.0,0.5160608887672424,0.8854166666666666,0.10416666666666667,0.11458333333333333,1.0,nan,nan,0.985468417716523,0.15468891108199992,0.1308540201668317,3.38275128581895,0.0022979638792917888,0.01644314803917447,0.012755948714944886,0.0005549704453654847,0.00012772045960804162,0.0018055386332876027,3.3487660141322038,27.790354828039806,0.3888616146308915,0.019693132578928373,85.75989169294876
|
| 9 |
+
40704,500170752,0.15122379502281547,1.3555549383163452,2.5300933569669724,0.0013203344968815145,0.0072596888758198475,0.18150976161996368,3.3475341796875,1.221641585980155,3.935777053612563,16.55447745745169,1.00132302608869,1.00745545423143,1.2388493746973461,28.438409566463303,7.628276216932169,0.9105397635487293,0.2504978838518423,0.24779273092840143,1.0,0.9869791697710752,0.9705393388867378,1.0,0.5315321087837219,0.8541666666666666,0.11458333333333333,0.10416666666666667,1.0,nan,nan,0.98618194522957,0.17759337222378235,0.1541990638943389,3.325319302297613,0.0019069173220904413,0.016154173207718222,0.012528053651060089,0.0005363085646545164,0.00010458986369142251,0.0017229896031666142,3.2923662874308772,26.78961455878222,0.3926176142837829,0.01945688144839207,85.70341736877428
|
| 10 |
+
45792,562692096,0.23585292848292738,1.3944892138242722,2.4699205234646797,0.0002452009701130464,0.008884292918082792,0.19199205071345204,3.313930630683899,1.3695959265093407,4.115690877983177,15.455154659956246,1.0002452761422687,1.0091801013850668,1.2873239561217864,27.496568181799432,7.390536997128188,0.9385828985789669,0.2376774902250522,0.2292071647666897,1.0,0.9869791697710752,0.9593253955245018,1.0,0.511274516582489,0.90625,0.10416666666666667,0.09375,1.0,nan,nan,0.9716719625672946,0.16492957122197063,0.13968703399101892,3.280439954953936,0.001712586308275484,0.015964495669208385,0.012365950113246598,0.0005248282342323026,9.459477450045529e-05,0.0016713928995391065,3.2481061252501777,26.03123458695112,0.39596585108189836,0.019156226183831337,85.72348402514717
|
| 11 |
+
50880,625213440,0.13193263416178524,1.3365391194820404,2.3662626668810844,0.00018999851070589102,0.007857101139052247,0.10560055645692046,3.286300539970398,1.2000317269347311,3.857960787167901,11.948292798773318,1.0001900406148665,1.0080951246872387,1.1219329531948847,26.747365027832522,6.697695494172208,0.9463754467916743,0.23623353106357822,0.25399059855986467,1.0,0.9869791697710752,0.9679924249649048,1.0,0.5273794531822205,0.90625,0.09375,0.13541666666666666,1.0,nan,nan,0.9834359975842139,0.17587218634434976,0.15209279294746617,3.245990631481012,0.0015957182339856296,0.015833419182963094,0.012245571851231006,0.0005203488045684674,8.905080964160396e-05,0.001631243420815117,3.2140752966837876,25.37810212261272,0.3986600871848048,0.018793456452196412,85.70739524310237
|
| 12 |
+
55968,687734784,0.23003534026793204,1.283374235033989,2.1604764834046364,0.0004930469914370406,0.00816468288348915,0.012959165830579877,3.266716778278351,1.36602524345479,3.635023003805337,10.12655272483519,1.0004933281613897,1.0084309690990194,1.0132989021335521,26.22749999786256,6.339617738478834,0.9580377477414608,0.276419965164183,0.2525129665345123,1.0,0.9869791697710752,0.9853174611926079,1.0,0.5533697009086609,0.9166666666666666,0.14583333333333334,0.125,1.0,nan,nan,0.9917137497104704,0.16799134595203213,0.1451401124941185,3.2202327009524754,0.0015037506734857605,0.0157445861437027,0.012104728892474851,0.0005179009133310019,8.565154749480456e-05,0.0016097850225063747,3.18866631587815,25.02280656253017,0.4000132856421969,0.018371334958694578,85.65683878549551
|
| 13 |
+
61056,750256128,0.126630119048059,1.331733614206314,2.2128999531269073,0.0002375150954350147,0.014226757987216843,0.01946410590016967,3.2464191913604736,1.1745824722029645,3.8266344461650528,10.708241494904172,1.0002376235326003,1.0150438144822878,1.0202044211600234,25.70366240674362,6.349800954170102,0.9381418953342517,0.24890223791222324,0.25190615321131654,1.0,0.9869791697710752,0.9916035383939743,1.0,0.5353740453720093,0.9166666666666666,0.11458333333333333,0.10416666666666667,1.0,nan,nan,0.9964456874877214,0.1611117016582284,0.14050483459141105,3.1980524560284316,0.001434105886847023,0.015662925400435866,0.012006922438392948,0.0005139623518331166,8.206266023128943e-05,0.0015974187342091704,3.166755076076062,24.459656450920885,0.40245779711495405,0.017893076407163026,85.62596301148905
|
| 14 |
+
66144,812777472,0.1416965461539803,1.4203665405511856,2.3084903731942177,0.00023527436938763913,0.005438609348857426,0.04360953302057169,3.2307671904563904,1.2289443743496111,4.174855052340119,12.527192813731293,1.0002353372282857,1.005548116699592,1.0489582525256695,25.301609798685078,6.612477677937093,0.950901735867206,0.251777511815465,0.2849174178780835,1.0,0.9869791697710752,0.9823412671685219,1.0,0.5571317672729492,0.9166666666666666,0.11458333333333333,0.17708333333333334,1.0,nan,nan,0.9894256429785552,0.16681462082002932,0.14243055674402663,3.1789545410857447,0.0013832474187638695,0.015597135948790412,0.011908884683265878,0.0005131868848980652,7.977942085963151e-05,0.0015809777405018058,3.147891346423787,24.085183755781667,0.4044993002225393,0.017362323018206012,85.59564885522768
|
| 15 |
+
71232,875298816,0.1648914209799841,1.3444708585739136,2.1951510831713676,0.0002712125406745258,0.012727732198300146,0.01515719715416708,3.218819737434387,1.2496779720105813,3.898533421462562,10.529798386617157,1.0002713037701811,1.0133574583876566,1.0156231834657157,25.002393478836865,6.24423645779296,0.955377109932565,0.27065202536036215,0.2924525261245463,1.0,0.9869791697710752,0.9823412671685219,1.0,0.5741462707519531,0.9166666666666666,0.125,0.17708333333333334,1.0,nan,nan,0.9897142552460233,0.16274928249185905,0.13653712397596487,3.1624491616333805,0.0013437328366422468,0.01554974089464489,0.011811255852057768,0.0005104071572512776,7.74421036631447e-05,0.0015815803561202598,3.1315750200418555,23.718304400174123,0.4060386490229072,0.016783116791537683,85.64214477173016
|
| 16 |
+
76320,937820160,0.28068525018170476,1.3420012146234512,2.355092652142048,0.0001788211844271359,0.0062895330606806965,0.07814517580436586,3.2008811831474304,1.4121038779334838,3.883013701913235,14.065606167051499,1.0001788694029974,1.006448427259033,1.096528894823246,24.55857740345312,6.717493905976659,0.9055570901860965,0.24464199826814037,0.3096210823420209,1.0,0.9869791697710752,0.9792162701487541,1.0,0.5619750022888184,0.8645833333333334,0.09375,0.1875,1.0,nan,nan,0.9887661066992829,0.1512854519241955,0.12528096414947262,3.146330834255091,0.0013151931628632362,0.015508099891093073,0.011679416684925838,0.0005078650719943236,7.52126307227772e-05,0.0015722535569575116,3.115672810729476,23.450695441204047,0.40753353268798415,0.016159868723772935,85.63379153003525
|
| 17 |
+
81408,1000341504,0.2571905527584022,1.334544688463211,2.247622363269329,0.00014394254114336036,0.0031761971863488725,0.024008598782302215,3.1980583667755127,1.4213986767630842,3.8267483662391797,11.835717072922156,1.000143986703626,1.003214700970808,1.0252069718346086,24.487695374678744,6.3714464500160295,0.9490685264579075,0.26780635693769483,0.2712670725322731,1.0,1.0,0.991540402173996,1.0,0.5607787370681763,0.90625,0.125,0.15625,1.0,nan,nan,0.9964662579198679,0.15945980591641273,0.13599085128710917,3.1330203568012274,0.0012882437206958143,0.015447224244334787,0.011562534207448879,0.0005067658540904719,7.380396016690457e-05,0.0015578104830156747,3.1025839903913206,23.2270653371541,0.4087196325754407,0.015497325216091833,85.6426872195387
|
| 18 |
+
86496,1062862848,0.14799151713668834,1.3515418767929077,2.060160405933857,0.0003048129902545327,0.00922306395796113,0.014235094154969374,3.183887481689453,1.2194578380773882,3.9444433486780976,10.262828889259383,1.0003050017908512,1.009564979976048,1.0146908038612565,24.143091936601195,6.084911828320604,0.9470067352157199,0.2656329813291783,0.3103796540073596,1.0,0.9869791697710752,0.9853174611926079,1.0,0.5799260139465332,0.8958333333333334,0.125,0.19791666666666666,1.0,nan,nan,0.9917137497104704,0.16346348385559395,0.13798874938705316,3.1203169466330194,0.0012636074437285963,0.015402759096413886,0.011448163532947758,0.0005029656116639196,7.207332347271147e-05,0.001564430351938927,3.0900629641570196,22.860533486559707,0.4103049317538832,0.014800531927597655,85.59950007908182
|
| 19 |
+
91584,1125384192,0.033821231743786484,1.2786605805158615,2.05217956751585,0.00030310902026187136,0.005186214098557684,0.015377203224488767,3.163091242313385,1.034846072702146,3.645878663738589,9.721219035377253,1.0003032355981283,1.0052925036480136,1.0158851154862776,23.64583146330401,5.867036584264917,0.94908541721888,0.280000032835926,0.30619297241728494,1.0,0.9869791697710752,0.9882936477661133,1.0,0.5874441862106323,0.90625,0.13541666666666666,0.16666666666666666,1.0,nan,nan,0.9936841785286864,0.16265454704019552,0.13779563685723892,3.1079637479482205,0.0012425455755310853,0.015364449246641565,0.011335895432553944,0.0005060122455487275,7.135248461686347e-05,0.0015525458387956768,3.077890962983171,22.57233165007717,0.4115675697747454,0.014074795349650518,85.63868829908374
|
| 20 |
+
96672,1187905536,0.22315626329509541,1.2922115251421928,1.889831192791462,0.00032023214578202897,0.005495855509707326,0.011678991242092707,3.154578685760498,1.3416340278139371,3.71661635913407,8.407695650999978,1.0003203367779336,1.0056159618944132,1.0120175174179469,23.444617181025272,5.704073862151936,0.9519633803243331,0.27169464579386116,0.3096661957517216,1.0,0.9869791697710752,0.9912698417901993,1.0,0.5841928124427795,0.9166666666666666,0.11458333333333333,0.1875,1.0,nan,nan,0.9955318241069714,0.16859541935264133,0.1425284679668645,3.095744621114067,0.0012241821002890163,0.015319463327408434,0.01125585837835159,0.0005025382332209656,7.026897759889195e-05,0.0015404659325596937,3.065831861183422,22.334568947928506,0.4131148651933426,0.013325642393809753,85.7812457095182
|
| 21 |
+
101760,1250426880,0.22355581421288662,1.2658047303557396,1.9158143065869808,0.0002229930843000716,0.006182119835330013,0.020624000524321673,3.1402618885040283,1.35460907570303,3.5968963720816465,9.001827674075244,1.0002230892257342,1.0063349544252604,1.0215008786003346,23.112422628622827,5.727687810390583,0.963222891800602,0.26923987941554817,0.29760698117364764,1.0,0.9869791697710752,0.9853174611926079,1.0,0.5765790939331055,0.9479166666666666,0.125,0.17708333333333334,1.0,nan,nan,0.9914664871369799,0.16313535125421672,0.13932430367761603,3.08381740809799,0.0012048748841966385,0.015303780511277874,0.011207919865634304,0.000499428957641489,6.979952146566243e-05,0.0015294418363856802,3.054002182077874,22.107318835438424,0.41420535730350316,0.012558778301146429,85.85266795473572
|
| 22 |
+
106848,1312948224,0.22665598451567348,1.2448271438479424,2.1718335673213005,0.00022310908656919537,0.009350289101803355,0.19540149890781322,3.1316614747047424,1.3541659505305534,3.5031745740270717,12.029335020739927,1.0002231850618042,1.0097063290090267,1.4163279360227157,22.9154356800111,6.175481239343171,0.9623868960685127,0.278479053168338,0.3145813854255223,1.0,0.9869791697710752,0.9795634895563126,1.0,0.5910714268684387,0.9270833333333334,0.11458333333333333,0.1875,1.0,nan,nan,0.9893025589796404,0.17908589845562042,0.15447597679061195,3.0720927152524955,0.00118278993821925,0.015283912226064247,0.011142630902585375,0.0005001036749012933,6.928159002597776e-05,0.0015088376147511405,3.042405177458371,21.813800647011345,0.41568243594945603,0.011780043193471874,85.72692731107377
|
| 23 |
+
111936,1375469568,0.11581438482971862,1.2826227098703384,2.19117159768939,0.0003745696010284405,0.007416690096988532,0.00035845697550485056,3.1206008195877075,1.1802211290853606,3.662192222802166,11.44055850009848,1.0003748292502246,1.0076403090792676,1.0003586748677356,22.66377908569125,5.993589250124926,0.9642129449496362,0.2860062023565413,0.3189457328841515,1.0,0.9869791697710752,1.0,1.0,0.5992091298103333,0.9479166666666666,0.125,0.19791666666666666,1.0,nan,nan,1.0,0.15283281981828623,0.1279984222104152,3.0621570606581257,0.0011663000302325553,0.015257741190066983,0.011097945376707873,0.0004895907646374054,6.814105790818584e-05,0.0015052618087051657,3.0325720991130987,21.682541946382642,0.41646397147772823,0.010995367597369719,85.8791072364673
|
| 24 |
+
117024,1437990912,0.1255586059851339,1.2559480145573616,2.129518311470747,0.00019676796877377,0.004217814171511236,0.026742901199668268,3.108653426170349,1.1941336920428283,3.557084495850344,11.992552505544815,1.000196823746168,1.004289036285013,1.0283882201824168,22.394616971125078,6.024465963539524,0.962619769367068,0.28917807051069233,0.3451833551841527,1.0,0.9869791697710752,0.9853174611926079,1.0,0.612851619720459,0.9479166666666666,0.13541666666666666,0.21875,1.0,nan,nan,0.9915778886837264,0.16641118143646358,0.14120041210359582,3.0519205748730296,0.0011496083823925823,0.015222115352099656,0.011058707629735346,0.0004894074700165942,6.78555880968741e-05,0.0015027766955893994,3.0224301210610465,21.41156333010152,0.4179543591684328,0.01021072727974235,85.90983323557268
|
| 25 |
+
122112,1500512256,0.12412230778863886,1.248071238398552,2.178366079926491,0.00012907553722740772,0.006518938136593988,0.0282178090823777,3.0980597734451294,1.2027126736821545,3.5202825330983964,13.088639350320788,1.000129113280722,1.0066909472459442,1.0300974658475957,22.15937018944234,6.143988896131134,0.9743778781962248,0.29598921041738085,0.31544937197631095,1.0,0.9869791697710752,0.9823412671685219,1.0,0.6031338572502136,0.96875,0.13541666666666666,0.17708333333333334,1.0,nan,nan,0.9893608273123391,0.17984736433330303,0.15420685118685165,3.042040818423595,0.0011356299308420792,0.015193184517373759,0.011016689172108312,0.0004912753381194929,6.615708167396174e-05,0.0014936509951741337,3.0126442482179536,21.261296854454017,0.4186330793378582,0.009432097738825898,85.73590521737289
|
| 26 |
+
127200,1563033600,0.009481508153839968,1.2557559311389923,2.2537356726825237,0.0001221755323044249,0.004134079551135983,0.03187772642226605,3.093813896179199,1.009570376947906,3.541587359096228,12.658747372734418,1.0001222084588184,1.0042025426561434,1.0343704889396363,22.06535938365058,6.044851390354819,0.9805260676188372,0.2755506068382616,0.3375001749769505,1.0,0.9869791697710752,0.9853174611926079,1.0,0.6013738512992859,0.96875,0.125,0.20833333333333334,1.0,nan,nan,0.991576440166682,0.1824259176986137,0.15665195773666105,3.0344071904982224,0.0011215365326942956,0.015148593089579892,0.010998938768610076,0.0004913086988601964,6.50859086300224e-05,0.0015000030291221337,3.0050817426504954,21.15900489903471,0.41929890515294466,0.008665408697250875,85.77930833326762
|
| 27 |
+
132288,1625554944,0.040472697211953346,1.283346951007843,2.264037609100342,0.00012676213207640785,0.007906446165122816,0.0059158437440203215,3.085277795791626,1.0446635282996337,3.6825537568477484,13.925805687086203,1.000126795780669,1.0081608185040731,1.006052458762277,21.878210056497338,6.220796157396849,0.9737314917908383,0.2923717432546007,0.3519483713004493,1.0,0.9869791697710752,0.9910714253783226,1.0,0.6188771724700928,0.96875,0.13541666666666666,0.22916666666666666,1.0,nan,nan,0.9939944399520755,0.1766351161932107,0.14966581385427466,3.026881415396929,0.0011080455669355233,0.015106108080985246,0.010976129046380637,0.000496251870843224,6.424758833733149e-05,0.0014827304675012274,2.997647920394984,21.018744714979857,0.42001037673919267,0.007916498943711495,85.8899437991045
|
| 28 |
+
137376,1688076288,0.15641980031796265,1.2528557106852531,2.3220024406909943,0.00017332752655541128,0.006929011205841107,0.006441149460201245,3.077893614768982,1.2368930268383442,3.5566561549695264,14.662670440138669,1.0001733797926946,1.0071239285378768,1.0066006733638455,21.717270181584645,6.312483969317943,0.9747885945503187,0.29116583275294056,0.3299712747510999,1.0,0.9869791697710752,0.9910714253783226,1.0,0.6080030798912048,0.96875,0.13541666666666666,0.20833333333333334,1.0,nan,nan,0.9939944399520755,0.18748986754993288,0.1614466788014397,3.019287815891534,0.001092371506938749,0.01508809830652954,0.01096596749541174,0.0004955399384431066,6.301250117582581e-05,0.0014829820305942284,2.9900998612221494,20.766348573677945,0.42143976171178826,0.007191071867150021,85.81645629543588
|
| 29 |
+
142464,1750597632,0.1404635772414622,1.3302903771400452,2.4331426173448563,0.0001416951312300796,0.004796345916076916,0.021408079514856126,3.0737903118133545,1.2327567017137417,3.86362807310301,15.21036775639179,1.0001417307774105,1.0048890653698874,1.0233221267981663,21.629005214674788,6.423444381261256,0.9668888858015712,0.3011033684859689,0.3222763987980518,1.0,0.9869791697710752,0.9880952388048172,1.0,0.6095685958862305,0.9583333333333334,0.14583333333333334,0.19791666666666666,1.0,nan,nan,0.9920453010126948,0.1830504859002152,0.15648657952745756,3.0130882045757845,0.001082682281542406,0.015045265392026917,0.010916447883241944,0.0004943815240208975,6.308328061347562e-05,0.0014840193908802009,2.984002342389065,20.709924922986602,0.4217194695166259,0.006494652022090374,85.79190252015643
|
| 30 |
+
147552,1813118976,0.12980822127428837,1.2509591281414032,2.483940839767456,0.00011438358373894175,0.004765618209262357,0.01801038883453998,3.0660236477851868,1.2166710226064548,3.549888322395163,19.347017145463084,1.0001144078717408,1.0048573052711132,1.0186741026272856,21.46342354871903,6.942949407850554,0.9743980211563029,0.2843358271204004,0.3510144201481197,1.0,0.9869791697710752,0.9853174611926079,1.0,0.6129477620124817,0.96875,0.125,0.21875,1.0,nan,nan,0.9914863320688406,0.1745513252778134,0.1497468154411763,3.005972785090313,0.0010752628700224733,0.015032187830539619,0.01085992485453937,0.0004902567216157239,6.289651563707174e-05,0.0014768090961379428,2.9769754631725966,20.562913267679935,0.42262869485238064,0.005832543055900948,85.77184092797206
|
| 31 |
+
152640,1875640320,0.1300680618514889,1.246540792286396,2.297078289091587,0.0001350318329542688,0.005632700825742631,0.021264183669956083,3.061741590499878,1.2168428481722382,3.5109609818917216,14.603870206016659,1.0001350619064344,1.0057609478697376,1.023158556302636,21.371201381491787,6.24741856909303,0.9710258943342069,0.3032508109778438,0.3815522946382253,1.0,0.9869791697710752,0.9880952388048172,1.0,0.6378973126411438,0.9583333333333334,0.14583333333333334,0.25,1.0,nan,nan,0.9920366693598529,0.177034840870571,0.1513888593763113,3.0000279560778877,0.001063822305084459,0.015008557709951669,0.010833152214086428,0.00048736244092412417,6.280855092826752e-05,0.0014718388042189799,2.9711004317615113,20.528502779749203,0.4227798976730438,0.005209787318395601,85.79358003641975
|
| 32 |
+
157728,1938161664,0.13097040152933914,1.2307926937937737,2.450614273548126,0.00011458185374735308,0.006492347001312737,0.021675833294779068,3.0569275617599487,1.2187799925183327,3.4588184277517486,17.226982575597194,1.0001146029861527,1.0066634190751644,1.0236432993708067,21.270205445874666,6.600743966167724,0.9750924597882691,0.28956696382373526,0.3427898381837402,1.0,0.9869791697710752,0.9880952388048172,1.0,0.6128815412521362,0.96875,0.13541666666666666,0.20833333333333334,1.0,nan,nan,0.9920366693598529,0.180925523241361,0.15641148291373005,2.995842114237354,0.0010530230029906066,0.014994581532920857,0.010807434661619939,0.0004884692892511483,6.286484860675301e-05,0.0014568593905285772,2.9669788992090984,20.41713028780694,0.4233174713785356,0.004631127461369502,85.74622547683751
|
| 33 |
+
162816,2000683008,0.13551222297246568,1.2558419182896614,2.436451356858015,0.00012776581460727243,0.0069445877693965485,0.016043899796052585,3.054756462574005,1.2301286906271223,3.5486759408096518,17.18324333118296,1.0001277902951184,1.007140633009385,1.0165844354151605,21.22350782002365,6.601344091623292,0.975773616917672,0.29173014365276834,0.3567631952552527,1.0,0.9869791697710752,0.9882936477661133,1.0,0.6205304861068726,0.96875,0.13541666666666666,0.22916666666666666,1.0,nan,nan,0.9935342107589046,0.16640198369471668,0.14110446461321166,2.991219144683542,0.0010441797793161623,0.014998830232729713,0.0107811759723901,0.0004930939921017097,6.236292345336473e-05,0.0014482730278423395,2.9623912441186935,20.335261624001856,0.42387759468793496,0.004100970320511953,85.72181720927014
|
| 34 |
+
167904,2063204352,0.1323710458673304,1.249154843389988,2.46707072108984,0.00014255009021368892,0.00559554355805858,0.030846368847655015,3.0540759563446045,1.224631661980216,3.5197398079598714,19.122859970460194,1.000142577756187,1.0057222234951477,1.0331489487079444,21.208979218561648,6.873603486988744,0.975773616917672,0.2968152726330021,0.353393676473042,1.0,0.9869791697710752,0.9853174611926079,1.0,0.6221267580986023,0.96875,0.13541666666666666,0.21875,1.0,nan,nan,0.9915850718195239,0.18193066914682277,0.1569559988177692,2.9891357101769476,0.001035110749702762,0.014972810762342793,0.01078950821765429,0.0004950678172105794,6.196595079627395e-05,0.0014504916725965368,2.9603307705794304,20.379971874772377,0.4237289923014787,0.003623353354756449,85.72764727623182
|
| 35 |
+
172992,2125725696,0.13610289287316846,1.280565857887268,2.690492607653141,0.00012718077391760119,0.004462859522874396,0.03145427668307832,3.053040325641632,1.2334537505425494,3.625961531635048,24.158226275401663,1.0001272062091031,1.004543196307103,1.0338880511735415,21.18648971538468,7.606098532379099,0.975773616917672,0.3157395539933382,0.3654188896913401,1.0,0.9869791697710752,0.9853174611926079,1.0,0.6388800144195557,0.96875,0.14583333333333334,0.22916666666666666,1.0,nan,nan,0.9915850718195239,0.17992832103239684,0.1543980451921622,2.9865173943071337,0.001036169225123684,0.014965062278781215,0.01078028962447721,0.0004948970080253617,6.154507514618875e-05,0.0014549717034730058,2.9577244731521457,20.32742215233779,0.42386907488459125,0.003201913898652705,85.69966410722367
|
| 36 |
+
178080,2188247040,0.13985454568319255,1.2338714301586151,2.3920831829309464,0.00013646546277001903,0.00481371011170495,0.030839479283947924,3.0503867864608765,1.2374355664431071,3.4681408308438106,15.173320123021785,1.0001364965044992,1.0049072719491625,1.0331805203973867,21.13152016731604,6.292662996639399,0.9668888858015712,0.3048913821991228,0.35192349254752553,1.0,0.9869791697710752,0.9853174611926079,1.0,0.625909686088562,0.9583333333333334,0.14583333333333334,0.21875,1.0,nan,nan,0.9915850718195239,0.17989377958777672,0.15495743144614002,2.985876795349631,0.0010371352699501517,0.014948381927946536,0.010786584852652846,0.0004903672095176785,6.100481780596278e-05,0.0014686442681920472,2.9570846904608064,20.354310397457027,0.42355330936821567,0.0028398614619212603,85.75724667261927
|
| 37 |
+
183168,2250768384,0.13805057391437003,1.2290145829319954,2.4343845546245575,0.00013837798210758478,0.0051798583557740585,0.03205333823776613,3.0513755679130554,1.2390370401884943,3.449135806030573,15.601648506606464,1.00013840884701,1.0052883380441409,1.034618056041602,21.152521218745484,6.354626767786253,0.975773616917672,0.3000529014275515,0.36736830281092003,1.0,0.9869791697710752,0.9853174611926079,1.0,0.6301917433738708,0.96875,0.13541666666666666,0.23958333333333334,1.0,nan,nan,0.991576440166682,0.17510162886658995,0.14986575534567237,2.9850382472406016,0.0010360340164144713,0.014916024737333902,0.010799257588003902,0.00048270700023138525,6.063136651708854e-05,0.0014649841276873905,2.956278623726055,20.284429437803023,0.42441508067924083,0.0025399532871463355,85.67581161909368
|
| 38 |
+
188256,2313289728,0.13509098043869017,1.2217688485980034,2.461438737809658,0.00015601777160867414,0.004825036333159005,0.03161030201975734,3.048763930797577,1.231920249391175,3.421627830534751,18.89181150112233,1.0001560568581702,1.0049189260129898,1.034109235003612,21.09690846356025,6.811636037497611,0.9750924597882691,0.2989657596192814,0.36050358643600167,1.0,0.9869791697710752,0.9853174611926079,1.0,0.6265439987182617,0.96875,0.13541666666666666,0.21875,1.0,nan,nan,0.991576440166682,0.17854810254842354,0.15441700827796012,2.9843498596089817,0.001031918732242448,0.01491645617114377,0.010781810663874063,0.00047825462321891615,6.0864102171084756e-05,0.0014599816794760552,2.955620589859362,20.32502286482907,0.42402334917781864,0.002304473351749283,85.75594097380629
|
| 39 |
+
193344,2375811072,0.1357037064008182,1.2352837473154068,2.5416753254830837,0.00013335583289197928,0.0052062228138822775,0.031498115009583216,3.0480307936668396,1.233953109956616,3.4699040138054587,20.52905106960165,1.0001333826441814,1.0053157925744736,1.033961003279917,21.08152414236092,7.050548930603317,0.9717070514636099,0.30500663625476315,0.36923379941107287,1.0,0.9869791697710752,0.9853174611926079,1.0,0.6338203549385071,0.9583333333333334,0.14583333333333334,0.22916666666666666,1.0,nan,nan,0.9915850718195239,0.17294981636223383,0.1469761449455594,2.9829456019654588,0.0010269572270273108,0.014915782661464962,0.010775470129271337,0.00047276145551351556,6.091320653112131e-05,0.0014633155233878075,2.9542304173978806,20.202231234349544,0.4246108138160604,0.0021352149741550937,85.68693451430606
|
| 40 |
+
198432,2438332416,0.13560970235266723,1.2995658814907074,2.464612640440464,0.0001402021785903429,0.004417591647552399,0.031710170515808045,3.047609031200409,1.2333170717290676,3.6988556371979797,17.836017871996926,1.0001402308078964,1.0044962291224433,1.034213922655249,21.072436792035184,6.697068250792107,0.975773616917672,0.30210868832848226,0.35079464523614134,1.0,0.9869791697710752,0.9853174611926079,1.0,0.6242386698722839,0.96875,0.13541666666666666,0.21875,1.0,nan,nan,0.991576440166682,0.1848170731876356,0.15963343426119536,2.9816516631476158,0.001030278290131118,0.01489436865149801,0.010773595187204608,0.0004760081939236855,6.108531388642789e-05,0.0014468663164859076,2.952969478583842,20.258875392330516,0.4242039655024807,0.0020334671566161833,85.67793143197329
|
training_logs/run_metrics_text67/01_bittoken_xl/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: True
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 2
|
| 5 |
+
cache_base_path: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/cache
|
| 6 |
+
combine_strategy: zero_pad
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/configs/config_bittoken_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_stem
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 1600
|
| 45 |
+
n_head: 25
|
| 46 |
+
n_layer: 48
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: bittoken
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: bce
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/configs/config_bittoken_text67.py --tqdm --verbose --seed 42 --run_name bittoken_gpt2_xl_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 48 --n_head 25 --n_embd 1600 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/trained/bittoken_gpt2_xl --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group bittoken_gpt2_xl_text67 --tie_word_embeddings --tokenizer_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_repo/tokenizers/num_text/bittoken_gpt2 --model rope_stem --num_embedding_type bittoken --add_reciprocal --combine_strategy zero_pad --num_loss_type bce --dropout 0 --cache_base_path /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/cache --compile --difficulty_column difficulty --train_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Multiplication_binary_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/DivM_binary_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Division_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard standard standard --train_set_ratios 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized_number --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Multiplication_binary_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/DivM_binary_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Division_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt pretokenized_number --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Sat Oct 3 14:42:14 2026'}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: bittoken_gpt2_xl_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/trained/bittoken_gpt2_xl
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_repo/tokenizers/num_text/bittoken_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/cache/bittoken_gpt2_47200109/96841076'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/cache/bittoken_gpt2_47200109/46368368'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/cache/bittoken_gpt2_47200109/63050008'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/cache/bittoken_gpt2_47200109/4733157'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/cache/bittoken_gpt2_47200109/44316829'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/cache/bittoken_gpt2_47200109/48579485'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/cache/bittoken_gpt2_47200109/63589654'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/cache/bittoken_gpt2_47200109/28473695'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/cache/bittoken_gpt2_47200109/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized_number
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Multiplication_binary_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/DivM_binary_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'pretokenized_number']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Multiplication_binary_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/DivM_binary_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-01/native/paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: bittoken_gpt2_xl_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/01_bittoken_xl/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_binary_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_DivM_binary_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_binary_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_DivM_binary_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_binary_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_DivM_binary_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_binary_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_DivM_binary_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_binary_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_DivM_binary_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,3.886174979913184,3.4975148880887446,3.924837720412218,3.561068730692644,3.989171063757226,3.8852959609138225,0.7803921699523926,1.9637031049967932,4.6169311159756035,1.0000026058270728,1.0000049317413398,1.000008753992967,1.000002738977106,1.0000038503254784,1.000007126373307,2.1823826046338346,1.0113424614708806,98.84597089659881,12.00441399666009,0.37147350790226535,0.02005525409411443,0.022423080901971905,0.017811165497865405,0.024656886105113334,0.1354166716337204,0.05598958022892475,0.08010693080723286,0.5633021437791802,0.03856769576668739,0.09375,0.0,0.0,0.005208333333333333,0.005208333333333333,0.48734959834837355,nan,nan,0.465210591042099,nan,nan,5.260019815193032,0.023741856338924863,0.02380347298409407,0.0268041982981269,0.022775132807491025,0.025830504869597044,0.008449785481402319,0.002962460841952318,0.020330490346996022,5.105321891652713,732.8965539429922,0.2503310373175502,0.004997126321997158,39.753441147844136
|
| 3 |
+
15264,250085376,4.409245904605731,3.9860301470021113,4.522786403891587,4.130978319697988,4.665992270204697,4.243210753526455,0.7274050414562225,2.1846322277250736,3.931759465718642,1.0000177389159313,1.000004694399572,1.0000092041895727,1.0000140406238391,1.0000113351722932,1.0000735175518531,2.069706502843478,1.0001907073626475,50.01844686991816,6.565386067886371,0.3834881867875785,0.1482615663876618,0.10956693418389765,0.1342831831015619,0.12039447942883047,0.1458333283662796,0.06901041977107525,0.12902970984578133,0.908458524436715,0.13968446850776672,0.109375,0.078125,0.046875,0.09895833333333333,0.020833333333333332,0.505981948498326,nan,nan,0.5116635244630743,nan,nan,4.63863938426759,0.021377059982248058,0.021738052083349217,0.025035557931140692,0.020754843686689745,0.024040465795218287,0.007151098003690532,0.0026838173039122806,0.015731123911335954,4.500127344987741,58.2024491677994,0.35059609592929325,0.009998084214664627,39.49715384264539
|
| 4 |
+
22896,375128064,5.1598530737270885,4.189113129238677,4.89833868108432,4.222424560521366,4.856655743396081,5.233103007543528,0.6722145974636078,2.6305644478146686,3.747989873207795,1.000004343967139,1.0000009428072418,1.000001157942507,1.0000015236293178,1.0000041193607991,1.0000484508191672,1.9585909037213431,1.0000559357172252,41.73543900171001,5.632682931074973,0.38633450355790655,0.20469035665450847,0.15148965519457905,0.18784875979658688,0.15058337052805149,0.2500000074505806,0.186197929084301,0.2236752063035965,0.9782621120637051,0.22003351151943207,0.10416666666666667,0.10416666666666667,0.052083333333333336,0.13020833333333334,0.03125,0.6177734472197093,nan,nan,0.6127787160221487,nan,nan,4.320950764842503,0.020242124473432675,0.020487696472893828,0.023917764621987875,0.019538233291772976,0.022932761829902216,0.006426209341692344,0.0025650906145662476,0.013430254523107714,4.191410610641031,40.98880774819876,0.38057411121950335,0.014999042107332095,39.422593777785735
|
| 5 |
+
30528,500170752,6.116540136766314,4.790236473068717,5.6825490886263506,4.672805471086823,5.440449059362475,5.9471665162903555,0.5780430734157562,2.9697816039539857,3.669133212029313,1.0000332828647591,1.0000019073357183,1.000002989604055,1.000002585717789,1.0000072920280854,1.0004843442215487,1.7825977985793917,1.0000167356772909,38.583006608922894,5.262905949439059,0.40306621765925743,0.26072012899681374,0.18192349745785774,0.24418431136265323,0.1922561738340759,0.4166666716337204,0.296875,0.31850504875183105,0.9868019063094674,0.29002833366394043,0.109375,0.140625,0.0625,0.171875,0.041666666666666664,0.7415654716702799,nan,nan,0.7003826666235303,nan,nan,4.129813517070915,0.019514498514156455,0.019606201242840023,0.023113816660126983,0.01869085528428981,0.022141353348128508,0.005851240022329075,0.0024321190361836554,0.011928507843999576,4.006534906553273,35.97536308860379,0.39218520957672,0.020000000000000004,39.46196006045265
|
| 6 |
+
38160,625213440,6.624732849496056,5.112791449761811,6.125782222470539,5.109895550314889,5.980161275393044,5.587164885591752,0.37915724515914917,2.811569951426195,3.5682455238808566,1.000000235932546,1.0000005870427684,1.0000010837332092,1.0000012746505162,1.0000010390349703,1.000001119367515,1.4646874426300394,1.0000004650807561,34.85731642814624,4.8135566306242845,0.41225574595492354,0.2774940913642696,0.20229427829208502,0.2859139844166768,0.2104534907165102,0.640625,0.5703125,0.4547513574361801,0.9946704083052177,0.3496420085430145,0.109375,0.15104166666666666,0.057291666666666664,0.19270833333333334,0.036458333333333336,0.878237675409764,nan,nan,0.7891102401384463,nan,nan,3.996597244435886,0.01899045952312664,0.018972090729248693,0.022533143611831796,0.01807006895981448,0.0215527481682583,0.0052288892112947625,0.002244660931586169,0.01063467759589725,3.8783704881960492,32.7909306312757,0.4009105580931127,0.019965729849129636,39.6736129941337
|
| 7 |
+
45792,750256128,6.278170763204471,5.002125647539469,5.951198477297783,4.942024568087357,5.765646279164901,6.084523760990731,0.164895661175251,3.156408546328337,3.508742996122843,1.0000000186264266,1.0000001459067909,1.0000001378355132,1.0000004166032848,1.0000005764851174,1.0000716913445287,1.179270430031305,1.0000185834573436,32.87309314825655,4.561383905394096,0.41955509502384386,0.2954936383031703,0.21250997155370704,0.28661134199399896,0.2144372387746037,0.8489583432674408,0.837239608168602,0.6281473934650421,0.9979296334156468,0.38148370385169983,0.125,0.16666666666666666,0.06770833333333333,0.19791666666666666,0.041666666666666664,0.9474283432743201,nan,nan,0.8663872193719726,nan,nan,3.8950371143650875,0.01859024260624277,0.018494573864984314,0.022100915030900396,0.017603865295360913,0.02111656264544176,0.004614821109255948,0.0020105947236901084,0.009484282870468057,3.78102123870623,30.373602991459006,0.4083168341014808,0.019863180383909898,39.434672418460856
|
| 8 |
+
53424,875298816,6.748517751693726,4.116169054060968,4.818503224907772,4.014681621454479,4.5603625078995975,5.015031223234171,0.09110441245138645,2.808075294823053,3.4606779305225546,1.0,1.0000000384944616,1.0000001629774555,1.0000000437719854,1.0000000099341027,1.0000611217356363,1.0968868604427535,1.0000000030532084,31.34836874162164,4.382812998003471,0.4285438056528434,0.3109190083966066,0.21768510273194502,0.282775409777007,0.22658270623464255,0.953125,0.954427070915699,0.8029788434505463,1.0,0.40114787220954895,0.11979166666666667,0.17708333333333334,0.06770833333333333,0.19791666666666666,0.046875,0.9764763328130357,nan,nan,0.921477792939792,nan,nan,3.574523463840047,0.017359858301157754,0.017181371739651594,0.020926472798457135,0.016325126801357767,0.01993887038747855,0.003345668660170919,0.0015840237302121793,0.006529903748348021,3.471332151122563,28.887743273246212,0.41353555063890834,0.019693132578928373,39.439638524563485
|
| 9 |
+
61056,1000341504,8.00415833791097,4.122728904088338,4.7007862726847325,3.8287556171417236,4.375657240549723,4.849328796068827,0.037593255983665586,2.558321121683634,3.4296623089661202,1.0,1.0,1.0,1.0,1.0,1.0,1.0388650602972533,1.0000000771250672,30.396156615555963,4.270557972553143,0.4454306127630704,0.30582933865610873,0.21995354858167965,0.297315838582788,0.2282596968802062,0.9635416865348816,0.973958320915699,0.8919353187084198,1.0,0.40925154089927673,0.15625,0.171875,0.06770833333333333,0.21354166666666666,0.052083333333333336,0.9843198902284106,nan,nan,0.9546109585401913,nan,nan,3.454360148514758,0.016876554906213363,0.016530248670933792,0.020321765606965837,0.015695305299052904,0.01933614601184901,0.002462394231662893,0.0012140773024304384,0.004996889326067662,3.356926751466875,27.730272375187784,0.41778484293,0.01945688144839207,39.4298821155867
|
| 10 |
+
68688,1125384192,6.821866830500462,4.056830406188965,4.579728047053019,3.8370273510615034,4.252539105402793,4.949051142980655,0.01000863453373313,2.8399405025954088,3.39740608347347,1.000000000310441,1.0,1.0,1.0,1.0000000270082223,1.0000000012417631,1.0100795834750964,1.000000000065356,29.435153949387814,4.160581506832077,0.4410254611641774,0.338888051125976,0.23595448464839153,0.32317220763080307,0.24017813501728053,0.9791666567325592,0.986979179084301,0.888801097869873,1.0,0.4311613142490387,0.125,0.203125,0.08333333333333333,0.234375,0.052083333333333336,0.9908210425637662,nan,nan,0.9514217572286725,nan,nan,3.3836591414935433,0.016528786233269005,0.016142476127976924,0.019959992548280965,0.015318995935205894,0.018968159628018042,0.0017089315874610036,0.0008467386717817375,0.0037920010121706457,3.290393043900394,26.76408363138355,0.42148062172082235,0.019156226183831337,39.383151333994356
|
| 11 |
+
76320,1250426880,4.4427705245709745,3.706037600835165,4.242511987686157,3.6340473095575963,4.070701599121094,4.485308326130793,0.03467506295419298,2.223957826633116,3.3746772932936437,1.0000000434613572,1.0,1.0,1.0,1.0,1.0000472854525322,1.0358760903629864,1.0000006740396745,28.78051190972467,4.0907151114490246,0.45530902695637077,0.3088920844381523,0.23449686141195392,0.31942261862535837,0.2392682860850069,0.9791666865348816,0.986979179084301,0.9199457466602325,1.0,0.4259410500526428,0.16145833333333334,0.16666666666666666,0.08854166666666667,0.22395833333333334,0.052083333333333336,0.9874366925409145,nan,nan,0.9642891641706228,nan,nan,3.330563996062672,0.016268218777260916,0.01592037493190881,0.01975106417574379,0.01509497389552046,0.01875672551256928,0.0010910375414574505,0.0005251816676312503,0.0028131684284833525,3.2403432355919017,26.13510350423789,0.4240631976943908,0.018793456452196412,39.401843459774945
|
| 12 |
+
83952,1375469568,3.982466659198205,3.395921190579733,3.5094477732976275,3.3996784687042236,3.5503460963567104,4.257544702850248,0.0015070164809003472,2.1376155672272796,3.3470133763039485,1.0000000012417631,1.0,1.0,1.0,1.0,1.0000000263874407,1.0015081533226362,1.000000000065356,28.000927603729007,4.000270642749578,0.45919649081136804,0.3371582694591073,0.24607779937822047,0.3205460551086563,0.2425172910856374,0.9895833134651184,1.0,0.9481668770313263,1.0,0.43868789076805115,0.16666666666666666,0.19270833333333334,0.078125,0.21875,0.057291666666666664,0.9957898599095643,nan,nan,0.9729435096184412,nan,nan,3.2891496850124273,0.01603866310072327,0.01576022909582247,0.0196040422446729,0.01494405594144139,0.018611188280766484,0.0006938202809645532,0.00029004197372655425,0.0021214314825438144,3.201086195996701,25.54893152618808,0.4264631153629074,0.018371334958694578,39.400080761801334
|
| 13 |
+
91584,1500512256,4.228034886220099,3.623920481341581,3.6985570366183422,3.5853136814932616,3.691082398407161,4.8401764676533885,0.01671362851629965,2.368669516000842,3.3333268192557926,1.000000072021292,1.0000000009313226,1.0000000173846388,1.0000003215970223,1.000000000310441,1.000064936160497,1.0169873686588793,1.0000070451627288,27.60397181684081,3.9578923976741813,0.4616826894507956,0.31987596414630237,0.24320372354262776,0.3307485426186341,0.24742245808222024,0.9895833432674408,0.993489570915699,0.9719105660915375,1.0,0.43870168924331665,0.15625,0.1875,0.09375,0.23958333333333334,0.057291666666666664,0.9974222055946788,nan,nan,0.9834732247206072,nan,nan,3.257432898640591,0.015835161947278836,0.015642005109841917,0.01948133585309111,0.014828835041304514,0.018492365176877322,0.0004903337538327729,0.00016285650244818076,0.0016966797691739047,3.1708033087785052,25.084854819364768,0.4284465402575596,0.017893076407163026,39.43071959051096
|
| 14 |
+
99216,1625554944,4.3021076989049645,3.70800833000491,3.728024760882059,3.591908812522888,3.733983437530696,5.002489884694418,0.001351613667793572,2.484707378205799,3.3192512734870734,1.000000007140133,1.0000000009313226,1.0,1.0,1.000000000310441,1.0,1.0013530218036302,1.0,27.243941912727948,3.9161438825459416,0.46462522867233563,0.3382454785941117,0.2661119841600118,0.32100570570859377,0.2769747031381441,0.9947916567325592,1.0,0.9614939093589783,1.0,0.4583079218864441,0.15625,0.19791666666666666,0.09895833333333333,0.22916666666666666,0.109375,0.9981727311387658,nan,nan,0.9792048879899085,nan,nan,3.2314309475306944,0.015649496180761623,0.015537049010224187,0.019372155319100116,0.014731213117293638,0.018387551286588418,0.00040013389329952116,0.00010811290103318093,0.0014422950808521476,3.145802923158634,24.683252127353,0.4302160878093675,0.017362323018206012,39.42063252586921
|
| 15 |
+
106848,1750597632,4.7409991423288975,3.9425985021516676,4.132848978973925,3.7679980210959902,4.012636475572752,5.733714898427326,0.001710954587906599,2.6247642153785344,3.3022503872246793,1.0,1.000000000310441,1.0000000009313226,1.0000000012417631,1.0000001319346676,1.0,1.0017129785098846,1.0,26.788297182501584,3.8655566994921844,0.4766129795938165,0.3369212387102985,0.2680331558523279,0.3258665179175127,0.2751951498688555,0.984375,1.0,0.9629029631614685,1.0,0.4602639675140381,0.17708333333333334,0.19791666666666666,0.109375,0.21875,0.09375,0.9944431744515896,nan,nan,0.981752316467464,nan,nan,3.2092360761055656,0.015479339889750525,0.01544581959137965,0.01926834803589722,0.014647956025428097,0.018288261737335394,0.00035264844551410375,8.838315995046087e-05,0.0012826350403616564,3.1243826660381626,24.245924942148555,0.4320949299272689,0.016783116791537683,39.48427145271118
|
| 16 |
+
114480,1875640320,5.213934580485027,3.925733367912471,4.290604591369629,3.80515142747475,4.180854956619441,7.086023489634195,0.004522114322753623,3.928381147838774,3.283839452565492,1.0,1.000000000310441,1.0,1.00000004439268,1.000000000310441,1.0,1.0045394495535134,1.0,26.29754287321703,3.8113424853093454,0.47753902995938663,0.3353923165817353,0.274671702266031,0.3351524683987658,0.2915939496389316,0.9947916567325592,0.993489570915699,0.9644569158554077,1.0,0.4692840874195099,0.17708333333333334,0.18229166666666666,0.10416666666666667,0.24479166666666666,0.11458333333333333,0.9974584107597669,nan,nan,0.9857065884085993,nan,nan,3.1898898603765593,0.015315539793796044,0.015366532877582217,0.01917331048647354,0.014572320520864171,0.018196104403688025,0.00031750289571875145,7.826378256547638e-05,0.0011685097828825167,3.105701757880002,23.80692849931477,0.4339996607626634,0.016159868723772935,39.43288119057273
|
| 17 |
+
122112,2000683008,5.078071157137552,3.974624077479044,4.3849047025044765,3.8194727897644043,4.23966447574397,6.4352158705393485,0.0016198466473724693,3.3516321342934585,3.2769821965678907,1.0,1.0,1.0,1.0,1.0000000006208816,1.0,1.0016221143423314,1.0000000614814275,26.12328053657092,3.7916558570017287,0.48582628267078254,0.3472997456485397,0.2781821215586285,0.3403510750883051,0.29738888141504605,0.9947916567325592,1.0,0.986659824848175,1.0,0.4774036109447479,0.203125,0.19270833333333334,0.109375,0.2552083333333333,0.125,0.9970316933467984,nan,nan,0.992729383520782,nan,nan,3.1720809049977237,0.015144149412838372,0.015274597097749242,0.01907252542108979,0.01448942347547491,0.018088325471366715,0.000285772990304926,7.087265725638465e-05,0.0010515825081061044,3.088603637430486,23.46832770796442,0.4356547641124678,0.015497325216091833,39.31353708988276
|
| 18 |
+
129744,2125725696,4.9779423239329965,4.175894061724345,4.539364178975424,4.001238942146301,4.4068217277526855,6.43010139465332,0.000937519347644411,3.3508686792936393,3.2617070872802287,1.0000000161428757,1.0,1.0,1.0,1.0,1.0,1.000938179267778,1.000000000065356,25.732353427219007,3.74814351363278,0.48261718734538994,0.34681159761698543,0.2701717406481454,0.3413576076948826,0.29092269627505535,0.9947916567325592,1.0,0.9822218716144562,1.0,0.47251659631729126,0.18229166666666666,0.19791666666666666,0.10416666666666667,0.25,0.09895833333333333,0.9993034539123377,nan,nan,0.9890244978790482,nan,nan,3.156113569083891,0.01498508793121278,0.015197158926902021,0.01897934797080982,0.014414002707148014,0.017996232673622783,0.0002614891901715752,6.667159022227611e-05,0.0009633196310958619,3.0732502404946622,23.19777737336089,0.4368614270986376,0.014800531927597655,39.46007503067491
|
| 19 |
+
137376,2250768384,5.36064108212789,4.218136668205261,4.636204854895622,4.0418970584869385,4.487320002789248,7.198657194773357,0.0006575824081664905,3.8654381943262783,3.2395522986189462,1.0,1.0,1.0000000161428757,1.0,1.000000016763757,1.0,1.0006578215985344,1.0000000055009437,25.18129138829116,3.686883249810808,0.4912775752412476,0.36201448411299386,0.2768253302502901,0.3484493117909513,0.3160000622166401,1.0,1.0,0.9836967885494232,1.0,0.4875088632106781,0.18229166666666666,0.21354166666666666,0.109375,0.2552083333333333,0.140625,1.0,nan,nan,0.9904362857341766,nan,nan,3.1409687046026704,0.014826370366643344,0.015124620169636765,0.01889069420492158,0.014347721217704546,0.017908927931900673,0.0002399662575772562,6.303759152138817e-05,0.0008860045214065097,3.0586813444455214,22.88441239494198,0.4381629602721965,0.014074795349650518,39.558428714379616
|
| 20 |
+
145008,2375811072,5.141699711481731,4.170934756596884,4.485908349355062,3.943589925765991,4.297952259890732,6.857312520345053,0.0003872597881127149,3.557158424740746,3.230669892063209,1.0,1.0,1.0,1.0,1.0000000052774904,1.0,1.0003873348430163,1.0,24.955551182117237,3.661770946915305,0.49513436804166633,0.3504780241631714,0.28931400186367495,0.34270826555278455,0.30953181199695706,0.9947916567325592,1.0,0.9851322174072266,1.0,0.48653116822242737,0.19270833333333334,0.203125,0.13020833333333334,0.23958333333333334,0.11979166666666667,0.9992428394034505,nan,nan,0.994167630871137,nan,nan,3.1260541516384155,0.014661802054847546,0.01505042868837191,0.018804595368598007,0.014277450966415762,0.017819706315095295,0.00022148727987274762,5.9404240904270906e-05,0.0008171133991826156,3.0443421471096017,22.554677629558295,0.439937955026263,0.013325642393809753,39.60912026531689
|
| 21 |
+
152640,2500853760,5.3443942467371635,4.48759197244605,4.991519651375711,4.274471799532572,4.7809414127841565,7.401302655537924,0.011164456155711378,3.8835743741963555,3.222170876979362,1.0,1.000077482599277,1.0000000015522041,1.0,1.0000000058983738,1.0,1.0112897123163798,1.0000000246567358,24.75149022242222,3.640317494382799,0.49737322442570864,0.3604345435393982,0.2848287525336974,0.3543651681209826,0.3178863721547402,0.9947916567325592,0.993489570915699,0.9881479144096375,1.0,0.4921426475048065,0.1875,0.20833333333333334,0.11458333333333333,0.2552083333333333,0.13020833333333334,0.9967450182884932,nan,nan,0.9945976184681058,nan,nan,3.112355234417713,0.014500888662444002,0.014973729078128809,0.018720536913438342,0.014202290684977905,0.017727282082930125,0.0002046124437163676,5.656404560426285e-05,0.0007584306569978775,3.0312108839207976,22.310033667262495,0.44103539406675335,0.012558778301146429,39.71311118022614
|
| 22 |
+
160272,2625896448,5.173884232838947,4.420727094945809,4.933160747090972,4.199047685290377,4.7588519267737865,6.3947753111521415,0.0005305162158038002,3.4172517912728444,3.2075417291101376,1.0,1.0000000009313226,1.0000000049670508,1.0000000006208816,1.0000000012417631,1.0,1.000530760154812,1.0,24.389738365660165,3.598918792619555,0.5052287041733952,0.3631688155162184,0.2997997424229016,0.3539550756060967,0.3363992892065519,1.0,1.0,0.9911372661590576,1.0,0.5034903287887573,0.21875,0.20833333333333334,0.125,0.25,0.16666666666666666,1.0,nan,nan,0.9957090284054478,nan,nan,3.099552579174647,0.014347184022984861,0.01489210395644543,0.01863400544377355,0.014128593493407237,0.01763177436488797,0.0001903653143423563,5.40710927998313e-05,0.0007048380315285889,3.018969627905567,22.030453636273897,0.44250520339833116,0.011780043193471874,39.601193891242204
|
| 23 |
+
167904,2750939136,5.435326894745232,4.686301946640015,5.2387841542561855,4.467894633611043,5.127938985824585,5.9957945346832275,0.00020648715599236311,3.5285357066563194,3.1975965213108184,1.0000000006208816,1.0,1.0,1.0,1.0,1.0,1.0002065243285934,1.0,24.155824962136126,3.5728923874539555,0.5049259607436088,0.36890190145355783,0.29082481174258357,0.3528341736798741,0.32111859072518556,1.0,1.0,0.992572695016861,1.0,0.4976356327533722,0.21354166666666666,0.21354166666666666,0.11979166666666667,0.24479166666666666,0.14583333333333334,1.0,nan,nan,0.9963979115709662,nan,nan,3.087135545646474,0.014197795731334405,0.014813859886824982,0.018548928829320657,0.014053809461896494,0.017539585778485803,0.00017812014283034728,5.203115218063549e-05,0.0006565602608397338,3.007094839188204,21.772896816458093,0.44386280445182846,0.010995367597369719,39.59143691788297
|
| 24 |
+
175536,2875981824,5.423363248507183,4.73477065594246,5.380975405685603,4.527192645652292,5.237599457613759,6.062571803728741,0.0023365928209386766,3.5506213278997514,3.1830559360290254,1.0,1.000000000310441,1.000000000310441,1.0000005298688812,1.0000000450135604,1.0,1.0023394895183644,1.0,23.816138592439227,3.5353865174956574,0.5067361062663593,0.37000658829305444,0.31407452043452827,0.35496382211378624,0.3209197118689985,0.9947916567325592,1.0,0.9911109209060669,1.0,0.5054060816764832,0.20833333333333334,0.203125,0.13020833333333334,0.24479166666666666,0.140625,0.9971723036530117,nan,nan,0.9944892587761084,nan,nan,3.074479753777624,0.014052608830250337,0.0147253165268362,0.018457536685773823,0.01397598134967154,0.01744062344561208,0.0001671717331120892,4.988467488363403e-05,0.000613268815929347,2.994997346704757,21.487300551225054,0.4453354034947712,0.01021072727974235,39.534954606552716
|
| 25 |
+
183168,3001024512,5.513161818186443,4.8033616940180455,5.482625762621563,4.583432873090109,5.317442046788699,5.662297606468201,0.00011974399239988998,3.4701474507649746,3.1752954069136954,1.0,1.0,1.0,1.0,1.0000000270082223,1.0,1.000119751706952,1.0,23.631356285018786,3.5146084515259957,0.5106078763049109,0.3708253605176306,0.31404548514915687,0.35555799407072763,0.3449709336810036,1.0,1.0,0.9925858378410339,1.0,0.5125829577445984,0.21875,0.21875,0.140625,0.25,0.17708333333333334,1.0,nan,nan,0.9957615549986562,nan,nan,3.063184347357744,0.013915658717222348,0.014639360859800079,0.01836931181223549,0.013894829562354955,0.017339832634306468,0.00015754068655009505,4.797546149439926e-05,0.0005769866271502823,2.984242836566633,21.397574426867926,0.44593237468922037,0.009432097738825898,39.58848487349719
|
| 26 |
+
190800,3126067200,5.742945869763693,5.039057334264118,5.810231367746988,4.800563255945841,5.687709650024771,6.105516354242962,6.648204180237371e-05,3.717158408392043,3.1639853516826406,1.0,1.0,1.0,1.0,1.0000000006208816,1.0,1.0000664843238662,1.0,23.36518319686652,3.4850277424234744,0.5145996333244549,0.3775595744850335,0.3247190789874581,0.37014802476158687,0.3398280172135121,1.0,1.0,0.9896360039710999,1.0,0.5194447636604309,0.203125,0.21354166666666666,0.140625,0.265625,0.15625,1.0,nan,nan,0.9938604834799966,nan,nan,3.052034962445935,0.013782646690799896,0.01455490321707134,0.0182791305238942,0.013818790163902748,0.017240931382624055,0.00014965019347348596,4.6491714027434746e-05,0.0005453814383860084,2.9736170218717297,21.08169178436637,0.44729519715099214,0.008665408697250875,39.61110175492055
|
| 27 |
+
198432,3251109888,5.620229051448401,4.934088786443075,5.715261499087017,4.763915141733984,5.617284582927819,5.716151595115662,7.752731289656367e-05,3.451796145666213,3.1566960050646835,1.0000000058983716,1.0,1.0,1.000000000310441,1.0000000068296948,1.0,1.0000775311126175,1.0,23.20633432787054,3.467379096891296,0.519564963499473,0.37860614487652394,0.3184141609264488,0.3739783457759703,0.3650031822589866,1.0,1.0,0.9940871298313141,1.0,0.5255130529403687,0.21875,0.21875,0.140625,0.2604166666666667,0.1875,1.0,nan,nan,0.9967545997351408,nan,nan,3.041649537131055,0.013657937149764876,0.01446727555657455,0.018184518692692823,0.013744142138923743,0.017139278560685506,0.00014365563867949253,4.5415235030552595e-05,0.0005177055590447364,2.963749593742072,20.971076210714735,0.4480525364306613,0.007916498943711495,39.590392514800435
|
| 28 |
+
206064,3376152576,5.785516699155172,5.029836174099649,5.8415255943934135,4.866334399518868,5.715295475907624,5.799745321273804,9.195556776830927e-05,3.494281541733515,3.151857055296811,1.0,1.000000075746587,1.0,1.0000000009313226,1.000000002173085,1.0,1.0000919599363403,1.0,23.096234816636837,3.4551474283804637,0.5203651585453468,0.3933852722946715,0.3160172954598027,0.37566025575090956,0.34822080175899545,1.0,1.0,0.9970501363277435,1.0,0.5243279933929443,0.234375,0.22916666666666666,0.140625,0.265625,0.15104166666666666,1.0,nan,nan,0.9979975270107388,nan,nan,3.032469457459254,0.013539774676428223,0.014380532106811573,0.01808895478290288,0.01365789950266871,0.017035224326010234,0.00013824600162293106,4.4682632446254835e-05,0.0004934632769088412,2.955090665230482,20.8605424937962,0.4487480732536166,0.007191071867150021,39.590515290865085
|
| 29 |
+
213696,3501195264,5.838533242543537,5.14047821735342,5.975492795929314,4.976663589477539,5.87445422634446,6.091727415720623,5.429079374152934e-05,3.7851653780256003,3.1461324282572605,1.0,1.0000000037252876,1.0000000006208816,1.0,1.0000000459448857,1.0,1.0000542928192737,1.0,22.969079201582076,3.441014838299156,0.5220376987052222,0.3977179072554706,0.3277550129387903,0.37628237001283527,0.3521944999281049,1.0,1.0,0.998525083065033,1.0,0.5300549864768982,0.234375,0.22916666666666666,0.140625,0.28125,0.15625,1.0,nan,nan,0.9994105901569128,nan,nan,3.0239375866324774,0.013432114662927628,0.01428936790470893,0.01798897681111544,0.013574894321926203,0.016927525990595098,0.0001337954958325551,4.4251673147181686e-05,0.00047360008353594104,2.9470730448580347,20.696966874836875,0.44963477151292675,0.006494652022090374,39.576633419641155
|
| 30 |
+
221328,3626237952,5.851189892428618,5.186402003280819,6.01173738669604,5.007764102580646,5.878153453270584,5.835917393366497,5.8624685152608436e-05,3.547399838765463,3.1408653988231285,1.0000000009313226,1.000000000310441,1.000000000310441,1.0000000015522041,1.000000009934097,1.0,1.000058627066298,1.0,22.85343433439363,3.4281658860553814,0.5248346602193672,0.39743577085584336,0.3348248716099611,0.3862780684466732,0.34843955488090833,1.0,1.0,1.0,1.0,0.5335902571678162,0.234375,0.234375,0.14583333333333334,0.2760416666666667,0.16145833333333334,1.0,nan,nan,0.9999694712460041,nan,nan,3.0171090414022586,0.013333329053027497,0.01420997338422205,0.01789684463171959,0.013497456801979744,0.016826036258468534,0.00013059695390400903,4.362171965740112e-05,0.0004614682095826677,2.9407096987214016,20.647887138200755,0.44963917671165876,0.005832543055900948,39.604118599306105
|
| 31 |
+
228960,3751280640,5.897963841756184,5.199633439381916,6.067133149132132,5.004808505686621,5.916863163786655,5.872481465339661,7.026528737696935e-05,3.5361540885198686,3.136359247379005,1.0,1.0,1.0000000006208818,1.000000000310441,1.0000008746765991,1.0,1.000070269449064,1.0,22.754177284246847,3.4171387143670926,0.528971949471488,0.39740232274022286,0.3295325542367403,0.3751765961103839,0.3636208410461599,1.0,1.0,1.0,1.0,0.5339050889015198,0.234375,0.22916666666666666,0.15104166666666666,0.2552083333333333,0.18229166666666666,1.0,nan,nan,1.0,nan,nan,3.0098717615091655,0.013241313018253272,0.01412138904213507,0.017799473097704084,0.013415520905283071,0.016721860561610978,0.00012910026367449028,4.335788475138147e-05,0.000449238086488324,2.9339504928427673,20.48492289970256,0.45068828511272463,0.005209787318395601,39.56314984417559
|
| 32 |
+
236592,3876323328,5.718710823915897,5.073815981547039,5.89435915183276,4.91934577624003,5.787517619168586,5.963396231333416,2.8305008072493365e-05,3.6190065883454823,3.13350680328828,1.0000000040357278,1.0,1.000000000310441,1.0,1.0000023762797432,1.0,1.0000283055528483,1.0,22.68532905474311,3.4094844156579858,0.5268947897307548,0.4031859771136628,0.3350726363035599,0.3752882447272173,0.3597755725527111,1.0,1.0,0.9940871298313141,1.0,0.5353108048439026,0.234375,0.234375,0.13541666666666666,0.2604166666666667,0.171875,1.0,nan,nan,0.995753304567188,nan,nan,3.00465763777307,0.01316009382365869,0.014045259065566171,0.017711308009087504,0.013341573949723948,0.016627438626659528,0.00012742933879970574,4.299558547496347e-05,0.0004409303760789359,2.929160593688009,20.45338549235332,0.4508815153425782,0.004631127461369502,39.551844075700444
|
| 33 |
+
244224,4001366016,5.723026593526203,5.1200830936431885,5.94354510307312,4.961123466491699,5.832826739176744,5.937238732973736,2.868001138267573e-05,3.5382295790172757,3.127911857329309,1.0,1.0,1.0,1.0,1.0000000453240028,1.0,1.0000286805821812,1.0,22.565724081657233,3.396194756395935,0.5312467607937184,0.4068397959887665,0.3364606288038603,0.3799752914399184,0.36352333371842804,1.0,1.0,0.9985119104385376,1.0,0.5390336513519287,0.22916666666666666,0.234375,0.15625,0.265625,0.171875,1.0,nan,nan,0.9987393335128824,nan,nan,2.9998355515113793,0.013086440878045556,0.013977164278704358,0.017631121035349622,0.01327306940610788,0.016541890744842292,0.00012621929239147408,4.273659605184528e-05,0.00043543044241955834,2.924721462944956,20.380092538752645,0.4513098594793109,0.004100970320511953,39.56700507781456
|
| 34 |
+
251856,4126408704,5.8922769625981655,5.241761406262716,6.074867049853008,5.0438209374745675,5.9264611536097505,5.907166798909504,2.6913814508588985e-05,3.487361953372047,3.1283127911156043,1.0,1.0,1.0,1.0,1.0000005298688812,1.0,1.0000269143194838,1.0,22.570106511479935,3.396681550629811,0.5283702285199623,0.40761336909940993,0.3414245441391286,0.3849386549683983,0.35700778116089404,1.0,1.0,0.9970369935035706,1.0,0.5396760702133179,0.22916666666666666,0.24479166666666666,0.15104166666666666,0.265625,0.17708333333333334,1.0,nan,nan,0.9979346776381135,nan,nan,2.9954004852162384,0.013029829209651223,0.01391604110352229,0.01755758463489679,0.013221117932696657,0.016466891134852852,0.00012531287080008228,4.249350844173665e-05,0.00043184119338730457,2.9206093570814944,20.312477459260254,0.45171569147590446,0.003623353354756449,39.67136743264805
|
| 35 |
+
259488,4251451392,5.772683024716874,5.11196990807851,5.920718312263489,4.947510361671448,5.796195680275537,5.813024838765463,3.0788306503382046e-05,3.5331324170555494,3.126083838734,1.000000000310441,1.0,1.0,1.0,1.000000014280258,1.0,1.0000307888820954,1.0000000001941518,22.52467002489907,3.391633425396224,0.5297553206240442,0.4123717244481385,0.33983303084982186,0.3829003764063741,0.3648612143387819,1.0,1.0,1.0,1.0,0.5419175028800964,0.22916666666666666,0.24479166666666666,0.14583333333333334,0.2552083333333333,0.18229166666666666,1.0,nan,nan,0.9999694712460041,nan,nan,2.992350630191429,0.012976250901104327,0.01386138029392629,0.017492776870197773,0.013171275965605845,0.016400772162907053,0.00012407688684667892,4.1985785654492505e-05,0.0004287693454432993,2.917853325522088,20.325278223872935,0.45159626672477843,0.003201913898652705,39.57731082921799
|
| 36 |
+
267120,4376494080,5.882220069567364,5.223233103752136,6.0773080587387085,5.049000183741253,5.957518418940405,5.92125693957011,3.087358845732524e-05,3.564006033397856,3.1239501370388707,1.0,1.0,1.0,1.0,1.000000000310441,1.0,1.0000308741891137,1.0,22.480769943833877,3.3867556464814923,0.5288416135793419,0.4052162545751071,0.33971395891468115,0.3809349091792196,0.3704180571558018,1.0,1.0,1.0,1.0,0.5412827730178833,0.22916666666666666,0.234375,0.15625,0.265625,0.19270833333333334,1.0,nan,nan,1.0,nan,nan,2.9896655571233723,0.012936971392402795,0.01381743269136851,0.01743604512159549,0.013130383670275053,0.016341250040649763,0.00012312903038650533,4.1872995581736065e-05,0.00042431216176783907,2.915414144447077,20.304411179302125,0.451600159809337,0.0028398614619212603,39.511194722906325
|
| 37 |
+
274752,4501536768,5.830161572123568,5.214961767196655,6.0720378160476685,5.035195350646973,5.933963061310351,5.902261734008789,3.247563108743634e-05,3.5631966590881348,3.1233620314936466,1.0000000006208816,1.0,1.0,1.0,1.0000000009313226,1.0,1.0000324763252144,1.0,22.463611228052077,3.384849300658833,0.5272622520536043,0.405443464996146,0.3365513951956102,0.3826151767456093,0.37040238520875496,1.0,1.0,0.9985119104385376,1.0,0.5405669212341309,0.234375,0.234375,0.15104166666666666,0.265625,0.1875,1.0,nan,nan,0.9996640638758739,nan,nan,2.9877371649296234,0.012905219222305395,0.01377620454576611,0.01739702072462482,0.01309911084990554,0.0162990105061471,0.00012185739508625392,4.1767472685721905e-05,0.000423291981668271,2.913673666527769,20.248319953367645,0.4519189534556566,0.0025399532871463355,39.47715105301155
|
| 38 |
+
282384,4626579456,5.815877125598485,5.175997018814087,6.024375557899475,4.971752564112345,5.8749092376470635,5.849146644274393,4.334337654654519e-05,3.500588734944662,3.1217610741344592,1.0000000058983698,1.0,1.0,1.0,1.0000000751257068,1.0,1.0000433447507164,1.0,22.430280636930846,3.3811471180784043,0.5296010327007653,0.40344601099000127,0.3392553523602977,0.38025920051423967,0.3696101869745395,1.0,1.0,1.0,1.0,0.5405460596084595,0.234375,0.234375,0.140625,0.2552083333333333,0.1875,1.0,nan,nan,1.0,nan,nan,2.9860368049374872,0.012880147795691098,0.01373914034795055,0.017360218836643466,0.013064755371867423,0.016258439058805658,0.00012171042356880865,4.166060729532829e-05,0.00042073916206012844,2.9121499769207397,20.251469557827622,0.45208693220352353,0.002304473351749283,39.44773870746108
|
| 39 |
+
290016,4751622144,5.826674302729466,5.2089919646581,6.064109404881796,5.0244732697804775,5.927968820246556,5.887588858604431,5.5342737596220104e-05,3.5223000390189037,3.122147540833491,1.000000000310441,1.0,1.0,1.0,1.000000000310441,1.0,1.000055345162365,1.0,22.438504591792356,3.382062215286178,0.5314828744956039,0.4111248983091999,0.3404378341269598,0.3873385716974886,0.36986330861291794,1.0,1.0,1.0,1.0,0.5442441701889038,0.22916666666666666,0.23958333333333334,0.15625,0.2708333333333333,0.1875,1.0,nan,nan,1.0,nan,nan,2.9852185412606187,0.012859604705484995,0.013713780936025537,0.017333952973939738,0.013042010119686634,0.016229144031273696,0.00012130409421022236,4.1683234988303935e-05,0.000418786695727052,2.911458258038734,20.275893268845117,0.4519464341607296,0.0021352149741550937,39.41399482189225
|
| 40 |
+
297648,4876664832,5.854490598042807,5.209081967671711,6.073200305302937,5.018773277600605,5.921093026796976,5.931278467178345,4.216315255689551e-05,3.549978392464774,3.1216088882453428,1.0,1.0,1.0,1.0,1.0,1.0,1.0000421645629087,1.0,22.42906435759927,3.381011835795798,0.528392870744641,0.40514766486868437,0.3410521290837426,0.38517530142216594,0.3677990133887429,1.0,1.0,1.0,1.0,0.5419085621833801,0.234375,0.234375,0.140625,0.2604166666666667,0.18229166666666666,1.0,nan,nan,1.0,nan,nan,2.984780516629032,0.012838062683590912,0.013687387561691751,0.017314566227241018,0.013017711924046672,0.016210525563139025,0.00012156432268439253,4.160617355608785e-05,0.00041728916490905147,2.911131786878899,20.268649197911817,0.4519246232186286,0.0020334671566161833,39.445789242748425
|
training_logs/run_metrics_text67/02_bpe_xl/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: False
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 2
|
| 5 |
+
cache_base_path: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/cache
|
| 6 |
+
combine_strategy: sum
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty_sd
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/configs/config_td_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_gpt2
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 1600
|
| 45 |
+
n_head: 25
|
| 46 |
+
n_layer: 48
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: bittoken
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: mse
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/configs/config_td_text67.py --tqdm --verbose --seed 42 --run_name bpe_gpt2_xl_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 48 --n_head 25 --n_embd 1600 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/trained/bpe_gpt2_xl --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group bpe_gpt2_xl_text67 --tie_word_embeddings --tokenizer_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_repo/tokenizers/num_text/td_gpt2 --model rope_gpt2 --dropout 0 --cache_base_path /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/cache --compile --difficulty_column difficulty_sd --train_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Division_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard --train_set_ratios 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Division_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_prompt efficient_prompt efficient_prompt efficient_prompt efficient_prompt efficient_prompt pretokenized --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Sat Oct 3 14:42:12 2026'}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: bpe_gpt2_xl_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/trained/bpe_gpt2_xl
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_repo/tokenizers/num_text/td_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/cache/td_gpt2_88525419/96841076'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/cache/td_gpt2_88525419/63050008'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/cache/td_gpt2_88525419/44316829'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/cache/td_gpt2_88525419/48579485'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/cache/td_gpt2_88525419/63589654'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/cache/td_gpt2_88525419/28473695'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/cache/td_gpt2_88525419/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'pretokenized']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-02/native/paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: bpe_gpt2_xl_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/02_bpe_xl/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,3.3425554633140564,4.120983342329661,3.8741276264190674,2.5529162089029946,0.7359418620665868,4.445624848206838,4.385375022888184,29.302815610081623,72.5743208939677,59.20024919843093,13.442198410598573,2.090342644618917,150.56230697991617,80.68529191446501,58.26536080743985,0.31569491774778596,0.008322646292385459,0.015061137936339319,0.3177083333333333,0.13932291666666666,0.322614128390948,0.6707343294384381,0.037944626063108444,0.09895833333333333,0.0,0.0,0.5991308814045624,nan,nan,0.6066849184862804,0.12140693872894796,0.09894004392360027,5.368776317974806,0.047255014200681185,0.05261206303662631,0.04180865429499942,0.021727020672641584,0.0018288808882961822,0.049733980010782,5.153810672243311,639.1531038824118,0.21174070061267442,0.004997126321997158,38.59461401253636
|
| 3 |
+
15264,250085376,3.149580697218577,4.023216744263967,3.7036112745602927,1.6637665654222171,0.7291304568449656,3.611341526110967,3.8503588438034058,32.20816690044226,69.41864679375341,53.573060223662544,8.090062333600164,2.075233943066326,60.65064861572279,47.2831813818404,39.04271431315542,0.4051120514869558,0.12721726590285662,0.1288741320435994,0.703125,0.09895833333333333,0.5765120064218839,0.9328537755398862,0.2158990204334259,0.1875,0.078125,0.036458333333333336,0.8458286861544669,nan,nan,0.7804328400767796,0.17214970968026932,0.14856708948112404,4.635512618523724,0.0376306387807674,0.04686152951721594,0.03683933738298886,0.012116380257199045,0.0016100490858921256,0.027962925330382717,4.472491729067369,53.1082531651111,0.3358227004584089,0.009998084214664627,38.52804060961926
|
| 4 |
+
22896,375128064,2.8871656358242035,3.9326178630193076,3.5761327147483826,0.9321132736901442,0.6048606435457865,1.9570469359556835,3.6870805819829306,22.814056561680818,69.53388968807842,44.701833678408484,3.071907626571342,1.835174180449778,9.000857399122122,40.164542295797474,27.303180204301203,0.4658729638218139,0.14365575633942892,0.1686063360826081,0.8541666666666666,0.2578125,0.7458129276831945,0.9835848309608516,0.3118651807308197,0.2604166666666667,0.078125,0.06770833333333333,0.9143015179264088,nan,nan,0.8672552271649087,0.1741129905258371,0.15036631253315136,4.297067242837885,0.03297288556193385,0.0438565906969626,0.0341573240650996,0.008565174501812775,0.0015022507096977729,0.019948633022046344,4.156064360073861,39.54426305336522,0.36352749627310027,0.014999042107332095,38.4184682229668
|
| 5 |
+
30528,500170752,2.5141910910606384,3.9923749963442483,3.4986358980337777,0.5156365477014333,0.3311871886253357,1.7499707241853077,3.610133091608683,17.481769024034705,73.93345511933117,42.42520328462941,1.7895422806103192,1.39754878073171,6.632403771588483,37.181407968666235,25.834475747084575,0.515524037445665,0.1885485764860277,0.20482662591583103,0.90625,0.6875,0.811258797844251,0.989121286769338,0.4135339558124542,0.296875,0.109375,0.09375,0.9386773429675183,nan,nan,0.8893581640101426,0.14922709281436633,0.12344109123417486,4.099495208722615,0.029900769526758804,0.0420464447715839,0.03257462159382469,0.006709793746929301,0.0013721642831677355,0.015772600183081294,3.9711187925088205,35.24797398358021,0.37517403925239917,0.020000000000000004,38.38649049030744
|
| 6 |
+
38160,625213440,2.156473845243454,3.9031655192375183,3.6245133876800537,0.13000501915424442,0.07766264832268159,1.3493400551378727,3.5167693297068277,14.073339495556377,68.42542598159179,48.91311484153412,1.1616201841256217,1.0824336028655297,5.45557303578821,33.844901889557896,24.708058433002787,0.5364368179230959,0.2219479693192218,0.2119572020479086,0.9583333333333334,0.9283854166666666,0.86837038397789,0.9979296334156468,0.4556722044944763,0.3385416666666667,0.13541666666666666,0.08333333333333333,0.973778188820385,nan,nan,0.9309025850670878,0.1786334246333657,0.15385356871411204,3.9633192647740527,0.027544688898728346,0.04076622417157582,0.03148083933931727,0.005553972199436573,0.001194302291613717,0.013178124670980791,3.843601092774288,32.274057078911326,0.384306798402056,0.019965729849129636,38.3678319490964
|
| 7 |
+
45792,750256128,1.739032303293546,3.9619769851366677,3.5509179532527924,0.1073041102172283,0.04710422688125012,0.6880650305732464,3.4570862849553428,9.929306628693965,73.88085659601514,44.61562159418162,1.1346129994749805,1.0519426254459485,3.0054889787057792,31.91585513868056,23.647669223028284,0.6302640084884353,0.21344287265594653,0.23034581542183008,0.96875,0.9674479166666666,0.9279124786456426,1.0,0.4743632674217224,0.4427083333333333,0.13020833333333334,0.10416666666666667,0.9783401364014329,nan,nan,0.9605498841944305,0.1887417692779915,0.1666962640786854,3.859801944116147,0.025495624249636056,0.03977047233431646,0.030624823024160284,0.00476201208921576,0.0010376556770469924,0.011395204979127493,3.746716131480277,29.872939267998223,0.3924493897409497,0.019863180383909898,38.59775174447041
|
| 8 |
+
53424,875298816,0.8355016248921553,3.9321539998054504,3.608916699886322,0.13723306730632126,0.06006618707882202,1.0742410665504092,3.4104101260503135,3.0766462136549624,70.60967959519876,49.36703221979553,1.1697424564659038,1.0658539605785216,4.895974313143509,30.46582063406289,22.95010705612858,0.7492301954223272,0.2327316229522286,0.25383028521481915,0.9635416666666666,0.9674479166666666,0.9186499019463857,1.0,0.5097888112068176,0.5833333333333334,0.140625,0.11979166666666667,0.9823641298377576,nan,nan,0.9465843345484851,0.1796781437957217,0.15481387530841553,3.5133187338699416,0.019363473858031856,0.036737381678157904,0.027970362849984904,0.0012685472722188526,0.0007640448747911232,0.0034997641211272734,3.4237151420320524,28.386917505249286,0.3980972625986835,0.019693132578928373,38.37008273302647
|
| 9 |
+
61056,1000341504,0.4652949906885624,3.9818787574768066,3.488739848136902,0.005148873866952878,0.040099296088252835,0.5387352397665381,3.374135454495748,1.7215226093698606,77.97145314487803,41.27746428466204,1.0052410365433875,1.0429269534735812,2.605310048309257,29.375864487809775,22.142826080720845,0.8187462133376409,0.23773129226103454,0.24578858154834912,0.9947916666666666,0.9739583333333334,0.9515373210112253,1.0,0.51549232006073,0.703125,0.140625,0.109375,0.9948518580640666,nan,nan,0.9723909718062108,0.17818573434785398,0.1530588444438763,3.403897257788584,0.015907610974292555,0.0355615691186022,0.026925305713752023,0.0009734868012417628,0.0005591462417318558,0.0028397502140833975,3.321130373372659,27.189933166189014,0.40330488839629935,0.01945688144839207,38.3799259329662
|
| 10 |
+
68688,1125384192,0.38198996894061565,3.987388551235199,3.532276769479116,0.005104507437560339,0.034553352331082955,0.5148949222154139,3.344119429588318,1.818977463990475,76.3745517767957,44.48122639053062,1.0052307942236471,1.0368523969058383,1.8004541447810636,28.527346708768444,22.149234239427972,0.8652051643827964,0.23513439117529478,0.2565078448272701,0.9947916666666666,0.9674479166666666,0.945949986577034,1.0,0.5222625732421875,0.765625,0.13020833333333334,0.11979166666666667,0.9948518580640666,nan,nan,0.9668024171987781,0.17838705948916336,0.15434739327368638,3.336966603231609,0.012921032661287852,0.03489449742877534,0.0263596542526878,0.0008478189259953313,0.0003690185762309919,0.002543544316324046,3.25903102203912,26.445916882913316,0.4064766829150028,0.019156226183831337,38.417211638774404
|
| 11 |
+
76320,1250426880,0.36115122369180125,3.8879385590553284,3.4090824723243713,0.0581849055050346,0.03124795562689542,0.6194361412660024,3.326671083768209,1.8855860353869776,69.39368891812784,37.79491291811738,1.0794199574869847,1.0328005501736521,2.543334417541385,28.027556162366494,20.25104270845725,0.88674706115515,0.25064654272203485,0.2805694189868799,0.9895833333333334,0.9739583333333334,0.9531717995802561,1.0,0.5482467412948608,0.796875,0.14583333333333334,0.14583333333333334,0.9916751772980206,nan,nan,0.9701030956493923,0.171983704688652,0.14646788502189642,3.2847960476433578,0.010261986736646446,0.03437518204146167,0.02592297393390466,0.0007737626024142286,0.00022810216094267915,0.0023648533128442506,3.2108691732088723,25.802455576648253,0.40935751705565165,0.018793456452196412,38.38284675136723
|
| 12 |
+
83952,1375469568,0.19841297886644801,4.059370418389638,3.688612083594004,0.035911277074319514,0.04388907199821309,0.5003351187333465,3.3026591539382935,1.29723020135565,88.95507859772482,56.16273345089342,1.0417257506431918,1.0475529282054303,1.8073768271141664,27.35622094636157,25.381131243185457,0.9180893660005977,0.23779312201895778,0.2789036751620459,0.9895833333333334,0.9739583333333334,0.9624499082565308,1.0,0.5402776002883911,0.8697916666666666,0.140625,0.13541666666666666,0.9896495462611105,nan,nan,0.973216681862444,0.1886570088873706,0.1614468593422013,3.2431976877434074,0.00792485617454384,0.03399709226520619,0.025567486855160092,0.0007340821138472844,0.00016983879561256095,0.002252033187754841,3.172552285006544,25.148884763445245,0.41213214137753473,0.018371334958694578,38.29188479232139
|
| 13 |
+
91584,1500512256,0.18113120699611804,4.033641556898753,3.769643872976303,0.07053172499475598,0.01383103743501124,0.5462768555128908,3.2794030904769897,1.231859830217537,86.59947339699386,61.304620835063965,1.0916267068998662,1.0142420880402105,2.220561928885864,26.719941155999745,25.740332277443006,0.9243277575050165,0.2557406104227102,0.2917411276951104,0.984375,0.9869791666666666,0.953107530872027,1.0,0.5600877404212952,0.8697916666666666,0.140625,0.15104166666666666,0.9869089581576796,nan,nan,0.9643080697229985,0.19056644738636427,0.16487022710498422,3.21084136599421,0.005994057370391498,0.033749268913090325,0.025308189030061785,0.0007139669787083506,0.00014810485031547572,0.0021884257333293876,3.1427393409493165,24.63173613914404,0.414504485712455,0.017893076407163026,38.43484184845603
|
| 14 |
+
99216,1625554944,0.1657558202277869,3.9548503756523132,3.6379715402921042,0.003991679762369434,0.020984095443054684,0.37589845180870424,3.2601024309794107,1.212334621324488,75.74095404401986,49.93522910524362,1.0040693109553327,1.0218232415925794,1.6202877026271143,26.227631101180606,22.39461844670623,0.9376361383138613,0.2497124589274415,0.2735175889530052,0.9947916666666666,0.9739583333333334,0.9638049056132635,1.0,0.5470151901245117,0.875,0.14583333333333334,0.13020833333333334,0.9948518580640666,nan,nan,0.9732772782833005,0.165168549205589,0.13930630303608874,3.1846341723504596,0.004805471694827792,0.033529794587838974,0.02510912353809074,0.0006981441186054292,0.00013699529200992928,0.002136791252514744,3.1182178398613787,24.25417280309605,0.4161894540897955,0.017362323018206012,38.4480459459239
|
| 15 |
+
106848,1750597632,0.07950723454511414,3.9434582591056824,3.6176519195238748,0.003948416825096501,0.019811880002635007,0.519875392077059,3.24501899878184,1.0863700539682706,75.8290629174757,49.38525956861673,1.004030736139738,1.02069855876141,2.4252963197451147,25.826976189826226,22.368242049219027,0.9665636917504012,0.24170001438493768,0.2897431704423042,0.9947916666666666,0.98046875,0.9741331885258356,1.0,0.5522684454917908,0.921875,0.140625,0.14583333333333334,0.9948518580640666,nan,nan,0.9759089351622,0.19185713959626582,0.16611336538335308,3.1631442009257236,0.004104211963162922,0.03334967313340203,0.024927262909034208,0.0006862662867317406,0.0001289803651562365,0.0021053205808943065,3.097842474289077,23.852075524050235,0.41816030208029215,0.016783116791537683,38.44458108762193
|
| 16 |
+
114480,1875640320,0.13076636076827222,4.091046671072642,3.638049562772115,0.02634491189932457,0.022559527194819868,0.2538086507920525,3.2268877029418945,1.166685407775527,86.72707743836627,46.72679254470868,1.0302443154876975,1.023454361569341,1.387166007112728,25.3765495248841,23.34828137141491,0.9723419107553157,0.25021814471076986,0.27119719671330345,0.9895833333333334,0.9739583333333334,0.9740625669558843,1.0,0.5478936433792114,0.9375,0.13541666666666666,0.13541666666666666,0.9900856389237257,nan,nan,0.9779961333600417,0.18473898015685108,0.16058376512955874,3.1430351631383977,0.003618822206658272,0.033180201016132185,0.02475811122107345,0.0006797717737773773,0.00012325568545772884,0.0020891165022980827,3.0785858735577536,23.417954864366997,0.4202787759918837,0.016159868723772935,38.41856856673551
|
| 17 |
+
122112,2000683008,0.0390078691028369,4.107647180557251,3.6853287617365518,0.0037419614327329023,0.01639954868460336,0.19155594132280385,3.215188185373942,1.040453720019584,95.78527257822786,54.346066282864854,1.0038177961101573,1.0168680705718043,1.3103622543811257,25.065284475373776,25.65258931107845,0.9757065687659741,0.26224910076670577,0.3072662044622993,0.9947916666666666,0.9739583333333334,0.9816986074050268,1.0,0.5764548182487488,0.9427083333333334,0.15625,0.16666666666666666,0.9948518580640666,nan,nan,0.9859517983762393,0.20213920390652143,0.1782297994553422,3.1252857108556835,0.0032734576130840193,0.03306924445568606,0.02460628692124164,0.000672307113139357,0.00011900797269689317,0.0020580396695142012,3.0614873548692376,23.13537924088522,0.42155193236795113,0.015497325216091833,38.44705670169299
|
| 18 |
+
129744,2125725696,0.04009622735126565,4.166816532611847,3.50462673107783,0.003098349846519947,0.01681006686294495,0.315268722821808,3.1973406871159873,1.0417840650602923,104.3307584059324,43.799655951161746,1.0031533203145038,1.017297905388738,1.539365115455669,24.631514446631275,25.337647029992095,0.9764770838047774,0.2450128097758001,0.2905238395274188,0.9947916666666666,0.9739583333333334,0.9752134333054224,1.0,0.5553370714187622,0.9427083333333334,0.13541666666666666,0.14583333333333334,0.9948518580640666,nan,nan,0.982129015144892,0.1639329988512183,0.13817823742283508,3.108894973826692,0.0030057155699972113,0.03293498267999191,0.024475918438449595,0.0006652011044513185,0.00011455653867969188,0.002034222898790673,3.0456643641297,22.761171002807856,0.42333529430365413,0.014800531927597655,38.36229425131412
|
| 19 |
+
137376,2250768384,0.05408386452472769,4.196555574735005,3.4949356019496918,0.11343978318586778,0.015721881212433193,0.34422096822102805,3.1839675505956015,1.0594044865409502,103.29609034380466,43.88121998447658,1.1598657096344287,1.016348845154538,1.8647983680969602,24.317821697881147,25.227935633655612,0.965673876495388,0.27086690333521707,0.3220898609400015,0.984375,0.9869791666666666,0.972039669752121,1.0,0.5887324810028076,0.9270833333333334,0.16145833333333334,0.17708333333333334,0.9875018786212119,nan,nan,0.9798949258790041,0.18733977461664608,0.1626412698533386,3.093586786685916,0.0027910130281520317,0.032812633692722304,0.02436224234185151,0.0006605123498049786,0.00011122157726585488,0.0020095061005842747,3.0308396448202273,22.449437631376135,0.42499828039115956,0.014074795349650518,38.46072229240746
|
| 20 |
+
145008,2375811072,0.02296752489443558,4.222238024075826,3.5508587857087455,0.003431489899223076,0.014733119823479987,0.19444038408209963,3.1749384005864463,1.023513813963299,112.4402314279589,45.255489245848,1.0034805410303058,1.0151378200515835,1.3284081995206387,24.104086786571983,26.59576397642068,0.9813667833438932,0.2679260874458773,0.3223289697017568,0.9947916666666666,0.98046875,0.9782090783119202,1.0,0.588175356388092,0.9583333333333334,0.15104166666666666,0.18229166666666666,0.9948518580640666,nan,nan,0.9830220975369836,0.1842695950397418,0.15927228820510209,3.079464948170569,0.0026086432186570386,0.0327021228777574,0.02423257051934055,0.0006550546819585752,0.00010770612802971109,0.0019929397926500333,3.017165898271868,22.2415063056306,0.42592021246563716,0.013325642393809753,38.48597223154026
|
| 21 |
+
152640,2500853760,0.029303575793164782,4.19922689596812,3.7086327274640403,0.004050027488299672,0.016377354171974428,0.17669068588308315,3.1581486463546753,1.0305026418005772,108.44431218686101,54.52349004625693,1.0041412424672045,1.017364798879836,1.2888962147831893,23.70230735545805,27.287287783786688,0.9902577130729151,0.27124356000167277,0.3074114586206897,0.9947916666666666,0.9869791666666666,0.9832458545764288,1.0,0.5840634703636169,0.96875,0.16145833333333334,0.16145833333333334,0.9948518580640666,nan,nan,0.9863744959293399,0.18893898698297562,0.1645651970175095,3.065960033543559,0.0024574422691009293,0.0326004189154059,0.02411098627322812,0.0006513481044054841,0.00010476456273963595,0.0019682010040855875,3.0040668589743094,21.96993580622993,0.42740079516103685,0.012558778301146429,38.43461589717691
|
| 22 |
+
160272,2625896448,0.034610174800036475,4.228796462217967,3.787488947312037,0.003602570878096382,0.008823412410786583,0.22004782068688655,3.1437391440073648,1.0358429216512557,112.9424032584467,63.342706470280525,1.0036738285623523,1.0092162086182173,1.3746460409393693,23.383922341651935,29.156058724307194,0.9863787291258092,0.27703945147409564,0.3099006906780056,0.9947916666666666,0.9934895833333334,0.9877046992381414,1.0,0.5894994139671326,0.96875,0.16666666666666666,0.16666666666666666,0.9948518580640666,nan,nan,0.9866816238160633,0.19710201206174816,0.17069139870970199,3.0533410854101346,0.002328316374820555,0.03249709814379173,0.023991594953759583,0.0006437451281339755,0.00010134794364757373,0.0019499400841976866,2.991829029239511,21.664731390196323,0.42908390869134627,0.011780043193471874,38.50733716204612
|
| 23 |
+
167904,2750939136,0.02492073293251451,4.309519052505493,3.78327148159345,0.004978576286570539,0.019534697496358906,0.17172750691285424,3.1279557148615518,1.0257018259798472,127.42694870946974,62.085626808312725,1.0050986388136882,1.020451276228082,1.2814285672785968,23.03098743715446,30.98232046617673,0.9864756174953432,0.27511940806126295,0.3131063921998837,0.9947916666666666,0.9869791666666666,0.9877046992381414,1.0,0.589564323425293,0.953125,0.15625,0.171875,0.9948518580640666,nan,nan,0.9892453002491189,0.18419876180505904,0.15894782704223567,3.040905425159728,0.0022117741322236996,0.0323733952320926,0.023862701076148622,0.0006414493434238775,9.788839129804798e-05,0.001933056430331003,2.9797851476638697,21.43697204867249,0.43009129799590046,0.010995367597369719,38.45076071515418
|
| 24 |
+
175536,2875981824,0.02333554418631441,4.448386748631795,3.8745095332463584,0.004672278116724253,0.011729117913195827,0.1571981366135636,3.1183509031931558,1.0242713309012692,150.79897501608923,68.65987524357168,1.0047965189928436,1.0123632819718436,1.2789195438589431,22.81931242416793,35.22835905136481,0.9903890843832177,0.2748039231078445,0.3183699542224414,0.9895833333333334,0.9934895833333334,0.9877046992381414,1.0,0.5922572016716003,0.9739583333333334,0.15625,0.16666666666666666,0.9897480490326416,nan,nan,0.9894057621616715,0.17791086829790478,0.15285320742987096,3.0300105304739056,0.0021115823895271235,0.03227534870137763,0.023742789155211375,0.0006354654051355375,9.482646967192482e-05,0.0019201885409317605,2.969230317206732,21.27495827121305,0.4308544416019627,0.01021072727974235,38.48690410896385
|
| 25 |
+
183168,3001024512,0.02240962167282608,4.418444752693176,3.8438433706760406,0.00516798267485304,0.01724546617660394,0.19859825273109286,3.109211007754008,1.0232451074024127,143.69670966921254,67.39312254122713,1.0053242402229674,1.0184730317927266,1.339634113969982,22.615419309580346,34.01313257334401,0.9920636382506235,0.2884646608550879,0.32930788019953533,0.9947916666666666,0.9934895833333334,0.9798430651426315,1.0,0.6063933968544006,0.9739583333333334,0.17708333333333334,0.18229166666666666,0.9948518580640666,nan,nan,0.9842684672912583,0.1940075447685293,0.1687573139788583,3.019449036956291,0.0020208562473945438,0.032155968048635825,0.023617660573889986,0.0006288921988590949,9.214243190108412e-05,0.0019047523206585657,2.959028752485536,21.019945200696682,0.43254460908659975,0.009432097738825898,38.44241034113231
|
| 26 |
+
190800,3126067200,0.02443321034418962,4.466665069262187,3.916031370560328,0.005779404409158436,0.014792478548296609,0.21268128321071345,3.097264210383097,1.0255487897644737,154.43303835500146,72.06869185948379,1.0059790053911148,1.0156346139802341,1.3671951889522314,22.35154989867621,36.18109110160707,0.991756028339264,0.29312736318413113,0.31485951631107195,0.9947916666666666,0.9869791666666666,0.9782090783119202,1.0,0.601554274559021,0.96875,0.171875,0.16666666666666666,0.9948518580640666,nan,nan,0.9830220975369836,0.19438666957461237,0.16972998977871612,3.0090650361262985,0.0019409280003270756,0.032055297955137806,0.023500985054341214,0.000624745669173642,8.990581353212739e-05,0.0018872375671855235,2.94896592343243,20.865345059638255,0.43334600079093966,0.008665408697250875,38.45793402380241
|
| 27 |
+
198432,3251109888,0.02199632947789117,4.5863208174705505,3.9512324035167694,0.004899997340177246,0.01297071980451392,0.2098858855732336,3.089693069458008,1.0229876227401584,180.4830686002181,77.85691586637707,1.0050420060144682,1.013495953203465,1.3752458628985893,22.185275347263754,40.706004465530796,0.990352444363361,0.28625018586648915,0.33899528470666,0.9947916666666666,0.9869791666666666,0.98607071240743,1.0,0.6094568371772766,0.9791666666666666,0.171875,0.19791666666666666,0.9948518580640666,nan,nan,0.9880791614511205,0.18848875934033535,0.16343494141862416,2.9992632320232735,0.0018691543737032663,0.031959816465443255,0.023398198577029346,0.0006172011359407802,8.788297880389014e-05,0.001877979166161748,2.939452987453223,20.677897837052054,0.43428439593071455,0.007916498943711495,38.46057430026853
|
| 28 |
+
206064,3376152576,0.020387174939969555,4.631117184956868,3.9281607965628305,0.004786709382486265,0.012397492076506941,0.15840912037941735,3.0822372436523438,1.0211975867138197,199.11852960144515,75.18786630182964,1.0049239642337822,1.0128757739333525,1.2860293094944437,22.023287762931904,42.95067290008315,0.9961024128663317,0.2941682440442656,0.3387980766288059,0.9947916666666666,0.9869791666666666,0.9909726728995641,1.0,0.6149804592132568,0.984375,0.17708333333333334,0.19791666666666666,0.9948518580640666,nan,nan,0.9918311903311405,0.18630453727316384,0.15949161921162158,2.9906366182182693,0.0018106091059102083,0.031867443239459524,0.023306777429186746,0.0006115709434650311,8.69068965268927e-05,0.0018582301961090713,2.9310950685649133,20.54141516230641,0.43500322026964006,0.007191071867150021,38.57467880388829
|
| 29 |
+
213696,3501195264,0.08389178791245892,4.710480411847432,3.972352623939514,0.004774616248568009,0.013352160066586313,0.2013108019344827,3.07560924688975,1.1174709836905583,227.9018886325313,80.41789313662072,1.0049027285051786,1.0140429381418186,1.345261444020994,21.883502707024576,47.812137510076454,0.9882116064276173,0.284358790934908,0.3442394700188298,0.9947916666666666,0.9869791666666666,0.98607071240743,1.0,0.6104941368103027,0.96875,0.171875,0.203125,0.9948518580640666,nan,nan,0.9880791614511205,0.16937157954023027,0.14413207340597486,2.982705651956222,0.0017605529733916668,0.031781225081986816,0.02323344892485895,0.000606977719123224,8.558285820638493e-05,0.00185241491226283,2.9233854368464423,20.422020668878496,0.43577756621802627,0.006494652022090374,38.45660931521546
|
| 30 |
+
221328,3626237952,0.020216211531078443,4.761237760384877,4.076508780320485,0.0045080715859917575,0.011297562539058484,0.28057335221546964,3.069929917653402,1.0209984591698413,240.26642735924136,88.68766043878765,1.004626571237057,1.0118674655722566,1.5517640523495737,21.7625046736635,50.75797843143162,0.9959088239229562,0.2864533465872911,0.34466760676226094,0.9947916666666666,0.9934895833333334,0.9828027387460073,1.0,0.612653374671936,0.9739583333333334,0.16666666666666666,0.203125,0.9948518580640666,nan,nan,0.9829295949360434,0.20039769889378553,0.1740544829905654,2.9751458188777904,0.001714687457895136,0.03169928592716615,0.023153939894527447,0.0006041485803342697,8.452212306938564e-05,0.00184492776480101,2.9160442941963343,20.327692623470814,0.436349854521469,0.005832543055900948,38.41342056550591
|
| 31 |
+
228960,3751280640,0.013320661061394881,4.835614800453186,4.134815752506256,0.004118788745652561,0.012628112849661477,0.209124833281957,3.06600288550059,1.013749271751413,258.9798112995414,92.77988749062064,1.0042174437125437,1.0131669853054386,1.3645077543704096,21.683061915634443,53.97691459441946,0.9993379700798769,0.3007206966749937,0.3416558267642405,0.9947916666666666,0.9869791666666666,0.98607071240743,1.0,0.6202658414840698,0.9895833333333334,0.18229166666666666,0.19791666666666666,0.9948518580640666,nan,nan,0.9880791614511205,0.18007406633715087,0.15465491876238957,2.968743417948318,0.001673056188743287,0.03164661075888302,0.023076431974347236,0.0006043964570382155,8.34857543754389e-05,0.001832988609452464,2.909826435703214,20.20224924088774,0.43704186655206123,0.005209787318395601,38.44815958890758
|
| 32 |
+
236592,3876323328,0.013938743969750552,4.9323716560999555,4.105215132236481,0.004588091001212528,0.011402493318862858,0.28567763728456913,3.0617531140645347,1.014489129753862,304.1017716100242,94.19228438373159,1.0047103254477232,1.0118463349226101,1.5703037908852517,21.59275015240125,60.64116510388093,0.9976288682160362,0.29254296335201935,0.345010962150813,0.9947916666666666,0.9934895833333334,0.9828027387460073,1.0,0.6168257594108582,0.984375,0.171875,0.203125,0.9948518580640666,nan,nan,0.9829295949360434,0.19805415715618815,0.17247343794830763,2.9628835748373463,0.0016451653225613796,0.031574312969456494,0.023016269335518043,0.0006033079691478946,8.249885545522622e-05,0.0018309671384632508,2.904131040705992,20.12915762746609,0.43753493246885966,0.004631127461369502,38.4488420931801
|
| 33 |
+
244224,4001366016,0.013005008495383663,4.962633868058522,4.186636994282405,0.004561978208395582,0.010470109793027168,0.21286921856299537,3.0589105685551963,1.0133904079136926,314.7988205785089,103.06488391507177,1.0046830465340377,1.0108532242272168,1.3767441133424079,21.536215993959473,63.40079875422249,0.9942841534784931,0.2990391606558584,0.3506761941048068,0.9947916666666666,0.9869791666666666,0.98607071240743,1.0,0.6231095194816589,0.9791666666666666,0.17708333333333334,0.20833333333333334,0.9948518580640666,nan,nan,0.9880791614511205,0.1953521667206625,0.1706587893422693,2.9577385968835226,0.0016195052204473414,0.031516408390098054,0.022958470317074646,0.0006064679581935237,8.19142923610774e-05,0.0018330371424349315,2.899122780218856,20.04216121427668,0.4381083448181237,0.004100970320511953,38.4216090342173
|
| 34 |
+
251856,4126408704,0.01334796689752693,5.034732063611348,4.166682451963425,0.004782726187016806,0.011050316193736384,0.2856496615001542,3.0578197638193765,1.0138039508981906,363.12215928378487,97.88635699456711,1.0049141226464924,1.011435532493107,1.568615948493959,21.508700286715097,69.58799801708555,0.9951273226692915,0.30072863796997223,0.34652208826729863,0.9947916666666666,0.9869791666666666,0.9828027387460073,1.0,0.6221172213554382,0.984375,0.17708333333333334,0.203125,0.9948518580640666,nan,nan,0.9829295949360434,0.18612356982394354,0.16045748034957796,2.9540397935396596,0.0016015589438738665,0.03147972446101946,0.022914859166552372,0.000605439226344322,8.126874659273357e-05,0.0018306171524373463,2.8955263122760027,20.08583610668872,0.43786254644284583,0.003623353354756449,38.46917120074212
|
| 35 |
+
259488,4251451392,0.012934938906255411,5.004019995530446,4.182938536008199,0.004530239328029022,0.009539278549536808,0.2132318426844222,3.055120348930359,1.01332691293012,362.9357139191566,100.12144159341454,1.0046454296584901,1.009803670952187,1.3758307477230742,21.45441573878622,69.84502543037446,0.9993379700798769,0.3030806114067908,0.3625390040900274,0.9947916666666666,0.9869791666666666,0.98607071240743,1.0,0.631142795085907,0.9895833333333334,0.1875,0.21875,0.9948518580640666,nan,nan,0.9880791614511205,0.20125867221147095,0.1766940791082258,2.950636213815854,0.0015849356120140412,0.03144832933964279,0.02288797381777377,0.0006038105493446306,8.070186879605674e-05,0.0018270606546681835,2.892203389603845,20.00416097451056,0.43823414452108445,0.003201913898652705,38.44866653591559
|
| 36 |
+
267120,4376494080,0.013740851936442292,5.019575834274292,4.287370473146439,0.004537401248214223,0.009976849042350485,0.17052972367071106,3.0532929499944053,1.0141927558803296,353.5105478261296,121.09300859685607,1.0046544009739091,1.0102553257084492,1.3211174025976509,21.417565089635858,71.48162019968312,0.9976275679925729,0.30266211447627517,0.35602994158495854,0.9947916666666666,0.9869791666666666,0.9909726728995641,1.0,0.628214955329895,0.984375,0.17708333333333334,0.21354166666666666,0.9948518580640666,nan,nan,0.9918311903311405,0.19438057966599823,0.1691769768949598,2.947857905783223,0.001573511928171754,0.03142458023903677,0.022851601388680625,0.0006039077025425639,8.06374013471203e-05,0.0018236852004652475,2.889499970121474,19.987151086205458,0.43839218975113126,0.0028398614619212603,38.481774171786256
|
| 37 |
+
274752,4501536768,0.013719044647890163,5.03670612970988,4.349505245685577,0.004552830662433394,0.009147180810671784,0.21430643918697,3.0523800452550254,1.0142299106137878,356.78338456199987,127.49580083815475,1.0046699022195626,1.009381986602479,1.37892170115129,21.398740675032357,72.86930422511058,0.9992919100954403,0.300119510916608,0.35459922989510545,0.9947916666666666,0.9869791666666666,0.98607071240743,1.0,0.6258183717727661,0.9895833333333334,0.17708333333333334,0.21354166666666666,0.9948518580640666,nan,nan,0.9880791614511205,0.19989332468073448,0.17558827976851413,2.9457438184348,0.0015677731908954197,0.03139198360569934,0.022825030095248524,0.0006004634185884083,8.055468899547465e-05,0.0018363438892393554,2.8874416577517157,19.95000495375827,0.4384598640424338,0.0025399532871463355,38.39925574037477
|
| 38 |
+
282384,4626579456,0.013870591164353149,5.063213745752971,4.266215781370799,0.004644187694729851,0.009764121340383033,0.17139958074625383,3.0511662562688193,1.014348819864356,388.9857314230783,111.73287875648172,1.004767850768541,1.0100274770078341,1.3238934628766257,21.37197286649565,75.20623152236759,0.9988711021231812,0.3011366820634615,0.3574233431796277,0.9947916666666666,0.9869791666666666,0.9909726728995641,1.0,0.627959132194519,0.984375,0.18229166666666666,0.21354166666666666,0.9948518580640666,nan,nan,0.9918311903311405,0.2016946876501038,0.17799890347911665,2.9440583291298896,0.0015566731296870978,0.03138988085667415,0.02282045357847608,0.0005992474207167734,8.070092985384508e-05,0.001836667577761889,2.8857746936802595,19.918155470546687,0.4388085930204054,0.002304473351749283,38.336429167717654
|
| 39 |
+
290016,4751622144,0.013963221502005277,5.084708472092946,4.279467473427455,0.004590670396159642,0.009557247819810044,0.16927550900364471,3.05106254418691,1.0144283145990516,387.5294821160501,116.52080295348867,1.0047119873870674,1.009833914909266,1.3162517280077692,21.371780178429734,75.68104159898166,0.9988711021231812,0.29752068943898524,0.35486626300665275,0.9947916666666666,0.9869791666666666,0.9909726728995641,1.0,0.6245682835578918,0.984375,0.171875,0.21354166666666666,0.9948518580640666,nan,nan,0.9918311903311405,0.20679844856137683,0.18183360213879496,2.9428138117287066,0.0015532862927812151,0.03138226997071473,0.02280068223919963,0.0005966730089303265,8.057312976397672e-05,0.0018294287400917627,2.8845708867356668,19.893550309182714,0.43895752189497406,0.0021352149741550937,38.371752988585776
|
| 40 |
+
297648,4876664832,0.012759880766376833,5.083545724550883,4.339352905750275,0.004405095265141767,0.010610666127925347,0.24090725803836222,3.0496081511179605,1.013159621246483,381.8998606792127,128.21368305484296,1.0045163695689545,1.0109591934897313,1.431722616821511,21.340602541529307,76.55921486810165,0.9992919100954403,0.2982632546092847,0.3610720021069618,0.9947916666666666,0.9869791666666666,0.9877046992381414,1.0,0.6275855302810669,0.9895833333333334,0.17708333333333334,0.21875,0.9948518580640666,nan,nan,0.9866816238160633,0.20171898467555138,0.17554924082166204,2.9416048627849185,0.001544360243241125,0.03136275469548063,0.02278768548627718,0.0005982580950571558,8.037932508156876e-05,0.001830074703023141,2.8834013391772952,19.934409130106932,0.43884363066818977,0.0020334671566161833,38.371228921341014
|
training_logs/run_metrics_text67/03_fone_xl/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: False
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 10
|
| 5 |
+
cache_base_path: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/cache
|
| 6 |
+
combine_strategy: sum
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty_sd
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/configs/config_FoNE_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_stem
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 1600
|
| 45 |
+
n_head: 25
|
| 46 |
+
n_layer: 48
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: fone
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: mse
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/configs/config_FoNE_text67.py --tqdm --verbose --seed 42 --run_name fone_gpt2_xl_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 48 --n_head 25 --n_embd 1600 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/trained/fone_gpt2_xl --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group fone_gpt2_xl_text67 --tie_word_embeddings --tokenizer_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_repo/tokenizers/num_text/bittoken_gpt2 --model rope_stem --num_embedding_type fone --base 10 --dropout 0 --cache_base_path /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/cache --compile --difficulty_column difficulty_sd --train_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Division_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard --train_set_ratios 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized_number_pos --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Division_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos pretokenized_number_pos --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Sat Oct 3 14:42:14 2026'}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: fone_gpt2_xl_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/trained/fone_gpt2_xl
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_repo/tokenizers/num_text/bittoken_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/cache/bittoken_gpt2_47200109_pos/96841076'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/cache/bittoken_gpt2_47200109_pos/63050008'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/cache/bittoken_gpt2_47200109_pos/44316829'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/cache/bittoken_gpt2_47200109_pos/48579485'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/cache/bittoken_gpt2_47200109_pos/63589654'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/cache/bittoken_gpt2_47200109_pos/28473695'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/cache/bittoken_gpt2_47200109_pos/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized_number_pos
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'pretokenized_number_pos']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-03/native/paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: fone_gpt2_xl_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/03_fone_xl/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,0.16580002009868622,7.946379992063157e-05,6.15333210589597e-05,0.00034197694913018495,0.7573548257350922,0.0005786893307231367,4.557001523440704,1.180379432763857,1.0000794673176352,1.0000615352210118,1.000342035540525,2.1327973341224715,1.0005788589099898,91.94459142547288,14.179832869906908,0.222184623036904,0.020727206476862527,0.01860187423153225,0.031250000931322575,0.1015625,0.03701801598072052,0.6034124725725053,0.03954314813017845,0.036458333333333336,0.0,0.005208333333333333,0.3796691782721003,nan,nan,0.3424742060266605,nan,nan,5.426339850891812,0.05108554059916823,0.05416660421904747,0.05123865638959146,0.026627483122408063,0.004391281578647836,0.0588387749392141,5.179991475503649,4881.847803541949,0.2553931348105552,0.004997126321997158,44.546575940620336
|
| 3 |
+
15264,250085376,0.051974190864712,0.00012654766760533676,9.305034109274857e-05,0.0005898983217775822,0.7245311439037323,0.00046740267134737223,3.928620999213308,1.0543614859570751,1.0001265556804029,1.0000930547672047,1.0005900753685597,2.0637778428970566,1.0004675150941682,50.1143078614458,8.176246341601468,0.4321641025833671,0.03868032233026565,0.032564095408853834,0.0729166641831398,0.0950520895421505,0.08008059300482273,0.906967509972339,0.07238855212926865,0.2708333333333333,0.005208333333333333,0.0,0.44565330931315356,nan,nan,0.4480751437222352,nan,nan,4.704674259796582,0.03457055774159082,0.04332151302757362,0.04102363480514685,0.019109062909110075,0.003990603412546177,0.04093543557507285,4.521723421723963,57.35550088814969,0.3578983692655367,0.009998084214664627,44.080429124237455
|
| 4 |
+
22896,375128064,0.04845516197383404,0.00013706366007681936,7.290902431122959e-05,0.00025491872656857595,0.7243456840515137,0.0008523414726369083,3.7412163889190806,1.049649042589202,1.0001370731506274,1.0000729116951708,1.0002549517179435,2.0634236097504774,1.0008527924542607,41.5897353420092,6.957732246195269,0.6184097907395183,0.049076395498175586,0.05026277928507382,0.140625,0.0885416604578495,0.11266066506505013,0.9776122052357404,0.09972471743822098,0.5520833333333334,0.005208333333333333,0.005208333333333333,0.5052862996672047,nan,nan,0.4787674690208708,nan,nan,4.343777188277595,0.02533434572435849,0.038506451956107746,0.03651077539274752,0.015490726257689364,0.003840862504224658,0.03198031585210912,4.192113684928975,39.55307264860321,0.3863509146328742,0.014999042107332095,44.08407377267236
|
| 5 |
+
30528,500170752,0.00043438882858026773,1.145603437180398e-05,1.1224689387745457e-05,8.145320680341683e-05,0.7135688066482544,0.00017352167196804658,3.6313989656434082,1.0004344832641683,1.000011456101547,1.0000112247528805,1.0000814565784575,2.0412671348295004,1.0001735368074822,37.34043029280172,6.340344226447965,0.7305834206596551,0.06766680977791875,0.06259110768814073,0.140625,0.108072929084301,0.1142277717590332,0.9902940889307081,0.12016727030277252,0.6666666666666666,0.020833333333333332,0.010416666666666666,0.5018923223057451,nan,nan,0.4816983042207236,nan,nan,4.131131614458823,0.020306760068717795,0.03590460423347432,0.034092227782882296,0.01355001589921435,0.003766770020101792,0.02715467057217438,3.9963565414275517,34.68358541765803,0.3981959994929204,0.020000000000000004,44.09644263177213
|
| 6 |
+
38160,625213440,0.0003306595390313305,1.0893212675000541e-05,1.2431009054125752e-05,2.5232611733372323e-06,0.7291081845760345,0.0003770545117731672,3.604563867634473,1.0003307210313188,1.0000108932754201,1.0000124310873368,1.0000025232645449,2.073308379201742,1.0003771671153812,36.16243451203699,6.176639518144676,0.5664860482966017,0.054327993425606666,0.0501370900226813,0.0885416641831398,0.0638020895421505,0.06235514022409916,0.989121286769338,0.08320154249668121,0.4010416666666667,0.015625,0.0,0.41026974634345,nan,nan,0.4037581701074184,nan,nan,3.9852238038488412,0.017103920540437044,0.03421176919865908,0.03253162764413067,0.012315222501546824,0.0037222187733521738,0.02409706104928003,3.8612419605473804,31.59891140142327,0.4072059510037522,0.019965729849129636,44.45875565846564
|
| 7 |
+
45792,750256128,7.732672020210885e-05,2.208201522080344e-06,4.3887016545340884e-06,3.4328507808822906e-05,0.7293423414230347,7.15856704118778e-05,3.466496523896543,1.0000773297699133,1.0000022082039601,1.0000043887113899,1.000034329338427,2.0737962334739244,1.0000715896210726,31.69963257896461,5.539088379726185,0.7800249518731416,0.08029717218214004,0.06885166836878984,0.265625,0.03645833022892475,0.17192108929157257,1.0,0.10562906414270401,0.765625,0.026041666666666668,0.005208333333333333,0.5788737556625468,nan,nan,0.5408923597230265,nan,nan,3.8745959538115144,0.014770172026425624,0.032947966220670206,0.03135055672978705,0.011366068468175165,0.0036920365592573653,0.021847287211489066,3.758621843816787,29.15985430617752,0.4149188398924389,0.019863180383909898,44.149709747769485
|
| 8 |
+
53424,875298816,0.001027127882480272,4.161809670222283e-07,5.079327394241773e-07,2.1534051484195516e-06,0.6960375607013702,3.320989276289765e-05,3.4173774202160225,1.0010281528553802,1.000000416181058,1.0000005079328693,1.0000021534080095,2.0058020318761924,1.0000332107768681,30.195674511193346,5.3146487120319605,0.7722827113097673,0.08777246035109687,0.07455450380089168,0.4375,0.2122395895421505,0.294919416308403,1.0,0.18672099709510803,0.7135416666666666,0.041666666666666664,0.005208333333333333,0.6767742017982528,nan,nan,0.6071763464715332,nan,nan,3.514625092808243,0.006739912198313324,0.028312373855854122,0.027009096811769582,0.00780178583387769,0.0035459590771072494,0.013554678798260578,3.4276612660008703,27.60309142736519,0.4206377460428004,0.019693132578928373,44.03060597676439
|
| 9 |
+
61056,1000341504,0.0004182407861037518,1.488125519699679e-06,1.4623817605752265e-06,3.703439801938657e-06,0.6249862909317017,8.656685963615018e-06,3.440812489056649,1.0004184154911573,1.00000148812663,1.0000014623828348,1.0000037034472535,1.8682535991249836,1.0000086567477213,30.813253650958433,5.383134425182717,0.7741713194901643,0.08425748751912554,0.07839590719105814,0.5885416567325592,0.23828125,0.436999574303627,1.0,0.19943007826805115,0.7291666666666666,0.026041666666666668,0.010416666666666666,0.7624648805552473,nan,nan,0.691428154706955,nan,nan,3.387161159850162,0.004186936690265879,0.027253440263987648,0.026046931503794873,0.006549366328081947,0.003481286114141046,0.010971497701550785,3.3086716828960214,26.294441224768978,0.4252039954980027,0.01945688144839207,44.17374731139785
|
| 10 |
+
68688,1125384192,7.28104878362501e-05,7.24002984497929e-06,1.4978148101363331e-05,2.6900337616098113e-05,0.5688187479972839,2.7815805879072286e-05,3.3519006899635615,1.0000728139038684,1.000007240056494,1.0000149782759111,1.0000269007021225,1.7662060946091942,1.0000278162148537,28.29598276178162,5.008905515077723,0.8056148102885818,0.09380178522190682,0.08519610562000007,0.6875,0.407552108168602,0.5703750550746918,1.0,0.23101438581943512,0.796875,0.041666666666666664,0.010416666666666666,0.7931361118292747,nan,nan,0.7581595433875918,nan,nan,3.313658480317046,0.0034761175058487114,0.02678543121529877,0.025625460086081887,0.0057353891598773046,0.0033804852946006606,0.009712267071016412,3.238943310671862,25.318473279288728,0.4291105925279281,0.019156226183831337,44.214758378052316
|
| 11 |
+
76320,1250426880,0.09101964021374442,2.70448788342037e-06,3.5616598097476526e-06,7.103092825033741e-08,0.48949332535266876,1.7431877950002672e-05,3.3288266837092424,1.0998297413056983,1.000002704491564,1.0000035616661638,1.0000000710309327,1.6318386078734268,1.0000174320513286,27.659010262330256,4.912957482964195,0.7250324062143909,0.09770027326982259,0.08034049794931246,0.65625,0.524739570915699,0.5704540610313416,1.0,0.23144963383674622,0.6354166666666666,0.052083333333333336,0.010416666666666666,0.7861660635098815,nan,nan,0.7549204889995357,nan,nan,3.258060985380832,0.0030236202599910698,0.026422168222154478,0.025276956691228994,0.004928012848446102,0.003234489502123182,0.008565828696550697,3.186609890980999,24.785661732875575,0.43173074123275856,0.018793456452196412,44.17503928198893
|
| 12 |
+
83952,1375469568,2.4716276129765902e-05,5.524637344933581e-06,5.965063110124902e-06,1.4839916730124969e-05,0.407532200217247,2.038123147940496e-05,3.302510218714209,1.0000247165843024,1.0000055246527102,1.0000059650809874,1.0000148400282147,1.50310625997956,1.0000203814397264,26.955511723382802,4.779812773021186,0.831597482432284,0.09614557523661774,0.10149968066773211,0.8177083432674408,0.622395858168602,0.6887905597686768,1.0,0.2618086040019989,0.8541666666666666,0.036458333333333336,0.020833333333333332,0.8598376843341006,nan,nan,0.8047998523494849,nan,nan,3.21206748337699,0.002691510219490707,0.026144932217474253,0.025007854213685596,0.0041068075570505,0.0030245109274432244,0.007429132987184587,3.143662717380801,23.958392497815925,0.4350793474633051,0.018371334958694578,44.06510264747515
|
| 13 |
+
91584,1500512256,7.164921385083289e-06,1.7838170833783806e-06,2.925043304458086e-06,6.417349140974693e-07,0.37837444245815277,1.9262002638242848e-05,3.2727015852966965,1.0000071649632973,1.000001783818678,1.0000029250475966,1.0000006417351672,1.4600160314572392,1.0000192623588178,26.211802336709578,4.667407163727197,0.8286608248549983,0.08773093008730015,0.09248158036157214,0.8125,0.635416641831398,0.7110593318939209,1.0,0.2445063441991806,0.828125,0.036458333333333336,0.010416666666666666,0.8546125681217139,nan,nan,0.8263065057496229,nan,nan,3.1768594030173696,0.00251444378290255,0.025988272227171823,0.024866231269405535,0.0033545866835059082,0.0027647765505242603,0.006410590168495233,3.1109604850453065,23.58748553858863,0.43671302529016753,0.017893076407163026,44.108026617917936
|
| 14 |
+
99216,1625554944,0.0012716909986920655,3.991083872278978e-06,8.80021252669394e-06,8.180796839951654e-07,0.33471299707889557,2.230257109658851e-06,3.2556494458888965,1.0012726667738565,1.0000039910919272,1.0000088002652485,1.0000008180800237,1.397653296884103,1.000002230259597,25.77617318578832,4.5964449984490106,0.8368232097614341,0.09509423659228289,0.09042516751614829,0.828125,0.7265625,0.7628660798072815,1.0,0.25303199887275696,0.8229166666666666,0.026041666666666668,0.005208333333333333,0.8594661570775012,nan,nan,0.843494312139228,nan,nan,3.146687350831379,0.002307248790660964,0.025848988845168626,0.024736719734684953,0.002784894843969203,0.0024718006348495203,0.005551263095674663,3.08298641797819,23.007941938733655,0.43912392393519917,0.017362323018206012,44.165364891511295
|
| 15 |
+
106848,1750597632,3.60445865226211e-05,3.574166271391732e-06,4.704763341578655e-06,5.530799171538092e-05,0.2650633975863457,3.411659326957306e-05,3.2347398272055825,1.0000360452408328,1.000003574172663,1.0000047047750975,1.000055309738549,1.3041558570812013,1.00003411723137,25.25103610896905,4.5079036738869664,0.8549959710345647,0.10671239901743049,0.09589473776459574,0.8958333432674408,0.739583358168602,0.7896254658699036,1.0,0.27233651280403137,0.8645833333333334,0.057291666666666664,0.010416666666666666,0.9018310443498194,nan,nan,0.8579602944276606,nan,nan,3.121221450798521,0.0021104042812179046,0.02575893599213284,0.02464272496244948,0.0023807074885782253,0.002200421298075246,0.004911991355703029,3.0592162479066625,22.539912386510355,0.4411963644892415,0.016783116791537683,44.10819456745947
|
| 16 |
+
114480,1875640320,3.868111548399611e-06,1.922108424423641e-06,1.8651833784133487e-06,1.355575236061668e-06,0.23619358986616135,2.78499584283054e-07,3.2164721016112403,1.000003868120137,1.000001922110338,1.000001865185126,1.0000013555767748,1.2665814641708133,1.0000002784996236,24.812510123987646,4.4398715539500655,0.8520161751486045,0.09692805464915898,0.10502860140372908,0.9375,0.837239570915699,0.8222055435180664,1.0,0.2746267318725586,0.875,0.03125,0.015625,0.9201029961307844,nan,nan,0.8681354899114618,nan,nan,3.100391694470188,0.0020879376472337317,0.025699823888381233,0.02457307535028417,0.0021473842684741305,0.0019471026366452045,0.004535695505846488,3.0394006585253233,22.339908856078512,0.44302017558014617,0.016159868723772935,44.12505082765705
|
| 17 |
+
122112,2000683008,1.5343158338509966e-05,5.918869874221855e-06,5.083671339889406e-06,3.3296023502771277e-06,0.0505067678168416,4.209189228276955e-06,3.195837769017089,1.0000153432766754,1.0000059188880357,1.0000050836842655,1.0000033296082607,1.052391690035019,1.0000042091980885,24.326368776421162,4.339827764444501,0.8778375739861562,0.10563553148356263,0.0901172988541272,0.9375,0.967447891831398,0.856286883354187,1.0,0.2695368826389313,0.9322916666666666,0.041666666666666664,0.005208333333333333,0.9142031329489934,nan,nan,0.8856787255499512,nan,nan,3.0784932828783615,0.0018821113188567796,0.025573351596590133,0.024464816681241487,0.001876783243623863,0.0016034794447789972,0.004054325825063039,3.019038398016653,21.715213349905152,0.4452323956606398,0.015497325216091833,44.104935683967035
|
| 18 |
+
129744,2125725696,2.162274995498592e-05,3.6971448480471736e-06,3.765127530641621e-06,1.534600323793711e-05,0.031850475585088134,2.852041598089272e-05,3.1814990791414552,1.0000216230268926,1.000003697151885,1.0000037651348257,1.0000153461308274,1.0327427414312653,1.0000285208316386,23.97140510711218,4.28631725725993,0.8595878516977522,0.1060122505359738,0.10296147568252473,0.9114583432674408,0.973958320915699,0.8191898167133331,1.0,0.2839388847351074,0.8645833333333334,0.052083333333333336,0.020833333333333332,0.9125854063313454,nan,nan,0.8751640738143275,nan,nan,3.0593664818954247,0.0016850703292026232,0.025454161798999307,0.02434874679182235,0.0016555272386241594,0.0012626345464424045,0.0036483913296012586,3.0013119331987315,21.353168225138443,0.44705174177635143,0.014800531927597655,44.074728878587635
|
| 19 |
+
137376,2250768384,0.00012373154822853394,6.2247693222161615e-06,6.387030452970066e-06,3.97955682274187e-05,0.01270743040367961,2.910774765041424e-05,3.1581140251946636,1.000123739334315,1.0000062247892194,1.0000063870512321,1.0000397963625125,1.0127968836968282,1.0000291081717725,23.458163528840547,4.210166524035204,0.8826737621070545,0.1169033532207105,0.10605391333574862,0.96875,0.986979141831398,0.888972282409668,1.0,0.3006032705307007,0.9583333333333334,0.057291666666666664,0.020833333333333332,0.9325096385243038,nan,nan,0.9015647474831591,nan,nan,3.0400237414613627,0.0015193282009732295,0.025334944750867644,0.024234859443759086,0.0014706014399713935,0.0009631169431763448,0.003302972824979226,2.9831979008460485,20.997812440247905,0.44915105245491993,0.014074795349650518,44.18392846584054
|
| 20 |
+
145008,2375811072,3.7513433198910207e-05,7.629986384927179e-06,5.889006388315465e-06,0.0001370154022879433,0.023681042715907097,3.5534599192033056e-05,3.1456546438857913,1.0000375141380093,1.0000076300160348,1.0000058890237287,1.0001370296005772,1.024099204484837,1.0000355353191162,23.161839940673904,4.169451820465172,0.8976005911864935,0.11727401953061918,0.11630229173488621,0.9479166567325592,0.973958320915699,0.9022729396820068,1.0,0.3120904564857483,0.9635416666666666,0.0625,0.036458333333333336,0.9240461216929058,nan,nan,0.9031811592479547,nan,nan,3.0226205905533647,0.0014073545621966187,0.02522636726646665,0.024134589159362677,0.0013123316323416126,0.0006954856119723428,0.003017980193898528,2.96682646516307,20.7054289592512,0.4502969536914703,0.013325642393809753,44.09730580135173
|
| 21 |
+
152640,2500853760,2.9015447580604814e-05,1.1226191872992786e-05,6.948080226720776e-06,2.769588172668591e-05,0.019213504623621702,2.064371255983133e-05,3.1337705488549545,1.0000290158900635,1.0000112262558547,1.0000069481044145,1.0000276962655419,1.0194077604848415,1.0000206439275856,22.898241809904775,4.1311064429761535,0.8843423029136415,0.11008585400944236,0.11285221450867705,0.953125,0.986979141831398,0.9080410003662109,1.0,0.3012126684188843,0.9375,0.046875,0.020833333333333332,0.9214545737486333,nan,nan,0.9050988301169127,nan,nan,3.0058866370544823,0.001292430766981247,0.025080699067914908,0.0239906555353576,0.0011689938356335625,0.0004541516181486807,0.002729167623918782,2.9511705225452016,20.390637177528824,0.4521600691761736,0.012558778301146429,44.13578281365714
|
| 22 |
+
160272,2625896448,1.1742657306967885e-05,4.8849422000785125e-06,4.163791572864284e-06,3.303809307908523e-06,0.004227367928251624,3.20078652293887e-06,3.1244669642183,1.0000117427607664,1.0000048849543104,1.000004163800243,1.0000033038152925,1.004241243532787,1.0000032007916912,22.707859776570686,4.101732616603682,0.8853964537294774,0.12701619947964524,0.10952266495197266,0.984375,0.993489570915699,0.9275047481060028,1.0,0.31483927369117737,0.9479166666666666,0.057291666666666664,0.020833333333333332,0.9384853090159595,nan,nan,0.9138747130831083,nan,nan,2.9888258515788824,0.001014961552926234,0.02489337396027311,0.02382164231459542,0.0009575762347673528,0.0002370124915124385,0.002313934702853936,2.935587333826378,20.133450788959266,0.4534324368606448,0.011780043193471874,44.137524829683095
|
| 23 |
+
167904,2750939136,3.987611307820771e-05,1.3259099887363845e-05,8.206990059989039e-06,9.954436609405093e-05,0.0068986546248197556,2.9918062864453532e-05,3.1091704427575073,1.0000398769082988,1.0000132591891329,1.000008207023912,1.0000995523215739,1.006929271905014,1.0000299185182056,22.364313362438384,4.0530619211863605,0.9060347177439392,0.12286225442937865,0.11613984248066186,0.9895833134651184,0.993489570915699,0.9378160536289215,1.0,0.31910136342048645,0.9895833333333334,0.057291666666666664,0.020833333333333332,0.9416681326304873,nan,nan,0.9213843627367169,nan,nan,2.9740884381181747,0.0009257561943612781,0.024780756705242078,0.023706600616769352,0.0008492082617832257,0.0001528764969117073,0.0021053344962059855,2.9215678890931445,19.86610287809022,0.45507079718997645,0.010995367597369719,44.10017979888924
|
| 24 |
+
175536,2875981824,5.28467044205172e-05,1.360347732770606e-05,1.2831240837840596e-05,3.2506402931176126e-05,0.005597412528004497,2.769849743344821e-05,3.099883579222175,1.0000528481443611,1.0000136035724596,1.0000128313231897,1.0000325069312712,1.0056204284886983,1.0000276988819863,22.157464109858108,4.023317718171439,0.9028447596530984,0.11474841827005697,0.11654842743797789,0.9791666567325592,0.993489570915699,0.9348398447036743,1.0,0.31111136078834534,0.9791666666666666,0.036458333333333336,0.036458333333333336,0.9362920842443904,nan,nan,0.9170495904205987,nan,nan,2.9601759050694856,0.0008550384296594257,0.02466565394062408,0.023594881935324323,0.0007531001676739799,0.00012846477291792054,0.0019190688533429416,2.9082596813316197,19.617259644112497,0.4565466379500785,0.01021072727974235,44.218617972308444
|
| 25 |
+
183168,3001024512,2.476807731000008e-05,1.217367662320612e-05,1.0973384178214474e-05,2.0131869860051665e-05,0.001523108898254577,2.875165500881849e-05,3.0847704258436957,1.0000247683842902,1.0000121737527583,1.000010973445073,1.0000201320725712,1.0015251875512172,1.0000287521226783,21.847187808172794,3.9784013993573404,0.9094459024203981,0.11911780365580008,0.11931236501393938,0.9791666865348816,1.0,0.9540402591228485,1.0,0.31897297501564026,0.9947916666666666,0.046875,0.046875,0.9383470091658334,nan,nan,0.9229532665728281,nan,nan,2.94756915692857,0.0007905349383450039,0.02455465320035995,0.02347793711496128,0.000669269452722481,0.0001117814253538678,0.0017445136013403242,2.896220451757145,19.448743566776972,0.4573771968382397,0.009432097738825898,44.251047271947286
|
| 26 |
+
190800,3126067200,2.7138918994751293e-05,1.966720628843177e-05,1.9679234355862718e-05,2.4814973585307598e-05,0.011083653604146093,2.0925178432662506e-05,3.0781139653214877,1.000027139288695,1.0000196674010295,1.0000196794317966,1.0000248152839981,1.0111951999437965,1.0000209254034074,21.705899416996374,3.959600977678442,0.8873054985024371,0.13260089239923642,0.121831274656126,0.9895833432674408,0.993489570915699,0.9688421785831451,1.0,0.33451613783836365,0.9583333333333334,0.08333333333333333,0.026041666666666668,0.9421869657623271,nan,nan,0.9299845530185848,nan,nan,2.9358664937968193,0.000728927300424898,0.02444480143929685,0.023364883871614978,0.0005991702281574455,9.973870007357765e-05,0.001584950940774233,2.8850440058055398,19.285296348843186,0.4584302825372924,0.008665408697250875,44.282589619768764
|
| 27 |
+
198432,3251109888,2.9435962460411247e-05,2.4335969101230148e-05,1.944539963005809e-05,2.858107563952217e-05,9.943759141606279e-05,2.444618075969629e-05,3.0673441960825585,1.0000294363958933,1.0000243362892147,1.0000194455905755,1.0000285814848244,1.0000994428881143,1.000024446479571,21.471915628436605,3.924591616794971,0.9049459413495677,0.13016154228302745,0.12464068003513648,0.9895833134651184,1.0,0.9570691287517548,1.0,0.3354700207710266,0.984375,0.078125,0.03125,0.9415689818561077,nan,nan,0.924920812016353,nan,nan,2.9253295096643606,0.0006827596558971165,0.024335866848252202,0.023257751942541944,0.0005338006939414881,9.135332022160991e-05,0.0014492251642303268,2.8749787356772156,19.118614796447552,0.4594915536363107,0.007916498943711495,44.21831349941338
|
| 28 |
+
206064,3376152576,6.474851761595346e-05,2.391999623796437e-05,2.02924029508722e-05,4.517654451774433e-05,0.004349726579675917,3.6485896998783574e-05,3.062527922874627,1.0000647507095186,1.0000239202891956,1.000020292615012,1.000045177565374,1.004367849758541,1.0000364865677134,21.364820360315022,3.9099112625457684,0.8998325548625756,0.13874566251344328,0.13218961574542398,0.984375,0.993489570915699,0.9540797472000122,1.0,0.350803405046463,0.9791666666666666,0.078125,0.041666666666666664,0.938893002923578,nan,nan,0.926712874090299,nan,nan,2.9150060662136763,0.0006425645237407809,0.024238334418825055,0.02314790749473655,0.00047883729484647663,8.449764174838524e-05,0.0013363090229011785,2.8650776003009057,18.934298983160055,0.460687386125826,0.007191071867150021,44.20055267142034
|
| 29 |
+
213696,3501195264,3.8494752516271546e-05,2.2614115550823044e-05,1.7278213817917276e-05,2.9581444323412143e-05,0.0005872264737263322,2.792735722323414e-05,3.0549534335150383,1.000038495536692,1.0000226143764568,1.000017278365307,1.0000295818819467,1.0005875444423074,1.0000279277475421,21.207639849602113,3.8869090417074807,0.9047674649355762,0.13348380528295886,0.13146583977448761,0.9947916567325592,1.0,0.9674067795276642,1.0,0.34581106901168823,0.9895833333333334,0.0625,0.03125,0.9447691671860715,nan,nan,0.9310679105886569,nan,nan,2.9060287083818688,0.0006105727977579993,0.024141691955734512,0.02305081995837989,0.00043532180795837394,7.944532213402865e-05,0.0012302875229852808,2.8564805530941024,18.817159508509206,0.46120064584167375,0.006494652022090374,44.133686760134914
|
| 30 |
+
221328,3626237952,2.1806060431117658e-05,2.4570378627686296e-05,1.9155151676386595e-05,2.387389849900501e-05,0.0023722579335299088,2.128680807800265e-05,3.0494618608111828,1.0000218063032453,1.0000245706906674,1.000019155337417,1.0000238741841452,1.0023778649797872,1.0000212870346676,21.102752404998657,3.8721772805040837,0.9079449852252741,0.1340282295534766,0.13140684639542588,0.9947916567325592,0.993489570915699,0.9673672318458557,1.0,0.3462264835834503,0.984375,0.07291666666666667,0.036458333333333336,0.9462707960046828,nan,nan,0.9335704659267018,nan,nan,2.898241830446935,0.0005825967102573536,0.024052602558180033,0.022950907635397085,0.00040088968634015873,7.527242805842786e-05,0.001143089741593753,2.849036455720632,18.69405167763838,0.4619750980584564,0.005832543055900948,44.127637705153326
|
| 31 |
+
228960,3751280640,2.9804230507579632e-05,3.065178498218302e-05,2.4147823751263786e-05,3.290911081421655e-05,0.0008667071415402461,2.3610754396941047e-05,3.044365523441229,1.0000298046779168,1.0000306522651217,1.0000241481199739,1.0000329096523313,1.0008673777744415,1.0000236110335055,20.99955607567439,3.8572235113139546,0.9095295399182907,0.13903115733376026,0.1365193069188028,0.9895833134651184,1.0,0.9673540890216827,1.0,0.3560285270214081,0.9947916666666666,0.07291666666666667,0.036458333333333336,0.9414088957322141,nan,nan,0.9304279529023916,nan,nan,2.891309268309219,0.0005575031393620291,0.02395895179029187,0.02285437388941099,0.00037081735773813833,7.241490362795527e-05,0.0010691906791742848,2.842426000994706,18.639007745986216,0.4623435762607389,0.005209787318395601,44.13455663747627
|
| 32 |
+
236592,3876323328,2.8613627364393324e-05,3.732710138137918e-05,2.413437960058218e-05,3.0533631615980994e-05,0.0007255885284394026,2.5105900022026617e-05,3.040764943366715,1.0000286140409356,1.0000373277992645,1.0000241346740628,1.000030534098129,1.0007260807489358,1.0000251062158023,20.927827575838386,3.8469570533450734,0.9095485590600977,0.1326922114395762,0.14241286593353672,0.9895833134651184,1.0,0.9777444005012512,1.0,0.355495810508728,0.9947916666666666,0.0625,0.036458333333333336,0.9444384768915673,nan,nan,0.9338968420245995,nan,nan,2.885010580463917,0.0005379294456167882,0.023867342133542937,0.022756471868101986,0.00034536330237603286,7.027901928102835e-05,0.0010061103967936818,2.836427069281487,18.559374298814458,0.4626757457883603,0.004631127461369502,44.068340875713695
|
| 33 |
+
244224,4001366016,3.028363335033646e-05,3.775333607336506e-05,2.728427807596745e-05,3.1895728170638904e-05,0.007903950840045582,2.542473066569073e-05,3.0380666220444255,1.0000302840939483,1.0000377540641958,1.000027284656412,1.0000318962388546,1.0079666058398702,1.0000254250539338,20.874579877218114,3.8403855895950474,0.9095295399182907,0.1343374106256447,0.1449116606034516,0.9947916567325592,0.993489570915699,0.973280131816864,1.0,0.359279990196228,0.9947916666666666,0.06770833333333333,0.046875,0.9450967748028537,nan,nan,0.9344784933297584,nan,nan,2.8797434432201667,0.0005205563353606607,0.023792590771458136,0.022675408498832473,0.00032908806023093667,6.855420826681396e-05,0.0009588188374105102,2.831398412176544,18.507422789775102,0.46315199767768506,0.004100970320511953,44.099141531559226
|
| 34 |
+
251856,4126408704,3.188089613104239e-05,3.9277461837627925e-05,3.0894264455127995e-05,5.841021265950985e-05,0.002669337631232338,2.3569564291392453e-05,3.0354446874504597,1.0000318814046811,1.000039278244964,1.000030894753765,1.0000584121750025,1.0026764308545193,1.0000235698423503,20.81710325172437,3.831423388428522,0.9069170746356736,0.14270945126221343,0.12877900394604808,0.9947916567325592,0.993489570915699,0.9806942641735077,1.0,0.3516422510147095,0.984375,0.078125,0.026041666666666668,0.9462692493883272,nan,nan,0.935734711199378,nan,nan,2.8755675258334907,0.0005081108525398453,0.02371453304054602,0.022601267073796786,0.0003149692818941782,6.710644794941872e-05,0.0009204229523166079,2.827441101746697,18.449483181320645,0.4635521407420718,0.003623353354756449,44.15739475872926
|
| 35 |
+
259488,4251451392,3.083771844103467e-05,3.938820918847341e-05,3.034251494682394e-05,3.0393448469112627e-05,0.0031046466488078295,2.4403775569226127e-05,3.0349251624817652,1.0000308381952359,1.0000393889890828,1.0000303429810105,1.0000303939111468,1.0031142625159668,1.0000244040736566,20.812228470708092,3.8307854430534554,0.9084040194697578,0.13524280535987232,0.1566237384227139,0.9895833134651184,0.993489570915699,0.9747550785541534,1.0,0.3699270784854889,0.9895833333333334,0.0625,0.046875,0.9444623147137463,nan,nan,0.9340599271624038,nan,nan,2.8718717703680365,0.000495383146821833,0.0236440930651549,0.02252267429310305,0.0003049705109990302,6.605608305333688e-05,0.0008911516686558494,2.823947426790774,18.40235769545632,0.46392416132446107,0.003201913898652705,44.14332325309327
|
| 36 |
+
267120,4376494080,2.6743021408037748e-05,3.842059777525719e-05,3.052664396818727e-05,3.4610913644428365e-05,0.0071861066667224804,2.1984945306030568e-05,3.0345659279652564,1.0000267433802073,1.0000384213371465,1.0000305271161394,1.000034611516472,1.0072379276719667,1.0000219851870675,20.804562766892705,3.8302789975859577,0.9084230386115647,0.1467868996554503,0.148510008814637,0.9947916567325592,0.993489570915699,0.9732933044433594,1.0,0.37460824847221375,0.9895833333333334,0.08333333333333333,0.041666666666666664,0.9447676205697159,nan,nan,0.9344684046227485,nan,nan,2.8684195542901985,0.0004864944234579994,0.023580288916375883,0.022456342751660745,0.00029528008483166175,6.538619355634719e-05,0.0008689386725959286,2.8206668082106625,18.335847829997665,0.4643142259712926,0.0028398614619212603,44.219463228465145
|
| 37 |
+
274752,4501536768,3.12922975354013e-05,4.212287240079604e-05,3.2496291169081815e-05,3.6342958992463537e-05,0.004065119503593451,2.463792588969227e-05,3.0318835949486433,1.000031292790002,1.0000421237631978,1.0000324968264285,1.000036343619504,1.004081648209206,1.0000246382294062,20.750811199314395,3.822151391821734,0.9084040194697578,0.14441544178511645,0.14387278225722255,0.9895833134651184,0.993489570915699,0.9688421785831451,1.0,0.36793452501296997,0.9895833333333334,0.07291666666666667,0.036458333333333336,0.9447407000698149,nan,nan,0.9325597446877509,nan,nan,2.865770123789645,0.00047849396401208807,0.023530974258261308,0.02239460932329404,0.0002901400297443086,6.464180739582936e-05,0.0008543577890623023,2.818156891458835,18.343950929319334,0.4642646688835176,0.0025399532871463355,44.14017522499279
|
| 38 |
+
282384,4626579456,3.1371673685498536e-05,4.742741475638468e-05,3.545320032571908e-05,3.73648181266617e-05,0.003685861960548209,2.355280412302818e-05,3.0311826282607703,1.0000313721670526,1.0000474285406766,1.0000354538384144,1.000037365518833,1.0036994508015555,1.0000235530815114,20.735524391887555,3.819914145119371,0.9084040194697578,0.1440224189217434,0.14438560079350882,0.9895833134651184,0.993489570915699,0.9792193472385406,1.0,0.36825835704803467,0.9895833333333334,0.07291666666666667,0.036458333333333336,0.9447407000698149,nan,nan,0.9351683112327009,nan,nan,2.863709848358509,0.0004732985210658664,0.023488186533526394,0.02235356684715424,0.00028668904830037636,6.424466663430013e-05,0.00084512746164178,2.816198719502703,18.34299678175829,0.46427924960726713,0.002304473351749283,44.135807579750804
|
| 39 |
+
290016,4751622144,3.716807987075299e-05,5.014438102080021e-05,4.0275192077388056e-05,4.135750714340247e-05,0.0002929397733169026,2.8147111152065918e-05,3.0307041241127686,1.0000371687747398,1.00005014563831,1.0000402760183769,1.0000413583633128,1.0002930233225074,1.0000281475073431,20.724834235252487,3.817903479268154,0.9095295399182907,0.14156330452697444,0.1502653142562962,0.9895833134651184,1.0,0.9762563109397888,1.0,0.3712899386882782,0.9947916666666666,0.078125,0.046875,0.9447407000698149,nan,nan,0.9347790761385113,nan,nan,2.8623686397537997,0.0004686607635522126,0.023447671036153337,0.022313427848158603,0.00028153089783541235,6.416782660629318e-05,0.0008371893676006889,2.8149559763575835,18.367864149921346,0.464126520659345,0.0021352149741550937,44.16967658852889
|
| 40 |
+
297648,4876664832,3.0824178793409374e-05,4.705943320004735e-05,3.792851202888414e-05,3.912511056114454e-05,0.005133948777256592,2.352790852455655e-05,3.0302183935418725,1.000030824654099,1.000047060540984,1.0000379292454675,1.0000391258804457,1.0051603584132258,1.000023528185309,20.714400203032245,3.8171055757073966,0.9084040194697578,0.13888728834532826,0.14946177234456887,0.9895833134651184,0.993489570915699,0.9747550785541534,1.0,0.3677520155906677,0.9895833333333334,0.0625,0.041666666666666664,0.9444369302752117,nan,nan,0.9349033372321477,nan,nan,2.861333656501595,0.0004656214312901403,0.02341563154474372,0.0222812944842658,0.0002784609754617046,6.412656547278177e-05,0.0008288618482517871,2.8139996446596207,18.32943474802831,0.46448981937263123,0.0020334671566161833,44.18597134728992
|
training_logs/run_metrics_text67/04_bittoken_large/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: True
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 2
|
| 5 |
+
cache_base_path: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/cache
|
| 6 |
+
combine_strategy: zero_pad
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/configs/config_bittoken_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_stem
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 1280
|
| 45 |
+
n_head: 20
|
| 46 |
+
n_layer: 36
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: bittoken
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: bce
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/configs/config_bittoken_text67.py --tqdm --verbose --seed 42 --run_name bittoken_gpt2_large_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 36 --n_head 20 --n_embd 1280 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/trained/bittoken_gpt2_large --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group bittoken_gpt2_large_text67 --tie_word_embeddings --tokenizer_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_repo/tokenizers/num_text/bittoken_gpt2 --model rope_stem --num_embedding_type bittoken --add_reciprocal --combine_strategy zero_pad --num_loss_type bce --dropout 0 --cache_base_path /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/cache --compile --difficulty_column difficulty --train_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Multiplication_binary_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/DivM_binary_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Division_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard standard standard --train_set_ratios 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized_number --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Multiplication_binary_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/DivM_binary_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Division_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt pretokenized_number --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Sat Oct 3 14:41:54 2026'}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: bittoken_gpt2_large_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/trained/bittoken_gpt2_large
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_repo/tokenizers/num_text/bittoken_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/cache/bittoken_gpt2_47200109/96841076'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/cache/bittoken_gpt2_47200109/46368368'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/cache/bittoken_gpt2_47200109/63050008'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/cache/bittoken_gpt2_47200109/4733157'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/cache/bittoken_gpt2_47200109/44316829'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/cache/bittoken_gpt2_47200109/48579485'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/cache/bittoken_gpt2_47200109/63589654'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/cache/bittoken_gpt2_47200109/28473695'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/cache/bittoken_gpt2_47200109/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized_number
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Multiplication_binary_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/DivM_binary_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'pretokenized_number']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Multiplication_binary_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/DivM_binary_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-04/native/paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: bittoken_gpt2_large_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/04_bittoken_large/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_binary_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_DivM_binary_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_binary_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_DivM_binary_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_binary_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_DivM_binary_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_binary_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_DivM_binary_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_binary_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_DivM_binary_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,4.03637176937293,3.555820471899173,3.972556123374943,3.5524727225011556,3.9077191234272277,4.050698887410059,0.7703896164894104,2.1452657492821645,4.513022592601677,1.0000008561856553,1.000005053351662,1.000007877022518,1.0000094692089783,1.0000034452184494,1.0000192833954222,2.1608303775281783,1.0001295603022584,89.24524175735759,10.934027519952302,0.3741167427627262,0.02163780404580575,0.02534385458844543,0.01786948440997258,0.023201137740437874,0.1250000037252903,0.062499990686774254,0.09047092497348785,0.6207047640091482,0.04016553983092308,0.10416666666666667,0.0,0.0,0.010416666666666666,0.0,0.48606314504013426,nan,nan,0.46919475873194944,nan,nan,5.189877311013779,0.023712244601354033,0.023811001336471464,0.026823457715099582,0.022742137387570627,0.02579169165522672,0.008470457649313352,0.0029454854707135006,0.02053363002183607,5.035047178717029,737.3252025292854,0.25465611382671827,0.004997126321997158,26.147055035858244
|
| 3 |
+
15264,250085376,4.759754086615733,4.116185673960596,4.798321178778982,3.906037293011347,4.498007702826726,4.850851548114676,0.7363763153553009,2.4401097968295202,3.9331017376777404,1.0000297482347484,1.0007318243873198,1.0000636314631453,1.0000813463708869,1.0000504746020615,1.0000225036420929,2.0884036754169397,1.0001826737125243,50.11509827002218,6.578296016427989,0.38058338093240335,0.13580116781577284,0.11657498632984498,0.16414590821875255,0.12679125276138495,0.140625,0.01692708022892475,0.10385061055421829,0.9073200588579038,0.08221584558486938,0.09375,0.0625,0.052083333333333336,0.11458333333333333,0.026041666666666668,0.5111313008577175,nan,nan,0.4957323886580222,nan,nan,4.58817710925823,0.021335779599683354,0.021548423576373724,0.02485608002727371,0.020553550527497452,0.023836499791732937,0.007129124543298645,0.0026723532615006416,0.015505488198767934,4.45073978755178,56.17192996872796,0.35391949809525375,0.009998084214664627,24.122569550662103
|
| 4 |
+
22896,375128064,5.314593564037864,4.369203527348627,5.070896888339766,4.132978563434317,4.756819204053992,4.7115943665039595,0.7184672057628632,2.4819182691887924,3.7597964345865575,1.0000026331036382,1.000002065876933,1.0000023291181108,1.0000022301369458,1.0000027771450664,1.0000176664543572,2.051430431133907,1.0000345623006044,42.18316711812597,5.692740201488393,0.389959034891059,0.18496993104030954,0.1426870297325284,0.17992759986274592,0.1466038815637476,0.265625,0.1080729104578495,0.2147466316819191,0.9770383914575816,0.19541525840759277,0.11458333333333333,0.09375,0.041666666666666664,0.125,0.026041666666666668,0.6155909518226205,nan,nan,0.6085519450716674,nan,nan,4.288734986326396,0.02025338590853203,0.02036595129073421,0.02380852382073659,0.019380380267896936,0.022773414559644118,0.006425175957373915,0.0025677585680734766,0.013310709773410881,4.159849664002428,41.155929024489424,0.38041644065838065,0.014999042107332095,24.124430128951257
|
| 5 |
+
30528,500170752,6.189133451320066,5.0425257465490105,5.945971276300412,4.843753498666771,5.742858689276744,5.44160832200426,0.7222471237182617,2.7263893282853733,3.67835386681448,1.0000001251075101,1.0000002166875839,1.0000002641849686,1.000000518434671,1.0000006370232302,1.0000330735755019,2.059055057833941,1.0000188349044223,38.93609097683167,5.332799967175944,0.3986254745318126,0.22181042006178958,0.1667487253531274,0.20520610732938732,0.16377136813004198,0.4947916716337204,0.08203125931322575,0.3465813398361206,0.9845170569259669,0.21374712884426117,0.09375,0.13020833333333334,0.057291666666666664,0.13541666666666666,0.026041666666666668,0.7950385509757325,nan,nan,0.7174051549324455,nan,nan,4.109311060484038,0.019549658052443644,0.01965147680070923,0.023168580722995102,0.01868535818513073,0.022136309517516252,0.005777407633708727,0.0025112197517479116,0.011798767552836029,3.9860322610855854,36.48962488301895,0.39105819414059323,0.020000000000000004,23.986091942612916
|
| 6 |
+
38160,625213440,6.2215324264640595,5.032206009708716,5.943272471959819,4.790798271692877,5.6955382019884055,5.894107767027114,0.5399542301893234,3.017414562682764,3.5866292486704574,1.0000000021730857,1.0000006267261523,1.0000007555245727,1.0000002433827673,1.000000277221912,1.0000015378820843,1.7189092360339864,1.0000263781660235,35.500603174717185,4.913282470203085,0.4055408486254364,0.2574758277930271,0.17923631371394727,0.2501232287119263,0.17654914889278903,0.765625,0.35546875,0.6236962676048279,0.9938657362868036,0.31779739260673523,0.109375,0.140625,0.0625,0.17708333333333334,0.03125,0.9198759194731769,nan,nan,0.8702207181292275,nan,nan,3.984103869285973,0.019037825070454328,0.019092686081098185,0.022662978343454734,0.01818677716814294,0.021675629063804228,0.005093461124224499,0.002410901892829962,0.010453192284111075,3.865490398326385,33.42598114473515,0.39926381013209716,0.019965729849129636,23.9626018356874
|
| 7 |
+
45792,750256128,6.213891125594524,5.120037810566474,5.864951217547056,4.641164541554947,5.506890036476105,5.601014915853705,0.3614708483219147,2.856214994211382,3.5294517098421543,1.0000000167637562,1.0000000164533154,1.000000004346171,1.000000000310441,1.000000017695079,1.0000000235934705,1.4359459277808924,1.000007078545176,33.54441050559829,4.6644848434540656,0.42302428354842897,0.29238474311918344,0.20368798797303017,0.2667398212699372,0.20094265734941108,0.9322916567325592,0.615885391831398,0.780881255865097,0.9971087059441789,0.36886945366859436,0.125,0.171875,0.06770833333333333,0.19270833333333334,0.036458333333333336,0.9773977403917039,nan,nan,0.9205838528772196,nan,nan,3.8882512594908367,0.018640075456940037,0.01862295842771812,0.0222369926943555,0.01776040529661078,0.021283682874484594,0.004469597399699741,0.0022282903930132257,0.009256154368226813,3.7737530832843382,31.043383325920665,0.4063805774307226,0.019863180383909898,24.007156707453337
|
| 8 |
+
53424,875298816,5.656005944746232,4.556466341329117,5.041224822401722,4.261986216530205,4.932134553169208,4.917175422846412,0.18345341831445694,2.558904763207354,3.4870236909676655,1.000000005898372,1.000000000310441,1.0000000645714242,1.0000000006208818,1.00000000434617,1.0000562799123358,1.2029139588000404,1.0000000472606128,32.16424377871109,4.485246015603486,0.4290661011222584,0.3177626640859992,0.21748549410402546,0.297990489402601,0.22166087143314295,0.9635416865348816,0.811197929084301,0.8844553232192993,0.9991687465321138,0.40277618169784546,0.11979166666666667,0.18229166666666666,0.07291666666666667,0.21354166666666666,0.052083333333333336,0.9882733744258682,nan,nan,0.9534683728901049,nan,nan,3.5831625080822977,0.0174157431186128,0.017313410677692636,0.021065351539512722,0.01651160132618599,0.02014432906106135,0.00318993674703933,0.0018767754431693612,0.006212361066588422,3.4794329812295945,29.55024471235475,0.41133605447877003,0.019693132578928373,23.877041203606108
|
| 9 |
+
61056,1000341504,5.453815857879817,4.675820132677151,5.142014231969961,4.3351496079305685,4.834567909439402,5.216901681074848,0.09424066916108131,2.6992583364802343,3.4613644014073843,1.000000000310441,1.0000006562318497,1.0000176896631956,1.0000000813345045,1.0000000049670508,1.0000001800550697,1.0993422536025248,1.0000000089961036,31.36388705065038,4.384805325090124,0.4365431651096686,0.2891551186039237,0.215984228858773,0.2987491877358813,0.23155668237614135,0.984375,0.921875,0.9288743138313293,1.0,0.40500184893608093,0.15104166666666666,0.15625,0.06770833333333333,0.20833333333333334,0.052083333333333336,0.9941917262040079,nan,nan,0.97106870651866,nan,nan,3.4724788270021216,0.016939326902349682,0.01672454867892717,0.02052278965607463,0.01595595276492806,0.01962134333977907,0.002315717204992687,0.0015590021960800269,0.004753993422385717,3.374086135119762,28.476255898955483,0.4151654080946615,0.01945688144839207,23.901588892707196
|
| 10 |
+
68688,1125384192,6.506463686625162,5.661144455894827,6.1592477560043335,4.940496484438578,5.483571605134646,5.959296027819315,0.020370779559016228,4.030693599155971,3.43728033969334,1.0,1.0000000006208816,1.0,1.0,1.0000001154826563,1.0,1.0205801842115765,1.0,30.619543554134847,4.293347094938884,0.43903202571838884,0.33689807707960484,0.23602933080896005,0.3103878195327294,0.23936138873523047,0.9895833134651184,0.973958358168602,0.9170617461204529,1.0,0.42836707830429077,0.14583333333333334,0.19270833333333334,0.078125,0.21875,0.057291666666666664,0.9960167314857244,nan,nan,0.968722619737188,nan,nan,3.406284178020777,0.016581731596470247,0.01633250207539679,0.020155754916491617,0.015598125149141854,0.019279471936422207,0.001575528111395738,0.0012065430290944892,0.0035359087931023975,3.3120185955109336,27.629662426137823,0.41839591847963054,0.019156226183831337,24.087120700012907
|
| 11 |
+
76320,1250426880,6.938718557357788,5.709149638811747,6.034729878107708,5.129888654686511,5.436059237768252,6.2037529548009225,0.04012706596404314,3.713933403143896,3.414549643173814,1.0,1.0,1.0,1.0000000009313226,1.0000000012417631,1.0,1.0409872463052374,1.0000000033985,29.954096608793105,4.221675984518881,0.4478767530715144,0.3302949798789174,0.24023865411749978,0.31902067485611457,0.2381980069223452,0.984375,0.973958320915699,0.9362753033638,1.0,0.4313167631626129,0.140625,0.1875,0.08333333333333333,0.22916666666666666,0.057291666666666664,0.9908006130717695,nan,nan,0.9735592775978148,nan,nan,3.3568336920907744,0.016297196415379894,0.016004111757567042,0.01984362757758574,0.015290827825317287,0.018988492325596528,0.0009999029396816634,0.0008397366477011563,0.0025377136001706276,3.2660320662612854,27.13106862529019,0.4204304724759901,0.018793456452196412,23.95805505240303
|
| 12 |
+
83952,1375469568,6.931008911381115,5.4445453506583945,5.917727947235107,5.074262421267727,5.580669470131265,5.397782325744629,0.02313023991882801,3.133156797478261,3.4021268960398934,1.0000000161428757,1.000000002173085,1.0,1.0000000009313226,1.000000027318663,1.0,1.0234135886108864,1.0011570270138832,29.55610993067432,4.175631176985004,0.4595686652043107,0.3394109688645135,0.24176968891961903,0.316785214372176,0.24034329372614155,0.9895833432674408,0.98046875,0.9763090014457703,1.0,0.436246395111084,0.15104166666666666,0.203125,0.078125,0.21875,0.052083333333333336,0.9947619661688805,nan,nan,0.9893937942882379,nan,nan,3.319381300302692,0.016056302857030753,0.01576368006370613,0.019617902373769442,0.015036002118521189,0.018741108230861836,0.0006611086814754707,0.0005239951242924086,0.001880834517964902,3.2311003489782975,26.705342504713272,0.4221946784535294,0.018371334958694578,23.912240862240502
|
| 13 |
+
91584,1500512256,6.305318474769592,5.119822422663372,5.278163353602093,4.718279639879863,4.950106143951416,4.085937378384794,0.014096436789259315,2.6438411076863613,3.391597936337348,1.0,1.0,1.0,1.0,1.0,1.0034316759511968,1.0142682819618005,1.0,29.23590316991742,4.1392892364256015,0.4599391295346733,0.3054712243790003,0.24026994331114954,0.31835918270315344,0.25214045021522663,0.984375,0.986979179084301,0.9704093039035797,1.0,0.43318116664886475,0.15104166666666666,0.16666666666666666,0.08854166666666667,0.22916666666666666,0.07291666666666667,0.9926855793843666,nan,nan,0.9861601089748243,nan,nan,3.2920008057573513,0.01585610173164283,0.01561227934700219,0.01946598067197943,0.014861414341938082,0.018564636342583457,0.0005005379376511848,0.00032190070989487023,0.0015262037827593765,3.205291733900664,26.336394307498413,0.42380140834650026,0.017893076407163026,23.89860363174447
|
| 14 |
+
99216,1625554944,6.118584871602555,5.425667365392048,5.513063675413522,4.859486222888033,5.131382347395022,4.031880696614584,0.0031762002618052065,2.4672216802128197,3.3796238548044735,1.000000000310441,1.0,1.0000000062088108,1.0000000006208816,1.0000000012417634,1.0,1.0031826351293034,1.0000167237105315,28.903574866744083,4.100752692662869,0.46699284781385175,0.33208983935789027,0.24463932112893838,0.3213696085581557,0.25173929666444367,0.9895833432674408,1.0,0.9777839183807373,1.0,0.4420967102050781,0.15625,0.19270833333333334,0.08854166666666667,0.22395833333333334,0.0625,0.9946594367114207,nan,nan,0.9903249243895212,nan,nan,3.2705534566050636,0.015677655745228683,0.015495677873089383,0.019348076335789024,0.01473654924941026,0.018431444090558705,0.0004154292888932862,0.000197805845732395,0.001311515351094649,3.1849392859289942,25.968953541177374,0.4253141184608829,0.017362323018206012,23.9473384286646
|
| 15 |
+
106848,1750597632,6.030339797648289,5.3169387684513945,5.402314067818224,4.794945319493612,5.154229204170407,4.1358888149261475,0.023313908372074366,2.5662928781516494,3.364437067648396,1.000000000310441,1.0000000105549711,1.0000000009313226,1.0,1.000000000310441,1.0,1.0236960285840229,1.0155903735148253,28.47492588591437,4.057134700013377,0.48573243082774753,0.344652270774246,0.256007485624333,0.3286090398704202,0.26940923229548797,1.0,0.98046875,0.9659581780433655,1.0,0.45739901065826416,0.18229166666666666,0.203125,0.08854166666666667,0.22916666666666666,0.08333333333333333,1.0,nan,nan,0.9845409364594767,nan,nan,3.2525382202657513,0.015516993844576419,0.015400793563569997,0.019242446357091457,0.014637808172682405,0.018317885131163197,0.0003679956851783096,0.00012951163740176235,0.0011734923054128178,3.1677512764633953,25.546672963808668,0.4270879747717011,0.016783116791537683,23.873446657461095
|
| 16 |
+
114480,1875640320,6.518341104499997,5.212678511937458,5.220403472892941,4.774969418843588,5.0648543449712715,4.337981939315796,0.005934932210948318,2.7379272110050623,3.3420441283184723,1.000000000310441,1.0,1.000000000310441,1.0,1.000000120448377,1.0,1.0059622498184493,1.000000001241762,27.839768282542746,3.9828589616302463,0.4876369628512707,0.3482558406527329,0.25895051263777213,0.3232712807441317,0.2685692379333509,0.9947916567325592,0.986979179084301,0.9940871298313141,1.0,0.45859667658805847,0.20833333333333334,0.19791666666666666,0.09895833333333333,0.21875,0.09375,0.9991825828328729,nan,nan,0.9977422673255205,nan,nan,3.2366981918810964,0.015357426530811714,0.01531755324860854,0.01914535838045547,0.014554321518669635,0.018216906358249706,0.0003147274831705495,0.00010173540414370772,0.0010516263328411168,3.152638520251447,25.112628209453458,0.42883483488776525,0.016159868723772935,23.926390523406525
|
| 17 |
+
122112,2000683008,6.683272202352651,5.336358149846394,5.364366173744202,4.922142429898179,5.3689144076779485,4.443094333012899,0.0018419506959617138,2.7287334488205297,3.332046562708759,1.000000873745218,1.0,1.0,1.0000000037252876,1.000000002173085,1.0,1.001843735291243,1.000000000065356,27.56836776171242,3.952245819634734,0.4864744420117769,0.35759962615137786,0.2676341904574109,0.3345168209750111,0.27651458717607247,0.9947916567325592,1.0,0.9911241233348846,1.0,0.4685426652431488,0.19270833333333334,0.203125,0.08333333333333333,0.234375,0.078125,0.9988490318258604,nan,nan,0.9957891988257567,nan,nan,3.2216908297365485,0.015198610878889894,0.01523487920771014,0.019054789003619312,0.014477130029936908,0.018117590219037544,0.0002903503330781026,9.033138536967108e-05,0.0009762350325194478,3.1382508975180983,24.780416025118257,0.430297210234641,0.015497325216091833,23.982028945682174
|
| 18 |
+
129744,2125725696,6.43808404604594,5.260818998018902,5.303209622701008,4.839763641357422,5.355708400408426,4.4261592626571655,0.01095319219166413,2.759583109901065,3.320834786204311,1.0,1.0,1.0,1.0,1.0,1.0,1.0110606264308517,1.0,27.259870483865825,3.918992345588519,0.49360581369147416,0.35370823294680465,0.27129116552416754,0.3260386766896991,0.2923089108036844,1.0,0.993489570915699,0.9821691811084747,1.0,0.472390353679657,0.18229166666666666,0.19791666666666666,0.09375,0.22395833333333334,0.10416666666666667,1.0,nan,nan,0.9915868431950609,nan,nan,3.2074436299700704,0.015038276059653253,0.015162414402240832,0.018969036602290837,0.01440467673982217,0.018029187293023597,0.0002736037388180222,8.202493047798223e-05,0.0009157437466980373,3.124568650193101,24.50276400258206,0.43144364440247446,0.014800531927597655,23.902321722353218
|
| 19 |
+
137376,2250768384,5.634427269299826,5.29994010925293,5.308737358699241,4.775896271380286,5.30618965656807,4.677400787671408,0.024215975077822804,2.8181866916076674,3.3086118744492223,1.0,1.0,1.0000000012417631,1.000000000310441,1.000000000310441,1.0,1.0246856090801635,1.0000000678981298,26.941237877647684,3.8851026173876244,0.5074450245610735,0.35865284145586973,0.27745912298938413,0.3476415028463905,0.30985714862612646,1.0,0.986979141831398,0.9852243959903717,1.0,0.48661765456199646,0.203125,0.19791666666666666,0.09895833333333333,0.23958333333333334,0.140625,1.0,nan,nan,0.9931132787217697,nan,nan,3.1933573597324925,0.01485924393689985,0.0150888808335846,0.01887986702655112,0.014337994006871111,0.017940280352202378,0.00024589687632453253,7.577140756835099e-05,0.0008457353277822586,3.1110836734421223,24.183541143152947,0.43265912623634756,0.014074795349650518,23.924630119189995
|
| 20 |
+
145008,2375811072,6.042944789243241,5.361623438384637,5.347682040184735,4.918386336408711,5.364988247553509,5.219066739082336,0.001682605070527643,3.0423824351936624,3.2956620372521384,1.000000000310441,1.0000000717108557,1.0000000012417631,1.000000194949602,1.0,1.0,1.00168405269877,1.0000003765505738,26.593564823885888,3.8439166134830995,0.5138266140013299,0.3551294423215815,0.28048318924885096,0.333556651548876,0.3028254835687309,1.0,1.0,0.9852375686168671,1.0,0.48276686668395996,0.203125,0.19791666666666666,0.109375,0.22916666666666666,0.125,1.0,nan,nan,0.9949112189933658,nan,nan,3.179240796577048,0.014656100345103187,0.015015051727102461,0.018794484278606933,0.014267656171972948,0.01785178002349806,0.00023139859190295095,7.049895739595384e-05,0.0007943098907061438,3.0975595011573045,23.847081338834464,0.43433046553369836,0.013325642393809753,23.944956054007395
|
| 21 |
+
152640,2500853760,6.257820526758829,5.432021220525105,5.405835908527177,4.9479607741038,5.549466013908386,5.207735101381939,0.005604798760032281,3.113965762323494,3.2869503197725862,1.0,1.0,1.0000000018626447,1.0,1.0,1.0,1.0056334862462046,1.0000000012296275,26.37301260258835,3.819849565769647,0.5167475381268635,0.3673209961429398,0.2795258582939506,0.3495093890257892,0.31577613146334294,1.0,0.993489570915699,0.985198050737381,1.0,0.492217093706131,0.203125,0.21354166666666666,0.109375,0.25,0.13020833333333334,1.0,nan,nan,0.9931280196954807,nan,nan,3.1662224786737476,0.014447468240041828,0.014940855169120805,0.018713512202623288,0.01419504652680682,0.017761721044461873,0.00021988771386582723,6.627146699317326e-05,0.0007506216953661376,3.0851270802173265,23.604678426711065,0.43530241672957093,0.012558778301146429,23.844795318732118
|
| 22 |
+
160272,2625896448,6.1771996815999355,5.403276880582174,5.3220679536461315,4.917365312576294,5.39987584544966,5.211146076520285,0.001297308161156252,3.043969018118722,3.275503420329187,1.0,1.0,1.0000000173846382,1.0,1.0000000058983698,1.0,1.0012983524158254,1.0,26.0692105360545,3.7856121013059263,0.5296878896593187,0.3656999530646281,0.2873603792304704,0.35704244076507774,0.3208001320181444,1.0,1.0,0.991150438785553,1.0,0.4991886019706726,0.20833333333333334,0.21354166666666666,0.10416666666666667,0.2552083333333333,0.13020833333333334,1.0,nan,nan,0.9938154096404711,nan,nan,3.154050914160673,0.01422997972344528,0.014859922994224317,0.01862751169103848,0.014121680107041882,0.01766516688389796,0.00020696315797416582,6.2483294460583e-05,0.0007041785933315821,3.073573013409649,23.331299242920846,0.4366881512574055,0.011780043193471874,23.882833983191723
|
| 23 |
+
167904,2750939136,6.347605586051941,5.313564938182633,5.147258450898025,4.81997827688853,5.232649843208493,5.233733455340067,0.0005842237733304501,3.0387089366004583,3.2671690432083174,1.0,1.000000001862645,1.0000000102445408,1.0,1.000000000310441,1.0,1.0005843961526675,1.0,25.850463779393902,3.7612275764404663,0.5337435071011662,0.3645417390705161,0.28805808490559864,0.3508493972556482,0.3220731301473241,1.0,1.0,0.998525083065033,1.0,0.49875760078430176,0.21354166666666666,0.203125,0.109375,0.25,0.13541666666666666,1.0,nan,nan,0.999795559172829,nan,nan,3.1422428938603795,0.01402073372597985,0.014782873974009245,0.01854379155035609,0.014048831500757322,0.017572899184864637,0.00019663959805909126,5.896848821699551e-05,0.0006650624318830623,3.0623530787210793,23.078158397844504,0.43793124038983816,0.010995367597369719,23.953998589724574
|
| 24 |
+
175536,2875981824,6.239434281984966,5.38321403585627,5.36470949711899,4.935518900553385,5.427261988321941,5.543798605600994,0.001393282349454239,3.1079389936101616,3.2582358688620543,1.0,1.0000008734348524,1.0000000006208816,1.0,1.0,1.0,1.0013945095162349,1.000000000065356,25.616901128575208,3.7353662791347255,0.5353936558408139,0.37452407299938917,0.2951386318036491,0.3533951313288215,0.3237796312262993,1.0,1.0,0.9941003024578094,1.0,0.5041952729225159,0.22916666666666666,0.20833333333333334,0.11458333333333333,0.24479166666666666,0.13020833333333334,1.0,nan,nan,0.9981461549177766,nan,nan,3.1303469806852284,0.013816551137686291,0.014694102524189442,0.018451725201668892,0.013972816176359117,0.017472732613829,0.00018595393859705298,5.641209859987597e-05,0.0006273353408008614,3.05106933809593,22.804099225085736,0.4392633353686058,0.01021072727974235,23.965444452797655
|
| 25 |
+
183168,3001024512,6.629194617271423,5.515606430048611,5.486076871554058,5.02969006759425,5.519721110972264,5.535000562667847,0.0006275189953157678,3.11687015351795,3.2511102704253667,1.0,1.0000000266977818,1.0,1.0000000006208816,1.000000000310441,1.0,1.0006277385174696,1.0,25.444057359175634,3.716076125035801,0.5413331724726211,0.3815944563165481,0.30193913075380846,0.34783234685740777,0.3303626880851531,1.0,1.0,0.9970501363277435,1.0,0.5089150071144104,0.21875,0.22395833333333334,0.11979166666666667,0.24479166666666666,0.13541666666666666,1.0,nan,nan,0.9975405208145579,nan,nan,3.1199051549646466,0.013629930767807967,0.014609890267305638,0.01836523212636297,0.01389472496290356,0.017374446420737686,0.00017496168088127875,5.354816551657398e-05,0.0005945401309747772,3.041207866096292,22.733887711348522,0.43969699631853176,0.009432097738825898,23.919414731457785
|
| 26 |
+
190800,3126067200,6.55667742093404,5.462266529597127,5.401500864575305,4.936521251996358,5.3979525963465385,5.189440449078877,0.0006485345293185674,2.9344411157998755,3.2416199051658623,1.0,1.0000001241736578,1.0000000037252876,1.0,1.0,1.0,1.0006488344450049,1.0000000034632173,25.20321456640248,3.689318170245516,0.5503737011715286,0.3878040040494767,0.3063387562698259,0.3706599539164003,0.3439268083597669,1.0,1.0,0.9955357015132904,1.0,0.5211379528045654,0.24479166666666666,0.22916666666666666,0.125,0.2552083333333333,0.15625,1.0,nan,nan,0.9978896860654155,nan,nan,3.109708107096761,0.013471566665374851,0.014527596105140674,0.018277681419566656,0.013823935913688688,0.017279046919563907,0.0001676406201131854,5.1390164562954864e-05,0.0005652176754653486,3.031544016144394,22.419935219692736,0.4409110539200358,0.008665408697250875,23.95944528083575
|
| 27 |
+
198432,3251109888,6.540345748265585,5.468631587612132,5.416602691014607,4.956045548121135,5.386638641667862,5.591535369555157,0.0002015497666434385,3.1376014437647846,3.234728875278961,1.0,1.000000002173085,1.0,1.0,1.000000000310441,1.0,1.0002015721820448,1.000000000065356,25.028734903547186,3.66988183091979,0.5495918952581385,0.3949989220160221,0.31559496354932837,0.3684030034522563,0.3436436753468476,1.0,1.0,0.998525083065033,1.0,0.5249133110046387,0.22916666666666666,0.22916666666666666,0.125,0.2604166666666667,0.140625,1.0,nan,nan,0.9987881227085987,nan,nan,3.1002419561038352,0.013332599695172528,0.014440357354960966,0.018184088644894406,0.013754083127174514,0.017179694580149157,0.00016052557403129885,4.967549409520413e-05,0.0005393659927432492,3.022601549831889,22.3150279135824,0.4416153206288877,0.007916498943711495,23.998452187253434
|
| 28 |
+
206064,3376152576,6.445185542106628,5.3772521031399565,5.325240097629526,4.851263960512975,5.256947875022888,5.706901152928669,0.0005775890786026139,3.1505785761744027,3.232693446994138,1.0,1.0000000012417631,1.0000000021730857,1.000000000310441,1.0,1.0,1.0005778872082312,1.0000000092144767,24.980988464237843,3.66461848493176,0.550045838510999,0.39739888474468293,0.3185247872645234,0.3723760070540187,0.35866432040073337,1.0,1.0,0.9970501363277435,1.0,0.530960738658905,0.22916666666666666,0.23958333333333334,0.13020833333333334,0.265625,0.171875,1.0,nan,nan,0.9975405208145579,nan,nan,3.091965784907736,0.013210886998989138,0.014355243446034003,0.01809150550426924,0.013673077350338186,0.017080683207626186,0.00015469544752121865,4.823677449755101e-05,0.0005182293957958283,3.0148332101806403,22.21465379187146,0.44223100312085395,0.007191071867150021,23.999652727886563
|
| 29 |
+
213696,3501195264,6.503522992134094,5.407836437225342,5.345969597809018,4.8123280207316075,5.159709056218466,5.727702975273132,5.6378357840003446e-05,3.1449253721521755,3.2255964567302726,1.0,1.0,1.000000000310441,1.0,1.0,1.0,1.0000563799862316,1.0000036833592891,24.80544149636944,3.6450557288917116,0.5525773635428273,0.40493205794335363,0.3177904320605276,0.3742155154759834,0.3537036658844915,1.0,1.0,0.9955620467662811,1.0,0.531603217124939,0.23958333333333334,0.234375,0.13020833333333334,0.2708333333333333,0.15104166666666666,1.0,nan,nan,0.9967685844749212,nan,nan,3.0842507498574414,0.013103246300954404,0.01426704654243601,0.01799535683795854,0.01359384006159712,0.016977188065059087,0.00015008120894375644,4.729063136261916e-05,0.0004979323536122662,3.0076187517907886,22.045892363136918,0.443100500926063,0.006494652022090374,23.86025275701139
|
| 30 |
+
221328,3626237952,6.513107419013977,5.529143412907919,5.514358679453531,4.947335918744406,5.377792120290299,5.702961087226868,6.22807474428555e-05,3.1552292755251443,3.220265008509159,1.0,1.0,1.0,1.0,1.000000000310441,1.0,1.0000622831783788,1.000000183880891,24.68091980800723,3.63122025281966,0.5516907077453873,0.39624235574856315,0.323406620052864,0.38538947886218866,0.34872981137791287,1.0,1.0,0.9940739572048187,1.0,0.5326487421989441,0.23958333333333334,0.22395833333333334,0.140625,0.2760416666666667,0.140625,1.0,nan,nan,0.9959797312815984,nan,nan,3.0781095942611216,0.01300808587239256,0.014190274979836215,0.01790561155761982,0.01351749007080849,0.016878984909121323,0.0001470222062513495,4.629528872584088e-05,0.00048754708642714334,3.001928265304789,22.004385806229628,0.44307543756733153,0.005832543055900948,23.850168733552135
|
| 31 |
+
228960,3751280640,6.606680552164715,5.434385340350369,5.442104736963909,4.880340894063314,5.31558096408844,5.876959959665934,5.020540083933156e-05,3.2494012514750157,3.2165430291788653,1.0,1.0000000009313226,1.0,1.0,1.0,1.0,1.0000502067013077,1.0,24.591147689064602,3.6212442107441367,0.5575166806946057,0.39785032589826147,0.32608938639297147,0.37329262256276574,0.3636532351725431,1.0,1.0,0.9925858378410339,1.0,0.5353947877883911,0.23958333333333334,0.234375,0.13020833333333334,0.2604166666666667,0.18229166666666666,1.0,nan,nan,0.9949040741970142,nan,nan,3.071450913911991,0.012922862367022777,0.014104085019439822,0.01781157971511756,0.013437478887297963,0.016777346103148214,0.00014526368478688846,4.56319691324542e-05,0.00047596490145614616,2.9957306848449736,21.839120836440372,0.44404392630111744,0.005209787318395601,23.942872262456717
|
| 32 |
+
236592,3876323328,6.550989747047424,5.501219630241394,5.52673836549123,4.905669887860617,5.375631252924602,5.767499605814615,9.501012391410768e-05,3.2015627855170394,3.213365128477259,1.0,1.0,1.0,1.0,1.0,1.0,1.0000950158440274,1.0000013572671609,24.512605505447564,3.6125224309509725,0.5568855622022393,0.4121394853902889,0.323042416064259,0.37252616945143247,0.3724308426967989,1.0,1.0,0.9970501363277435,1.0,0.5392441153526306,0.25,0.23958333333333334,0.13541666666666666,0.2552083333333333,0.18229166666666666,1.0,nan,nan,0.9975405208145579,nan,nan,3.0666728446443625,0.012848156287040688,0.014030044893957061,0.017726176274695662,0.013365235410866299,0.01668537099503051,0.00014382112374063718,4.484420241232226e-05,0.00046898694622341165,2.9913601921148896,21.808906282581372,0.4441978212609981,0.004631127461369502,23.943352290709594
|
| 33 |
+
244224,4001366016,6.564970016479492,5.443135106315212,5.433613061904907,4.8243049780527745,5.2766313155492135,5.74575702349345,4.1765608330024406e-05,3.194724516629117,3.210660573696562,1.0,1.0000000037252876,1.0,1.0,1.0,1.0,1.0000417664820813,1.000000024964777,24.44651103840396,3.605172537064012,0.5598226382408265,0.40927658430378083,0.3311531177153558,0.38285715075798354,0.36715606607079304,1.0,1.0,0.9970501363277435,1.0,0.5425640344619751,0.25,0.234375,0.13020833333333334,0.2708333333333333,0.17708333333333334,1.0,nan,nan,0.9975405208145579,nan,nan,3.062222918192254,0.012780324959685565,0.013963076237979287,0.017647703616309764,0.013295940257526562,0.016601621599283505,0.00014307938175630507,4.4183426645150714e-05,0.0004651950368251673,2.987281777094828,21.73443938270055,0.44461743748381705,0.004100970320511953,24.02179764448072
|
| 34 |
+
251856,4126408704,6.599566539128622,5.5318621794382725,5.564337373400728,4.887136022249857,5.381445686022442,5.865751504898071,3.885080877807923e-05,3.248838131389838,3.2091321921131266,1.0,1.0,1.0000000006208816,1.0,1.0,1.0,1.0000388515796192,1.0000000018921282,24.41059787391482,3.601181858667494,0.5556581023275742,0.41441337127403893,0.3381005345260808,0.38569703446081005,0.3697330315526932,1.0,1.0,0.9970501363277435,1.0,0.5464235544204712,0.234375,0.23958333333333334,0.13541666666666666,0.2708333333333333,0.171875,1.0,nan,nan,0.9975405208145579,nan,nan,3.058060669892739,0.012728507026281874,0.013902466345110168,0.017574725155852042,0.013243840258177384,0.016528037720578537,0.00014243868917000882,4.3731918985435695e-05,0.00046308031919226923,2.983433825485174,21.66560954673985,0.44500219281099,0.003623353354756449,24.03680234316156
|
| 35 |
+
259488,4251451392,6.586755911509197,5.5211661259333304,5.583295623771847,4.886099497477213,5.385701060295105,5.864908695220947,3.6406342587724794e-05,3.231696018262866,3.2075498772901483,1.0,1.0,1.000000000310441,1.0,1.0,1.0,1.0000364070822299,1.000000071069906,24.36893950789052,3.5965528873725665,0.55725417079374,0.41551965851520045,0.3356318228815899,0.38742244614055465,0.36755628990281397,1.0,1.0,0.9970501363277435,1.0,0.5459380745887756,0.23958333333333334,0.23958333333333334,0.140625,0.2604166666666667,0.15625,1.0,nan,nan,0.9975405208145579,nan,nan,3.055267083741709,0.012681066667499017,0.013846143285430197,0.017509743542889436,0.013192904402894326,0.016462483869443805,0.0001419897720484711,4.32459697047817e-05,0.000461885730181879,2.980927603804799,21.68490815006462,0.4448284034068604,0.003201913898652705,23.942246167829374
|
| 36 |
+
267120,4376494080,6.508547862370808,5.498236815134685,5.547986626625061,4.877329667409262,5.378898580869038,5.811745325724285,3.86502169931191e-05,3.2082011603668605,3.2064988540951163,1.0,1.0,1.0,1.0,1.0,1.0,1.0000386510294828,1.0000000017524753,24.34782971678324,3.5942075966183555,0.5594776355629542,0.4101435553499843,0.3409561520790808,0.38876521569467337,0.3749391980231579,1.0,1.0,0.9970501363277435,1.0,0.5487542152404785,0.24479166666666666,0.23958333333333334,0.140625,0.265625,0.17708333333333334,1.0,nan,nan,0.9975405208145579,nan,nan,3.05274505318669,0.01264658763147741,0.01380050506430257,0.017452453787989812,0.013150802957464158,0.01640218949480051,0.00014147208998514948,4.296592723023089e-05,0.00045909893192468533,2.9786489610641587,21.659727002214836,0.4448867191441466,0.0028398614619212603,23.925469800163246
|
| 37 |
+
274752,4501536768,6.523422797520954,5.5222779512405396,5.527634223302206,4.865257342656453,5.3079527616500854,5.887892246246338,4.3006361011066474e-05,3.247111867857754,3.205534517765045,1.0,1.0,1.0,1.0,1.0,1.0,1.0000430072864581,1.0000000024053959,24.32443986488947,3.5916092082868136,0.5634013475582913,0.4154575527574532,0.3355258705208405,0.3908071384406535,0.3799986158125501,1.0,1.0,0.9970501363277435,1.0,0.5502679944038391,0.24479166666666666,0.234375,0.13541666666666666,0.2760416666666667,0.1875,1.0,nan,nan,0.9975405208145579,nan,nan,3.0509192546937616,0.012617670120855247,0.01375634445139948,0.01741106587199847,0.013115651804740995,0.016358209759905505,0.00014055684044643536,4.2728924797735625e-05,0.0004596517970782347,2.977017359395468,21.60143182993685,0.44521414116415464,0.0025399532871463355,23.958727348217202
|
| 38 |
+
282384,4626579456,6.603790521621704,5.480412443478901,5.4855382442474365,4.84246571858724,5.282943050066631,5.932856400807699,3.887775437760865e-05,3.2625214516916294,3.205523228350406,1.0,1.0,1.0,1.0,1.0,1.0,1.000038878565228,1.0000000258086552,24.327634336812626,3.5919636934651678,0.563207290744483,0.4182075482360186,0.3397680417037544,0.38791467208134844,0.37796550189931344,1.0,1.0,0.9970501363277435,1.0,0.5509144067764282,0.24479166666666666,0.24479166666666666,0.140625,0.25,0.17708333333333334,1.0,nan,nan,0.9975405208145579,nan,nan,3.04930618830978,0.012595417921893119,0.01371668593577779,0.017370495557652268,0.013077076852997175,0.01631396074099551,0.0001405096983320406,4.2546368340512926e-05,0.0004583294655383179,2.9755911500792585,21.606220723635495,0.4453095694109641,0.002304473351749283,23.89654431646945
|
| 39 |
+
290016,4751622144,6.555341879526775,5.491250157356262,5.500097274780273,4.8577191432317095,5.301368832588196,5.920136570930481,4.3340180127415806e-05,3.2509800159489166,3.2046469108900055,1.0,1.0,1.0,1.0,1.0,1.0,1.0000433411278769,1.0000000928822832,24.30826609995628,3.589812170440715,0.5636428987886096,0.41543830466581816,0.3394007855716439,0.38799189949851753,0.3775295300641389,1.0,1.0,0.9970501363277435,1.0,0.5502309203147888,0.2604166666666667,0.234375,0.13020833333333334,0.2708333333333333,0.171875,1.0,nan,nan,0.9975405208145579,nan,nan,3.0485404180030904,0.012577102690452554,0.013687261658521195,0.017341805186420735,0.013050030158033138,0.01628122231672478,0.0001403045345510585,4.260448100868991e-05,0.00045733708020507255,2.974962733206967,21.631032583411134,0.4452514298947477,0.0021352149741550937,24.00486176822284
|
| 40 |
+
297648,4876664832,6.563853621482849,5.501565535863239,5.497074170658621,4.882578452428182,5.308256944020588,5.918747385342915,3.341496540087974e-05,3.248344445759057,3.2044835286797024,1.0,1.0,1.0000000434613572,1.0,1.0,1.0,1.0000334155309778,1.0000000697854055,24.30072559943139,3.588973236467681,0.5633219893367587,0.4183059881059213,0.33409646292291734,0.38987388062538725,0.37973125316313705,1.0,1.0,0.9970501363277435,1.0,0.5501135587692261,0.25,0.24479166666666666,0.140625,0.265625,0.18229166666666666,1.0,nan,nan,0.9975405208145579,nan,nan,3.0481464617516636,0.012558572309708011,0.013657954471567509,0.017319061260853953,0.013019793834159793,0.016257384361290508,0.00014022166835645663,4.258928043180171e-05,0.0004564215517365363,2.974694447317255,21.62665720711964,0.4451441746595583,0.0020334671566161833,23.950917257902223
|
training_logs/run_metrics_text67/05_bpe_large/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: False
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 2
|
| 5 |
+
cache_base_path: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/cache
|
| 6 |
+
combine_strategy: sum
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty_sd
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/configs/config_td_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_gpt2
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 1280
|
| 45 |
+
n_head: 20
|
| 46 |
+
n_layer: 36
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: bittoken
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: mse
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/configs/config_td_text67.py --tqdm --verbose --seed 42 --run_name bpe_gpt2_large_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 36 --n_head 20 --n_embd 1280 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/trained/bpe_gpt2_large --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group bpe_gpt2_large_text67 --tie_word_embeddings --tokenizer_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_repo/tokenizers/num_text/td_gpt2 --model rope_gpt2 --dropout 0 --cache_base_path /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/cache --compile --difficulty_column difficulty_sd --train_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Division_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard --train_set_ratios 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Division_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_prompt efficient_prompt efficient_prompt efficient_prompt efficient_prompt efficient_prompt pretokenized --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Sat Oct 3 14:45:58 2026'}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: bpe_gpt2_large_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/trained/bpe_gpt2_large
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_repo/tokenizers/num_text/td_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/cache/td_gpt2_88525419/96841076'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/cache/td_gpt2_88525419/63050008'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/cache/td_gpt2_88525419/44316829'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/cache/td_gpt2_88525419/48579485'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/cache/td_gpt2_88525419/63589654'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/cache/td_gpt2_88525419/28473695'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/cache/td_gpt2_88525419/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'pretokenized']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-full-imports-r4/case-05/native/paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: bpe_gpt2_large_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/05_bpe_large/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,3.3028582533200583,4.065514028072357,3.6664005716641745,2.3054955999056497,0.7344203839699427,4.772740562756856,4.354380210240682,29.183025657431028,67.34238761018433,46.78231011773904,11.649780761589737,2.086383357832674,197.55264221781127,78.24753630660972,61.83486657559969,0.3222685176682835,0.0051320867511535145,0.011384170007533846,0.3802083333333333,0.12109375,0.33402260144551593,0.6850613973637999,0.03393346443772316,0.125,0.0,0.0,0.6493790281529073,nan,nan,0.6153063568805616,0.15538542258461044,0.1338748555087174,5.296676952852643,0.04625129133201277,0.051992586903680166,0.041424582615217476,0.02061326161437703,0.001833661820286049,0.04723325607501579,5.087328284354079,642.1334795204599,0.21583553273592054,0.004997126321997158,23.553429754328032
|
| 3 |
+
15264,250085376,3.078714927037557,4.071947515010834,3.8452245593070984,1.709767758846283,0.7255953898032507,3.761582096417745,3.8805050055185952,22.875488880804966,72.9628241670531,57.594406520266176,6.498107476727657,2.067624769733247,63.61029457008225,48.75275355031102,39.19449999071121,0.3919004193504348,0.09811417273842347,0.08232329094134543,0.671875,0.07421875,0.5461803923050562,0.9210575467497443,0.1636670082807541,0.19270833333333334,0.046875,0.015625,0.8232961408454381,nan,nan,0.7477049898152472,0.15121133302879267,0.12600960320560262,4.609260208607969,0.03719573932996633,0.04666099181878078,0.03672789876955509,0.0116029811710391,0.0016139316463347428,0.026845251098103123,4.448613387718016,54.09501228854841,0.33393980487322034,0.009998084214664627,23.2416690686815
|
| 4 |
+
22896,375128064,2.82289989789327,3.994681159655253,3.6613659262657166,1.1930107399821281,0.6636319557825724,2.6378702769676843,3.728533705075582,22.387784125756678,70.58644666549972,48.7127580763387,3.966731908995223,1.9440959952151793,20.911082604219462,41.89317237685995,30.05743882184071,0.4513191045645845,0.15707807601305418,0.1502898239375732,0.8125,0.18619791666666666,0.730962206919988,0.9758238144247439,0.28889766335487366,0.21875,0.09375,0.057291666666666664,0.9013401174549168,nan,nan,0.8659631247865036,0.16798991133070254,0.14382145784717673,4.291948180570209,0.03286468377580917,0.043811324655496836,0.03415822152244896,0.008266784841054864,0.0015243627581220092,0.019285682748347795,4.152037096145393,41.06245648810948,0.3597649833620907,0.014999042107332095,23.242223463856504
|
| 5 |
+
30528,500170752,2.6438929736614227,3.8796692291895547,3.5807477235794067,0.6337843351066113,0.45448459933201474,1.9960364773869514,3.653247276941935,18.32504879352558,64.12197895546377,45.95772568238312,2.3651738979306405,1.5840794255379478,10.63993505496428,38.853551941502225,25.978213393043937,0.46715073216627573,0.14903892076813235,0.184193850125549,0.8854166666666666,0.4661458333333333,0.7878019710381826,0.9868019063094674,0.35288041830062866,0.28125,0.08333333333333333,0.078125,0.9373281111402321,nan,nan,0.8990659036110932,0.1610928473248047,0.1373759254735584,4.106406316020578,0.03005607892492878,0.04209171661185061,0.032643655144079416,0.0065049846090697435,0.0014260942915247851,0.015312069538295744,3.9783716957344506,36.81820781150954,0.3708296883929388,0.020000000000000004,23.187630785071725
|
| 6 |
+
38160,625213440,2.3445176084836326,3.9142284790674844,3.4552410344282785,0.19527754375788695,0.1957768814948698,1.3371485103076945,3.556549390157064,15.779428065834571,65.15114933863538,39.18385211063882,1.252636763757368,1.2272963103284167,5.615964915978053,35.21986142299667,23.34716984688132,0.5376816863067049,0.19953862910811662,0.2211491433013499,0.9583333333333334,0.8307291666666666,0.8716212262709936,0.9946704083052177,0.44323107600212097,0.3333333333333333,0.11458333333333333,0.09375,0.9788342702862186,nan,nan,0.936138486712783,0.1726012767535868,0.1498273614755211,3.978425576102059,0.027940140332952456,0.04088301523588598,0.0315971115207694,0.005405304336405907,0.0012894120274898225,0.01283948047563759,3.8584710923537013,33.86884136157466,0.37929013234795517,0.019965729849129636,23.21903287622598
|
| 7 |
+
45792,750256128,2.0330047607421875,3.935248911380768,3.579081585009893,0.295787060187043,0.1028862795016418,0.7923505410241584,3.5035574038823447,11.630115783002141,70.26873236989866,44.88068879394522,1.5526405929768217,1.1116753626267957,2.5150033870730466,33.421822859660494,23.625811307026165,0.5650529579093951,0.1931843256454315,0.20145456849698326,0.9479166666666666,0.8893229166666666,0.9175503601630529,0.9979296334156468,0.4325960576534271,0.3802083333333333,0.13020833333333334,0.08854166666666667,0.9698069895481846,nan,nan,0.957485512655694,0.1560051616943383,0.13163055621165162,3.8811069901155304,0.02620286963827834,0.03992938688060674,0.03078354765081181,0.004648597262276264,0.0011411057074582846,0.011132125611330878,3.7672693382826283,31.480751271392815,0.38696642154027205,0.019863180383909898,23.200749281355872
|
| 8 |
+
53424,875298816,1.3727674980958302,3.947390377521515,3.4414391815662384,0.15620514765517632,0.0675344875974891,0.9537818689132109,3.4583969513575235,6.703824892612886,75.47651069483463,37.3418916362771,1.1984534479313276,1.0741071006536302,3.60687938431343,31.951056006095612,22.47896045181695,0.6276272920009366,0.21721767596038533,0.204823209054684,0.96875,0.9479166666666666,0.932929719487826,1.0,0.4593677818775177,0.4479166666666667,0.11979166666666667,0.07291666666666667,0.980249648066092,nan,nan,0.9534038526132159,0.18365239950455248,0.16024838962281743,3.556138997753024,0.02103863299322678,0.03704685200623072,0.028244475431377127,0.0013481049337792032,0.0008798182457284034,0.003670957097759228,3.463910140388708,30.02176079603861,0.3919121333177297,0.019693132578928373,23.174899819331923
|
| 9 |
+
61056,1000341504,0.7969511523842812,3.942318101723989,3.3805355926354728,0.1459803431183294,0.06379248283337802,0.7234832431810597,3.43119215965271,3.126896900251197,73.87094601991059,36.15640261061935,1.2556297452421792,1.0699367048836388,2.580046089222983,31.09703161535239,21.308127097926047,0.731120561445405,0.21919105830959126,0.2271006540779489,0.984375,0.9609375,0.9303692976633707,1.0,0.48393696546554565,0.59375,0.13020833333333334,0.09895833333333333,0.9887191548671884,nan,nan,0.9598975531262113,0.17571399338965912,0.15130158226626614,3.453125380885551,0.01815385272139703,0.035878204921755745,0.027221140272021785,0.0010443275575775734,0.0006818780062242379,0.0029824587555826267,3.3671635037416934,28.808384407928656,0.39677100837527457,0.01945688144839207,23.181384060854864
|
| 10 |
+
68688,1125384192,0.5091318190097809,3.9950585762659707,3.5009428362051644,0.004301525534780619,0.050342244750936516,0.9155550605986112,3.4014151891072593,2.1314277069378043,77.34005217611492,41.91375845128213,1.0043798273743245,1.053569911571793,3.870754088185945,30.193320218861544,22.501037482904064,0.7838720564065885,0.2292476929398323,0.26687296508765174,0.9947916666666666,0.9609375,0.9477308988571167,1.0,0.5190587043762207,0.640625,0.13541666666666666,0.140625,0.9948518580640666,nan,nan,0.9648939061977823,0.18607284971934254,0.16332281640886018,3.3897559267112753,0.015485169942872138,0.035213227685040434,0.026664285274540744,0.0009015612383639311,0.00048707361649531376,0.002661961748943151,3.308342632957995,28.055114362599713,0.3999238983070013,0.019156226183831337,23.40579938455702
|
| 11 |
+
76320,1250426880,0.3938756603747606,3.961635688940684,3.4127625028292337,0.16731366007176499,0.049002804992293626,0.2922468448523432,3.382171114285787,1.8404473333914444,73.69570011889799,37.889468432580166,1.2987957034275164,1.0556336924128682,1.4607050245246438,29.59949236781631,20.977177524721558,0.8173404221213705,0.23706992291376897,0.2500705298972629,0.9791666666666666,0.9674479166666666,0.9628900289535522,1.0,0.5172166228294373,0.6875,0.14583333333333334,0.125,0.9846698355807652,nan,nan,0.9743830202060053,0.178275553696191,0.15371485248518488,3.3403617165342316,0.013003124353554944,0.034689604092889174,0.026229959402518422,0.0008182163021591936,0.0003251544340571906,0.0024732855571837375,3.262822357584949,27.410040938191944,0.4026796761587793,0.018793456452196412,23.14875087805363
|
| 12 |
+
83952,1375469568,0.24895827223857245,3.960906902949015,3.494630495707194,0.007510223172327339,0.019708646465005586,0.6832846122609529,3.3643866777420044,1.3083717594878073,76.09826780436381,40.60807581034913,1.0077951505539913,1.0209453036352876,3.9682658878219677,29.088557187180612,21.871468414770373,0.842403146449315,0.24267896043379014,0.259043561797162,0.9947916666666666,0.9869791666666666,0.9546800851821899,1.0,0.5289404392242432,0.71875,0.14583333333333334,0.11979166666666667,0.9948518580640666,nan,nan,0.9715066891221795,0.19224598777251836,0.16802824802774316,3.300923482867011,0.01072080603917943,0.034290418761328725,0.025868450786199572,0.000769422835161565,0.00022735140825440317,0.00234871509047727,3.226698302289642,26.755533089990134,0.40532874146894216,0.018371334958694578,23.15903405225211
|
| 13 |
+
91584,1500512256,0.17628569047277173,4.005227565765381,3.572668661673864,0.020363162832078768,0.035252836196984084,0.23321652414354807,3.3409101963043213,1.207850758937827,83.83926012443533,46.67652912711284,1.0222046307070254,1.0378877432331912,1.342479379172971,28.398127211692366,23.36061985361308,0.8985781650792295,0.2521428015285018,0.28060632670291236,0.9895833333333334,0.9674479166666666,0.9666085839271545,1.0,0.5502579808235168,0.828125,0.140625,0.14583333333333334,0.9900856389237257,nan,nan,0.9781970415594211,0.17842188799462141,0.1525666571687907,3.27018742128988,0.008641826633357576,0.03402518687708268,0.025595214023346906,0.0007423500790100943,0.0001848085256906777,0.0022789787038266884,3.1987190410354227,26.241257838420648,0.4076139222658718,0.017893076407163026,23.14968602165001
|
| 14 |
+
99216,1625554944,0.31593073935558397,3.9729413191477456,3.580786148707072,0.004322982109594402,0.0317346237085682,0.23512957327329786,3.3286978006362915,1.493069398135793,75.82158337232585,45.81963901736072,1.0044213296129711,1.0334233987546597,1.3959453836304174,28.0641875532457,22.09032420758087,0.8964564544655192,0.23723571098126225,0.2750935085934397,0.9947916666666666,0.9674479166666666,0.9798308362563452,1.0,0.5374149680137634,0.8020833333333334,0.13541666666666666,0.13541666666666666,0.9948518580640666,nan,nan,0.9873756337328814,0.17227307633993405,0.14564132692369944,3.2449916060115807,0.006833422570121634,0.033802052715254376,0.025391503087418044,0.0007220355990356168,0.00016352597286206248,0.00222259181913931,3.1758564590137377,25.871682851382023,0.409185750819977,0.017362323018206012,23.140196365371896
|
| 15 |
+
106848,1750597632,0.2247885960775117,4.088389337062836,3.554368108510971,0.005409284213328647,0.04513578151818365,0.3391149797971593,3.3159721295038858,1.3544796835036905,92.6335443861655,45.534787225989255,1.0055555309139657,1.0503025675004425,1.5554862775923273,27.709034186111158,24.406169979682335,0.9364375930613823,0.2550259093196505,0.2756174791647796,0.9947916666666666,0.98046875,0.9741832415262858,1.0,0.5525332689285278,0.875,0.15104166666666666,0.125,0.9948518580640666,nan,nan,0.9793549359164899,0.16386578868796656,0.13735893537523225,3.224518462817707,0.005697765177400369,0.03362186955882576,0.025208526217573274,0.0007074370281774143,0.0001514522944181209,0.0021876174604452854,3.15694378041236,25.472093797829665,0.4109959418707736,0.016783116791537683,23.12191580787088
|
| 16 |
+
114480,1875640320,0.13302624908586344,4.065868516763051,3.559148440758387,0.0033663892243491014,0.021746017197680583,0.449784498300384,3.2989310026168823,1.150518690042145,86.94460739239594,46.26813653188166,1.003425303968443,1.0227722431573343,1.8133998035549894,27.249973886090277,23.636119121584404,0.958576510651813,0.24318137516587593,0.2902763109248718,0.9947916666666666,0.9869791666666666,0.9551886518796285,1.0,0.5526734590530396,0.90625,0.13541666666666666,0.16145833333333334,0.9948518580640666,nan,nan,0.9698137827751149,0.18009782129890559,0.15512959606712684,3.205479752251541,0.0049501325754817875,0.033448129158794415,0.025037913499426055,0.0006988829013043692,0.00014316290824239704,0.002165624548793333,3.1390358925303072,25.03245550718947,0.4130265253441764,0.016159868723772935,23.12273436835606
|
| 17 |
+
122112,2000683008,0.14952086672807732,4.010356664657593,3.544932633638382,0.004672004600403549,0.03444338079195101,0.33411690976208774,3.286708116531372,1.1983164320393844,80.53580183639282,47.09153772104571,1.0047929380551524,1.039641352309374,1.4917399917908647,26.91035535625291,22.753169375412316,0.9424381404079138,0.2580519673609083,0.29468358056415084,0.9947916666666666,0.98046875,0.9759148756663004,1.0,0.5654476284980774,0.8958333333333334,0.14583333333333334,0.15104166666666666,0.9948518580640666,nan,nan,0.9859749342043264,0.19843585231798974,0.17324852118811881,3.188729376543201,0.0043991612473540265,0.03333153098307167,0.024882987762531582,0.000690919058145266,0.0001371665025069757,0.0021312528755563524,3.1231563442434194,24.749713114035206,0.414211110131858,0.015497325216091833,23.181856193473447
|
| 18 |
+
129744,2125725696,0.09230708548178275,4.065225919087728,3.6482610205809274,0.005488069990519762,0.01215806001406842,0.4030376997161511,3.27351983388265,1.1072810458010538,87.25843517280646,51.88913829269583,1.0056551294551856,1.0125738080373115,2.8469579530107914,26.560929404591164,24.52585297234254,0.9562812333808576,0.2534401239075017,0.28348034632358515,0.9947916666666666,0.9869791666666666,0.9705933531125387,1.0,0.5569820404052734,0.9166666666666666,0.14583333333333334,0.140625,0.9948518580640666,nan,nan,0.9795164182723965,0.16685725304684715,0.14280535137125602,3.173333888809678,0.003970786918356442,0.03319292641059345,0.024752781706122495,0.000683823198974189,0.00013087563963184393,0.002104609747077361,3.1084980729064022,24.37965970462973,0.41584746124890615,0.014800531927597655,23.12570925497775
|
| 19 |
+
137376,2250768384,0.18795065624484172,4.106668869654338,3.5339675545692444,0.0060120452159632505,0.006892792088668405,0.2627881365782135,3.2594571510950723,1.3219288994475586,98.55397711648472,45.35241153541286,1.0062105450483334,1.0070662323889794,1.4363223016637876,26.20052844375786,24.982635010600582,0.9733117436026762,0.2735764013452399,0.29926757586910796,0.9947916666666666,0.9934895833333334,0.9764361182848612,1.0,0.580425500869751,0.9270833333333334,0.16145833333333334,0.16145833333333334,0.9948518580640666,nan,nan,0.9800939004780957,0.18780668856303842,0.16232889638437578,3.1589742345406115,0.0036373101591050745,0.03306842293204046,0.02463855832218206,0.0006786554722723958,0.00012616899237689864,0.0020760361384745065,3.0947490703421274,24.069555843963563,0.41744330254397055,0.014074795349650518,23.150975643189778
|
| 20 |
+
145008,2375811072,0.214533500916635,4.151662786801656,3.6048717002073922,0.003861234740649403,0.006730515255488474,0.30931600640906254,3.254075209299723,1.334532717799129,102.86592272266107,48.60930136393736,1.0039466293273935,1.006853376590817,1.5223897215020628,26.042576574433905,26.055074729464536,0.9467446918660762,0.2578812922237224,0.3088832317803112,0.9947916666666666,0.9934895833333334,0.9709459741910299,1.0,0.5731510519981384,0.8958333333333334,0.14583333333333334,0.16145833333333334,0.9948518580640666,nan,nan,0.9759095376308929,0.18103947696469427,0.1578374153468758,3.1457846196749717,0.0033753720648674367,0.03295396237527063,0.024512513053140086,0.0006737650241766204,0.00012114792845830107,0.002055892060320178,3.0820919547118066,23.869965860616237,0.418224456749341,0.013325642393809753,23.17949956113903
|
| 21 |
+
152640,2500853760,0.16919586046909293,4.134403685728709,3.5241601963837943,0.004234463313537162,0.01541017666204425,0.2758507862448217,3.238128940264384,1.272764119963828,98.81958081236235,45.560494375814166,1.0043249829750884,1.0160014227161718,1.489526060836961,25.63791637444311,24.971515449873092,0.9844133566513292,0.26097255083062354,0.30208794820605267,0.9947916666666666,0.98046875,0.9779502103726069,1.0,0.5735048651695251,0.9583333333333334,0.14583333333333334,0.15625,0.9948518580640666,nan,nan,0.9805951732123503,0.17744287339619783,0.15182640519924462,3.1331839292569854,0.003156043115117774,0.03285098438706655,0.024398929629442987,0.0006696022763245092,0.00011660822326568775,0.00202911627582586,3.0699626336786645,23.601012556870277,0.4195511179587626,0.012558778301146429,23.14867459490053
|
| 22 |
+
160272,2625896448,0.07898191828280687,4.208380202452342,3.658373713493347,0.00296924493323066,0.00737545485208102,0.3801500649942682,3.2253811756769815,1.098847613194244,109.31021915917859,53.38144962200038,1.0030192635592543,1.0075548051033534,1.688148333621626,25.320135483524965,27.54419632574034,0.9761283034884481,0.2733277586906165,0.3037681236794841,0.9947916666666666,0.9934895833333334,0.9712795515855154,1.0,0.5825366973876953,0.9635416666666666,0.16666666666666666,0.15104166666666666,0.9948518580640666,nan,nan,0.9758611320333633,0.19850628067776674,0.17182424066898724,3.121587291123003,0.002970007247302529,0.03274572821471328,0.024295610002172716,0.0006622216478585581,0.00011335768141893847,0.0020093496819219656,3.058791004847618,23.308869106089794,0.421069804090905,0.011780043193471874,23.159222933518663
|
| 23 |
+
167904,2750939136,0.08952013780556929,4.286608378092448,3.8360997835795083,0.004216905579051892,0.009651135950055808,0.4433599933642351,3.2143665154774985,1.1153803030596063,119.82439595560362,62.85004000659283,1.0043189983943732,1.0099057632965522,1.848479605693598,25.04645761326405,30.38556832084352,0.9630347611854108,0.28764863807197233,0.319208097652023,0.9947916666666666,0.9869791666666666,0.9752134333054224,1.0,0.5987284183502197,0.9322916666666666,0.171875,0.16666666666666666,0.9948518580640666,nan,nan,0.9795889819361037,0.1843813249747086,0.15818892500828952,3.1102111220781534,0.0028064520213931794,0.032618936252061755,0.024189192522105656,0.0006597776327400127,0.00011015832530360314,0.0019909241028172376,3.0478356692557314,23.086590164424987,0.4219887262479689,0.010995367597369719,23.164390819696358
|
| 24 |
+
175536,2875981824,0.07375160816203181,4.325997630755107,3.7520280381043754,0.004266417306231081,0.011218103804064109,0.27992235614859357,3.2076464891433716,1.0911024751335867,128.28844376972205,58.064324975027034,1.0043627292172108,1.011544840092724,1.5038667480695729,24.874703091570368,30.834049804118933,0.9849823981856404,0.2740816839920704,0.3142103903603766,0.9895833333333334,0.9869791666666666,0.9832458545764288,1.0,0.588871955871582,0.96875,0.15104166666666666,0.15625,0.9897480490326416,nan,nan,0.9861587729052795,0.18816889397061232,0.16272719282036027,3.100343610763362,0.0026641835561683255,0.03252040877800622,0.02409393131574466,0.0006524583696660965,0.00010741470158552003,0.001976792497385964,3.03832840916631,22.93350054383028,0.42257416950816,0.01021072727974235,23.16065564507812
|
| 25 |
+
183168,3001024512,0.06603088342914513,4.34539798895518,3.7406209806601205,0.004207959851688277,0.008134978011696603,0.36229103254299844,3.1991650660832724,1.0821046061532071,138.06441563546227,58.23659036075559,1.0043122522848957,1.00831614703833,1.7094780909567528,24.677624181404024,32.25469161057929,0.9876111354796481,0.2816758636676528,0.3221767020755621,0.9947916666666666,0.9869791666666666,0.9712223758300146,1.0,0.5975414514541626,0.96875,0.17708333333333334,0.171875,0.9948518580640666,nan,nan,0.9762453532942649,0.19566182704126808,0.17100075775912651,3.090842081802675,0.0025384235333618947,0.03239955493158861,0.02399194099899521,0.0006455389913263557,0.00010478153008541575,0.0019606914568706733,3.0292011381424366,22.685214058970505,0.42412876140177874,0.009432097738825898,23.135193868041007
|
| 26 |
+
190800,3126067200,0.07578229168818022,4.378143906593323,3.8602377275625863,0.0037890021668024096,0.007754687860123492,0.21345641991986972,3.1903634468714395,1.0928684922978762,137.124385275003,68.04089374437724,1.003872230034864,1.0079008384325705,1.4165063125544695,24.45826725001483,33.449242020387835,0.9859296393376081,0.2837262104379959,0.3207656759020354,0.9947916666666666,0.9934895833333334,0.9831110388040543,1.0,0.599043071269989,0.96875,0.171875,0.171875,0.9948518580640666,nan,nan,0.9868531564328199,0.19427675831178703,0.1687327364149193,3.0814888490393897,0.0024301837972367313,0.03230031760359687,0.023898133370246946,0.0006397201252570137,0.00010321922891669994,0.001943301298948624,3.02017396219611,22.538058061144888,0.42483611953536543,0.008665408697250875,23.18809255097607
|
| 27 |
+
198432,3251109888,0.03421553724668532,4.514066656430562,3.8755238354206085,0.002365324384598466,0.008123596192244046,0.2943016510706305,3.1841472387313843,1.036242443296261,170.24285349119083,69.58276267779063,1.0023936643629023,1.0083220003189945,1.5455504527236965,24.304597728545044,38.38896035117548,0.9931939551208275,0.2853702143319831,0.3220400960003242,0.9947916666666666,0.9934895833333334,0.982080896695455,1.0,0.6010487079620361,0.984375,0.171875,0.17708333333333334,0.9948518580640666,nan,nan,0.9859087066627884,0.19559826116811033,0.17152814014116302,3.07264769499813,0.0023347644147604573,0.03220305684425137,0.02381408687116862,0.0006318934062453647,0.0001014232593237249,0.0019308678931828462,3.0116315905751883,22.34730806265737,0.425717166052893,0.007916498943711495,23.1584325997831
|
| 28 |
+
206064,3376152576,0.02441350877052173,4.615103622277577,3.9674087166786194,0.00244885748169447,0.010883276117453988,0.19117923119938496,3.1772990624109902,1.0253166775690974,191.49059612585145,75.00841062689216,1.0024761313714117,1.0112418320743795,1.3568306995908461,24.14119935287717,42.148010206603786,0.9817697112719723,0.2878136815642422,0.3393899089537185,0.9947916666666666,0.9934895833333334,0.9924094527959824,1.0,0.610876739025116,0.96875,0.16145833333333334,0.19791666666666666,0.9948518580640666,nan,nan,0.9935564993259808,0.19361542376503849,0.16923959964575866,3.064836099941692,0.002248666526947701,0.032112216727354495,0.02373912034106879,0.0006258555607168718,9.951566036886777e-05,0.0019099346062665878,3.0041007787968046,22.21360032438482,0.4263819345747525,0.007191071867150021,23.165423357400424
|
| 29 |
+
213696,3501195264,0.02426189805676889,4.628656287988027,3.896370013554891,0.0028825876420199847,0.010180725491788204,0.14931579066812142,3.172246813774109,1.0253783793292623,202.6503984955657,70.61851260925822,1.0029156874814884,1.0104674501249742,1.2337987081438422,24.022307229425703,43.08053979418988,0.9813254905630694,0.28365447245159164,0.3292898428330892,0.9947916666666666,0.9869791666666666,0.9845478187004725,1.0,0.6026478409767151,0.9583333333333334,0.16666666666666666,0.18229166666666666,0.9948518580640666,nan,nan,0.9884946712021095,0.19475025296297113,0.16952079876015583,3.057592091346479,0.002177490129079802,0.03202617015410736,0.023675453586449843,0.0006203392308509521,9.766813190157208e-05,0.0019012321632847374,2.997093725861014,22.088216882831645,0.42707950748362633,0.006494652022090374,23.163430334183666
|
| 30 |
+
221328,3626237952,0.020636389175100096,4.746021012465159,4.008497387170792,0.0024037289716479413,0.008911659333098973,0.20265066952356392,3.167218645413717,1.0213746018522574,239.43034899080848,78.42262987227132,1.002433974397798,1.00914695045832,1.3752820647693278,23.903595985649748,49.45211606288675,0.9861131046635713,0.28645019433159025,0.33715693574010813,0.9947916666666666,0.9869791666666666,0.9845478187004725,1.0,0.6084210276603699,0.9739583333333334,0.17708333333333334,0.1875,0.9948518580640666,nan,nan,0.9885737012179258,0.20251479753576027,0.17643619456794113,3.0506148024805144,0.0021144282681756273,0.03194237103705912,0.023607144686347885,0.0006174630722604945,9.634672111967725e-05,0.001891775814407637,2.9903452612985526,21.994634275541365,0.4276104616762707,0.005832543055900948,23.176673536110314
|
| 31 |
+
228960,3751280640,0.019224588686483912,4.78908638159434,4.06974321603775,0.0026006009620616055,0.007502257287702226,0.23565757395484374,3.1627697149912515,1.0198807394059526,253.40758507591374,95.349710525055,1.002633926956011,1.0076641316169757,1.38534336833453,23.80073467342221,53.85336463438635,0.9931020355782035,0.2827519517163696,0.33423415075605273,0.9947916666666666,0.9934895833333334,0.9812798450390497,1.0,0.6052015423774719,0.984375,0.17708333333333334,0.18229166666666666,0.9948518580640666,nan,nan,0.9833451046870323,0.20281234159417258,0.17837678524665534,3.0446412675525236,0.0020557594065525294,0.031890515365525895,0.0235409475188975,0.000617541786816137,9.478715916899096e-05,0.0018787088571020011,2.98456299577265,21.862975340781723,0.4282440254757614,0.005209787318395601,23.222676649486743
|
| 32 |
+
236592,3876323328,0.019586491502802044,4.862565577030182,4.0564300417900085,0.0029571638696926548,0.00857519128994492,0.15809530398572255,3.159414609273275,1.0202573184272035,283.54534154908475,84.02106437166896,1.0030049749713152,1.008812514880322,1.258456776509229,23.717501263509792,56.510634109864505,0.9821443404689402,0.299570854347502,0.3428721374295054,0.9947916666666666,0.9934895833333334,0.9845478187004725,1.0,0.6194519400596619,0.9739583333333334,0.18229166666666666,0.1875,0.9948518580640666,nan,nan,0.9884946712021095,0.1939171311871822,0.16924684382198998,3.0391758234272004,0.0020114520395488975,0.03181778053549046,0.023486942672583968,0.0006170187921638263,9.355445560137063e-05,0.0018746569824225256,2.9792744050298348,21.79520148919813,0.4286901383334673,0.004631127461369502,23.289736344157966
|
| 33 |
+
244224,4001366016,0.07080468526934662,4.88222062587738,4.043873429298401,0.002391832423265091,0.007809524569950099,0.19610228223458157,3.1568278868993125,1.0927237107353183,294.5326743494343,82.88149070808474,1.0024240506316968,1.0079730297734955,1.3548237740099658,23.661071354761006,57.93331156820436,0.9811060694153082,0.30404658962891346,0.34510087281975227,0.9947916666666666,0.9869791666666666,0.9845478187004725,1.0,0.6227728724479675,0.96875,0.18229166666666666,0.19791666666666666,0.9948518580640666,nan,nan,0.9885737012179258,0.20000818427003728,0.17523636117887995,3.03434983321487,0.0019742188993057833,0.03176053247850904,0.023436534638349996,0.0006200077548421911,9.276867419654675e-05,0.0018771943812237917,2.9745885632869706,21.703995666538894,0.42923558379213017,0.004100970320511953,23.287576841463437
|
| 34 |
+
251856,4126408704,0.07213824931750423,4.911556680997212,4.123012969891231,0.002739217175206458,0.00876943444292048,0.2795776439688022,3.1552723248799643,1.0958147681916837,306.8256847021964,93.47586504511644,1.002781291826995,1.0089819083224811,1.4970456796405098,23.620109360026124,61.21804039361723,0.9814011577570233,0.29531742002598477,0.3503123272829866,0.9947916666666666,0.9869791666666666,0.9812798450390497,1.0,0.6196203827857971,0.9635416666666666,0.17708333333333334,0.19791666666666666,0.9948518580640666,nan,nan,0.9833498688967666,0.20500808324862874,0.17936821398325264,3.030876371770014,0.0019476157276140688,0.03172122117536758,0.02339935609062868,0.0006183710683067032,9.187477051292998e-05,0.0018740005945769725,2.9712239192482435,21.751846970564664,0.4290045278236676,0.003623353354756449,23.31094114639539
|
| 35 |
+
259488,4251451392,0.018901802597004764,4.944851100444794,4.072886894146602,0.0024561478147357243,0.00858862420863223,0.23193569132399716,3.1527026096979776,1.019508312710818,317.3906356600091,87.70608233686742,1.0024884404995307,1.008797723585185,1.4892846593174092,23.562246142264403,61.882720467893414,0.9901755220942529,0.29309541248095555,0.34185489923245377,0.9947916666666666,0.9869791666666666,0.9831110388040543,1.0,0.6149734258651733,0.9791666666666666,0.17708333333333334,0.19270833333333334,0.9948518580640666,nan,nan,0.9869274222389018,0.20425282577080375,0.1780028607851515,3.027670453671577,0.001924609062024,0.031689763462817225,0.023379733209930368,0.0006168541853071489,9.15281826776484e-05,0.001870906360869288,2.9680970469964794,21.66211360030704,0.42938028924182775,0.003201913898652705,23.258547795495577
|
| 36 |
+
267120,4376494080,0.0710845594900699,4.982665538787842,4.1711404124895735,0.0027340904532036823,0.00864427979576495,0.27867099867902806,3.1517926454544067,1.0934233707389551,332.48781290913905,101.69183402400522,1.0027751377088105,1.0088395267125745,1.5404932925273638,23.541220583418085,66.05234269203571,0.9905225663381545,0.2934387987388712,0.3478038079497136,0.9947916666666666,0.9869791666666666,0.9798430651426315,1.0,0.6177406311035156,0.9791666666666666,0.17708333333333334,0.19791666666666666,0.9948518580640666,nan,nan,0.9816988257080084,0.20821044750279802,0.18294116051401943,3.025035077012206,0.0019068669602777296,0.03166604732857349,0.023344812142491216,0.0006168967265322298,9.118786124015525e-05,0.0018685389986084287,2.965540714476031,21.645901425642037,0.42947801668782654,0.0028398614619212603,23.264156914352363
|
| 37 |
+
274752,4501536768,0.02102681970791309,4.947460472583771,4.11104283730189,0.0026766389407271163,0.008854807325320735,0.27482931458352294,3.150643825531006,1.0217438952857187,325.11387750000955,92.78705597496831,1.002716547820742,1.0090817191011723,1.5280658839991832,23.51320262955949,63.71082059296346,0.987392867703603,0.29441217128848257,0.346729420230473,0.9947916666666666,0.9869791666666666,0.9798430651426315,1.0,0.617694616317749,0.9739583333333334,0.171875,0.19791666666666666,0.9948518580640666,nan,nan,0.9816988257080084,0.20250778645013648,0.17691716268503418,3.0230498477025187,0.0018957652753588765,0.0316319651198837,0.023320424187953492,0.0006132233930712181,9.141156529986365e-05,0.0018808883849216328,2.963616157274701,21.60602860499478,0.4295655998403296,0.0025399532871463355,23.25094371049729
|
| 38 |
+
282384,4626579456,0.020386546641627017,5.008203188578288,4.165110786755879,0.0027234798208404243,0.00882823652973741,0.19585580219919999,3.1498684883117676,1.021142818756125,345.3970232016531,103.13543463968047,1.00276449558698,1.0090489104229958,1.3943857363621965,23.49534515886155,68.06502070876049,0.987392867703603,0.29024782785792225,0.3428496927436832,0.9947916666666666,0.9869791666666666,0.9831110388040543,1.0,0.6134734749794006,0.9739583333333334,0.171875,0.1875,0.9948518580640666,nan,nan,0.9868483922230856,0.20521992115188445,0.17941095439406732,3.0214372848444975,0.001883393810413152,0.031628193415829606,0.02331919856057102,0.0006118376572454902,9.153281949230327e-05,0.0018813110504292715,2.962021805392871,21.578132029162013,0.4298376133216377,0.002304473351749283,23.281147053314054
|
| 39 |
+
290016,4751622144,0.020462956848253572,5.0142693519592285,4.161146640777588,0.0024245352386635943,0.00941262989276955,0.2701484608054822,3.149296681086222,1.021168842522511,348.25996692033823,99.07507219012449,1.0024570116135842,1.0096907123600931,1.5118026462364262,23.481254432668333,67.90877325083767,0.9930559755937671,0.29400338867721015,0.35676740684460295,0.9947916666666666,0.9869791666666666,0.9798430651426315,1.0,0.6222078800201416,0.984375,0.171875,0.20833333333333334,0.9948518580640666,nan,nan,0.9816988257080084,0.20987091183915632,0.18439109646715224,3.020210934127339,0.0018785198628124909,0.0316183663596992,0.023296214580834616,0.0006089718857731946,9.100787779809467e-05,0.001873837939992417,2.9608440037652755,21.544569784177924,0.4300504337324599,0.0021352149741550937,23.297967254048736
|
| 40 |
+
297648,4876664832,0.019963260236788,4.987654527028401,4.194835325082143,0.002663495756286475,0.009573440612693199,0.1970450849871289,3.1483309666315713,1.0206515125822637,335.4005477250632,106.46007473945967,1.0027032950453285,1.0098406273922695,1.3992900525986671,23.457989887777376,67.10729969141697,0.9880973523501663,0.30607222954533314,0.34728403143533476,0.9947916666666666,0.9869791666666666,0.9831110388040543,1.0,0.6253158450126648,0.9739583333333334,0.18229166666666666,0.19270833333333334,0.9948518580640666,nan,nan,0.9868483922230856,0.2077646742278413,0.18199308856856078,3.019013991463568,0.0018689840579202799,0.03159839176500618,0.023285424247750805,0.000610540598867988,9.103959063425644e-05,0.0018735521646912725,2.959686046988189,21.588752873101324,0.4299044165002725,0.0020334671566161833,23.23191664088908
|
training_logs/run_metrics_text67/06_fone_large/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: False
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 10
|
| 5 |
+
cache_base_path: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/cache
|
| 6 |
+
combine_strategy: sum
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty_sd
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/configs/config_FoNE_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_stem
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 1280
|
| 45 |
+
n_head: 20
|
| 46 |
+
n_layer: 36
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: fone
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: mse
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/configs/config_FoNE_text67.py --tqdm --verbose --seed 42 --run_name fone_gpt2_large_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 36 --n_head 20 --n_embd 1280 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/trained/fone_gpt2_large --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group fone_gpt2_large_text67 --tie_word_embeddings --tokenizer_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_repo/tokenizers/num_text/bittoken_gpt2 --model rope_stem --num_embedding_type fone --base 10 --dropout 0 --cache_base_path /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/cache --compile --difficulty_column difficulty_sd --train_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Division_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard --train_set_ratios 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized_number_pos --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Division_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos pretokenized_number_pos --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Sat Oct 3 19:43:31 2026'}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: fone_gpt2_large_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/trained/fone_gpt2_large
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_repo/tokenizers/num_text/bittoken_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/cache/bittoken_gpt2_47200109_pos/96841076'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/cache/bittoken_gpt2_47200109_pos/63050008'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/cache/bittoken_gpt2_47200109_pos/44316829'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/cache/bittoken_gpt2_47200109_pos/48579485'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/cache/bittoken_gpt2_47200109_pos/63589654'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/cache/bittoken_gpt2_47200109_pos/28473695'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/cache/bittoken_gpt2_47200109_pos/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized_number_pos
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'pretokenized_number_pos']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-06/native/paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: fone_gpt2_large_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/06_fone_large/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,0.03187350672669709,0.000188978789083194,0.00013615743955597281,0.0007415749132633209,0.8027297258377075,0.0013395915739238262,4.4921739641577005,1.0327165409214991,1.0001889971364624,1.0001361667871094,1.0007418608950007,2.2329668801588944,1.0013406839886279,85.70635892234243,13.282064293175718,0.20505404542348565,0.027778299518913845,0.030411450764127126,0.03645833395421505,0.08854166977107525,0.023717341013252735,0.6346685305041558,0.04480860382318497,0.010416666666666666,0.005208333333333333,0.005208333333333333,0.3574766502715647,nan,nan,0.35290597069615615,nan,nan,5.348822953909698,0.05249967124504458,0.0557026749709133,0.05282246607621307,0.027758241252313742,0.0043437565382534885,0.06136948887826933,5.094326617332874,6085.548618548071,0.26059324880209445,0.004997126321997158,30.17846152043132
|
| 3 |
+
15264,250085376,0.0022523923544213176,0.0002615726552903652,0.00021719911455875263,0.001166448782896623,0.7283380031585693,0.0008371113217435777,3.9278065871136882,1.0022549313716644,1.0002616068718995,1.0002172227175823,1.0011672550460688,2.0716444362241084,1.000837466711074,50.044159984570996,8.16007755764477,0.41200470956723817,0.04525616078698301,0.03868292602972416,0.1093750037252903,0.0625,0.09047092497348785,0.9096226289621309,0.07984460890293121,0.234375,0.010416666666666666,0.0,0.4678522269023233,nan,nan,0.45176136746886186,nan,nan,4.655336522378136,0.03552104572542705,0.044454468777344665,0.042187108735620295,0.020032422594624556,0.0039626153707819104,0.04282598695105168,4.466352842545109,55.838781794786954,0.3604957317148552,0.009998084214664627,26.998289053367262
|
| 4 |
+
22896,375128064,0.044465868384577334,0.00016391747340094298,0.00017115088849095628,0.00032040449150372297,0.7211067378520966,0.00029302763869054615,3.763714769971557,1.046404474208011,1.000163931003132,1.000171165537111,1.000320455842537,2.0567392858626254,1.000293070581019,42.46746335833593,7.081650820195766,0.5290649952051019,0.05498720240174537,0.050246023392310116,0.1302083320915699,0.08203125,0.09344711527228355,0.9755675571581489,0.09779949486255646,0.4010416666666667,0.010416666666666666,0.010416666666666666,0.4522954105680886,nan,nan,0.432937858189689,nan,nan,4.314178397466384,0.026048927099227896,0.03918493484930479,0.03723369579997167,0.0161533042735057,0.0038213700694977324,0.03330737199306626,4.158428766281141,39.93669179325584,0.38578171641478,0.014999042107332095,26.685323973651975
|
| 5 |
+
30528,500170752,0.034440320247085765,1.1425659522501519e-05,1.4154728432913544e-05,8.247273945016786e-05,0.7146552801132202,0.00032832392025738955,3.6621024628014616,1.0356329395700876,1.0000114257250252,1.0000141548286172,1.000082476140469,2.043483315203453,1.000328389637522,38.47058261181115,6.507162187559475,0.5282987839433587,0.05275308258289042,0.043704005996615586,0.1197916679084301,0.1080729104578495,0.0919063426554203,0.989121286769338,0.09559866786003113,0.3854166666666667,0.010416666666666666,0.005208333333333333,0.45615627749551396,nan,nan,0.4465834822525115,nan,nan,4.113220955910985,0.020953966038652662,0.03644608937968272,0.034646373306326875,0.01408087475625217,0.0037492615971986385,0.02821960193203275,3.9751247635161975,35.27091350351739,0.3966470706848666,0.020000000000000004,26.667398086670797
|
| 6 |
+
38160,625213440,0.00169271495542489,1.4275696571530716e-06,8.719732704776106e-07,2.8811235779357958e-06,0.7217174470424652,3.924183624803845e-05,3.5641480397510654,1.001695079639622,1.000001427570679,1.0000008719736555,1.0000028811296302,2.0580842253432547,1.0000392431628877,34.934821612202306,5.999235048717433,0.7612484477587582,0.06961301215782514,0.0664453538334959,0.1510416641831398,0.08854166977107525,0.12601400911808014,0.9938657362868036,0.12155874073505402,0.7239583333333334,0.005208333333333333,0.0,0.5037038640196746,nan,nan,0.4887068899309573,nan,nan,3.974284551138023,0.017569824779712237,0.034604008031903576,0.03292212769568779,0.012727236745252314,0.0037029915514638343,0.024903971557866635,3.8478543669172804,32.156761484888364,0.40513281734656365,0.019965729849129636,26.688940309535884
|
| 7 |
+
45792,750256128,0.0005865100592927774,4.249032372172223e-06,2.2511634369948297e-06,2.002710630222282e-06,0.7246067523956299,8.473335810776916e-05,3.5070859663731726,1.0005868369747677,1.0000042490414838,1.0000022511659712,1.0000020027127248,2.0639197191905803,1.0000847401674453,32.99371351614807,5.722616187914435,0.7661010701937319,0.07689796659434874,0.07644145575505329,0.1875,0.0,0.13046513125300407,0.9979296334156468,0.04950783774256706,0.7708333333333334,0.026041666666666668,0.005208333333333333,0.5379557889827993,nan,nan,0.503468988094634,nan,nan,3.868584510079532,0.015149038300275676,0.0332670813171905,0.03166150057539969,0.01175979071081249,0.003671111296616679,0.02256168207579314,3.7505142834508733,29.698904375485153,0.4125310488248271,0.019863180383909898,26.69591485259777
|
| 8 |
+
53424,875298816,0.03003127196461719,5.632219881590572e-06,1.2494221664383076e-05,1.351770561086596e-05,0.7037492990493774,2.623927139211446e-05,3.46139296037533,1.0309501755524066,1.0000056322358657,1.0000124943020903,1.0000135177978486,2.0213180485386726,1.0000262396838364,31.527168075703983,5.511356311973529,0.7741438108944265,0.08974982618399108,0.08226573000360148,0.328125,0.1536458395421505,0.21936894208192825,1.0,0.17630383372306824,0.7552083333333334,0.041666666666666664,0.005208333333333333,0.6217437055505192,nan,nan,0.5649708263226785,nan,nan,3.5248018924243216,0.006837341063905461,0.028332018536846804,0.02701210710722797,0.008093561409918265,0.003530689398453042,0.013961506133734274,3.4370346502703564,28.132686112911696,0.41787961822401426,0.019693132578928373,26.729232133670628
|
| 9 |
+
61056,1000341504,0.00010873229439312126,1.0714690290569706e-06,5.186494576037148e-07,7.5470172760105925e-06,0.6148106753826141,9.782170309335925e-05,3.434809714866181,1.0001087396451411,1.000001071469635,1.0000005186495948,1.0000075470462972,1.8496857594173208,1.0000978292777165,30.687574302127995,5.362496538233386,0.7734930159398122,0.09080102220390623,0.08133136804441446,0.4947916716337204,0.3033854104578495,0.29339179396629333,1.0,0.20392166078090668,0.78125,0.036458333333333336,0.005208333333333333,0.6968867957087544,nan,nan,0.6146445031336043,nan,nan,3.405697262446981,0.004235902422506261,0.027185235427468653,0.025948826221740728,0.006825807973484843,0.0034718185157699287,0.011393384555637659,3.3266362699879832,27.0475788605663,0.4219759317618684,0.01945688144839207,26.79994173735513
|
| 10 |
+
68688,1125384192,6.21524968664744e-05,1.1526206549206108e-06,8.640042779006762e-07,1.7599571947357617e-05,0.5809676945209503,3.464110068307491e-05,3.3999502148944885,1.0000621549782898,1.0000011526213497,1.0000008640046514,1.0000175997268361,1.7877829585282068,1.0000346418140018,29.65869996946103,5.206657048733481,0.8117593989664125,0.09102448612262684,0.09605878767548551,0.5677083134651184,0.3619791604578495,0.3541930019855499,1.0,0.22598421573638916,0.8177083333333334,0.041666666666666664,0.015625,0.7314958030595639,nan,nan,0.6545723944048708,nan,nan,3.335846076320511,0.003479314294224044,0.026730581707611372,0.02552944929277164,0.00606674337569046,0.003365043506524877,0.010301380725112545,3.2603735462258028,26.134693101267146,0.4255495244672276,0.019156226183831337,26.701094417297575
|
| 11 |
+
76320,1250426880,0.00014634101171395741,2.2090924858275685e-06,2.671467427717289e-06,1.4866678270664124e-06,0.5416712462902069,2.805247549986234e-05,3.3756602113717236,1.000146354621248,1.0000022090949283,1.0000026714709975,1.0000014866689462,1.7189425873448263,1.0000280529136873,28.949007828788314,5.095447312986136,0.7870776858624378,0.0956245099898271,0.08837027202317077,0.6666666567325592,0.388020820915699,0.4339970499277115,1.0,0.23007047176361084,0.765625,0.036458333333333336,0.015625,0.7711117671569809,nan,nan,0.6785643853945658,nan,nan,3.2820033534273123,0.0029281362244740314,0.026302263855359467,0.02513452279130889,0.00530436161764401,0.003229276303813562,0.009288349414007071,3.209816426658322,25.4716403684901,0.42808544706451696,0.018793456452196412,26.680213200690837
|
| 12 |
+
83952,1375469568,2.4936909539974295e-05,1.2242599041201174e-06,5.632815600620233e-07,1.2794462094234405e-06,0.4505985826253891,4.7274967982957605e-05,3.3553466964125014,1.000024937347083,1.0000012242606557,1.0000005632817186,1.0000012794473836,1.569455829338199,1.0000472764554775,28.382503620578746,4.993147818672752,0.8445396900087505,0.10371133022785028,0.08771811606412787,0.75,0.4921875,0.4828803092241287,1.0,0.24422267079353333,0.84375,0.041666666666666664,0.010416666666666666,0.8190406034894598,nan,nan,0.713866473097975,nan,nan,3.2393844601009003,0.002647314418927833,0.02603534781489558,0.024890839387119276,0.004554098865038162,0.003069065518939881,0.008365286846660554,3.1698224908827815,24.86858860233045,0.43070808313084097,0.018371334958694578,26.625590277870504
|
| 13 |
+
91584,1500512256,8.251628514699405e-06,3.9584783735335805e-06,3.5938129485657555e-06,1.8951895526697626e-05,0.42282772064208984,2.883767865569098e-05,3.3314216711830036,1.0000082516625806,1.0000039584862912,1.0000035938195193,1.000018952224715,1.5265000605875882,1.0000288382189102,27.73641453552811,4.894711170075388,0.81942255566789,0.0989568867952776,0.08565585279663906,0.7395833432674408,0.55078125,0.5719026327133179,1.0,0.2420748770236969,0.8177083333333334,0.041666666666666664,0.015625,0.8348698598953584,nan,nan,0.755720482673496,nan,nan,3.2070588391250783,0.002456155536296834,0.025831275874199533,0.024715402032018285,0.0038041570554616356,0.002876327990531983,0.007443007689876779,3.139932496604659,24.457618485831613,0.43228862113251,0.017893076407163026,26.64527370330189
|
| 14 |
+
99216,1625554944,6.46130592940608e-05,2.390668100815674e-06,1.2916775062876695e-06,7.408511422113406e-07,0.08725886046886444,3.474279674264835e-05,3.3148811348946765,1.0000646153685366,1.000002390670999,1.0000012916783407,1.000000740851557,1.0914213535921835,1.0000347436249966,27.27014836902229,4.765953357829844,0.8462551269122719,0.09836902186842408,0.08435411515868177,0.8072916567325592,0.908854179084301,0.6593710482120514,1.0,0.24948446452617645,0.8802083333333334,0.036458333333333336,0.010416666666666666,0.8533586207195185,nan,nan,0.7882836247251058,nan,nan,3.180509005921192,0.002289600668166584,0.025696366233084085,0.02459392263677692,0.0031990674732621156,0.002554829985655876,0.006597006879311678,3.115578195742695,23.999472794417816,0.4346522007574888,0.017362323018206012,26.668327378612588
|
| 15 |
+
106848,1750597632,6.506347631329845e-06,1.478457534176414e-06,7.771524508370931e-07,4.296462367392451e-05,0.036575768142938614,3.840627323370427e-05,3.302555831226831,1.0000065063733117,1.0000014784586457,1.0000007771527546,1.0000429664635653,1.0375055048905923,1.000038407025915,26.95517682185854,4.713253208889047,0.8270324691435782,0.11453007513820201,0.09522678510124041,0.828125,0.973958320915699,0.6667456924915314,1.0,0.2780444920063019,0.8489583333333334,0.046875,0.015625,0.8568322412514439,nan,nan,0.8067574099404737,nan,nan,3.157534934814621,0.002058359496756924,0.025504983068332546,0.024418070775363662,0.002735476157654887,0.0020713314001740424,0.005834228806906224,3.0949124694208967,23.59693242987497,0.4368275048824289,0.016783116791537683,26.62917775779463
|
| 16 |
+
114480,1875640320,3.370883496245369e-05,4.1667206005513435e-06,1.9007792388947564e-06,5.281967162318324e-06,0.01207217713817954,5.954557082077372e-06,3.2813871664111502,1.0000337094296454,1.0000041667292823,1.0000019007810503,1.0000052819906995,1.012145829193945,1.0000059545755127,26.383990495816306,4.628026762645206,0.8417508647299202,0.1108917190631898,0.09074978717609608,0.890625,0.993489570915699,0.731879472732544,1.0,0.2721530497074127,0.8697916666666666,0.057291666666666664,0.010416666666666666,0.8887267912117144,nan,nan,0.8326189382156978,nan,nan,3.137333665956866,0.001875344665676132,0.025371399220628473,0.02428872576181905,0.002379371785146059,0.0016231328737293456,0.00519469495896626,3.076600981630244,23.171982576824583,0.4388705152799561,0.016159868723772935,26.654766246319998
|
| 17 |
+
122112,2000683008,1.646257112497551e-05,1.9307282741465315e-06,2.2275718265518663e-06,1.2300172329560155e-06,0.004045222187414765,1.4335295190903707e-05,3.260779373832823,1.0000164628050934,1.0000019307301562,1.0000022275744795,1.0000012300181949,1.0040550595017588,1.0000143354608397,25.867583061393194,4.553096329640531,0.8669583547520325,0.0982958300261304,0.09065320217133896,0.9322916567325592,1.0,0.7896518111228943,1.0,0.2622028589248657,0.9166666666666666,0.041666666666666664,0.010416666666666666,0.9086809562286362,nan,nan,0.8544266539004942,nan,nan,3.118821169754099,0.0017075361165317592,0.025258627293402346,0.02419195526107367,0.002086377567680835,0.0011982738075599695,0.004637055358814526,3.059741329144007,22.81427162323358,0.44025366904850527,0.015497325216091833,26.675654832312528
|
| 18 |
+
129744,2125725696,9.62048352448619e-06,8.823000428037631e-07,8.203189167943492e-07,1.4541371456289198e-05,0.04443851299583912,1.2802593801097828e-05,3.2507145011913963,1.0000096205399256,1.0000008823004325,1.0000008203192594,1.0000145415104187,1.0454580443431463,1.000012802704504,25.60345972468808,4.52127949091511,0.8480102469361017,0.10832551999156387,0.10975000823943289,0.8802083134651184,0.954427070915699,0.8088785409927368,1.0,0.2923682928085327,0.8854166666666666,0.0625,0.03125,0.8909822880523279,nan,nan,0.8576350957155228,nan,nan,3.102419380702542,0.0015357168254944336,0.02514363935757116,0.02407969529995959,0.001850603535779566,0.0007946175410041718,0.004166077400415859,3.044849015938952,22.558318371542846,0.4415293390271049,0.014800531927597655,26.654698899858026
|
| 19 |
+
137376,2250768384,1.2361493077150953e-05,1.4646667239048838e-06,8.064260441642546e-07,3.832495849565021e-07,0.020829382818192244,5.595568609351176e-06,3.2277460974873975,1.000012361613194,1.0000014646679727,1.000000806426372,1.0000003832496813,1.021074076318636,1.0000055955844456,25.038940974874198,4.437147951819214,0.877190067993862,0.11629108663912638,0.10015536423508335,0.9375,0.98046875,0.8414981067180634,1.0,0.2916842997074127,0.9114583333333334,0.052083333333333336,0.005208333333333333,0.9099794234498404,nan,nan,0.8739681070825706,nan,nan,3.0862980928482875,0.0014437340091453727,0.025085539797110746,0.024022021822098987,0.0016639492177982025,0.00042331588194786794,0.003795872277639874,3.0298636441570035,22.218139309365796,0.4435180175026383,0.014074795349650518,26.599600160679564
|
| 20 |
+
145008,2375811072,1.5409107618324924e-05,3.2502979365744977e-06,1.637958575884113e-06,1.8633487854913255e-06,0.005077364156022668,5.965855598333292e-05,3.2136483602419808,1.000015409240505,1.0000032503036627,1.0000016379599943,1.0000018633513181,1.0050935184863097,1.0000596614275379,24.71777219991373,4.388992505811865,0.8809517764920902,0.10864521460808232,0.12092789683801357,0.9322916865348816,1.0,0.8236277997493744,1.0,0.30573609471321106,0.9427083333333334,0.052083333333333336,0.041666666666666664,0.918502386736994,nan,nan,0.8680858187532673,nan,nan,3.071435971301411,0.0013628944244729279,0.02499609819260653,0.023941632319218333,0.0014961701985116886,0.00017966958953857147,0.003466294953424217,3.0159931946142335,21.947314638324254,0.44441032891925153,0.013325642393809753,26.66305917535996
|
| 21 |
+
152640,2500853760,3.695640657497279e-06,1.3592728009825805e-06,1.1339965908518934e-06,2.1710761188842298e-07,0.003482472035102546,1.7150202893390087e-06,3.203421796807864,1.0000036956476617,1.0000013592737513,1.0000011339972694,1.0000002171076354,1.0034933036147833,1.0000017150219467,24.461112773328917,4.352087742570281,0.8775151288225607,0.1164805718197219,0.10140665341575428,0.9479166865348816,1.0,0.8859565854072571,1.0,0.29449307918548584,0.9375,0.046875,0.015625,0.9200630016081656,nan,nan,0.8966200451832265,nan,nan,3.057157082302245,0.0012932218626483798,0.024921829353238285,0.0238648059488789,0.0013456039612797443,0.0001418560325807161,0.003166230687781315,3.0024235179084195,21.655982076559425,0.44610770295454144,0.012558778301146429,26.60491896196343
|
| 22 |
+
160272,2625896448,4.1197322389052715e-06,1.455331641864177e-06,6.285049778398388e-07,1.6367077932954999e-06,0.0037926784716546535,6.490288342320127e-06,3.188662677188404,1.0000041197423126,1.0000014553327112,1.0000006285051768,1.0000016367091407,1.0038041188650106,1.0000064903102501,24.140472723317874,4.306327310397497,0.8759659065281733,0.11107048430927921,0.11938074349519065,0.9791666567325592,0.993489570915699,0.90071901679039,1.0,0.3089211583137512,0.9375,0.0625,0.041666666666666664,0.9350424278527498,nan,nan,0.9011320876112828,nan,nan,3.044159120038558,0.0012313777175117135,0.024847970998281083,0.023798417518742558,0.0012154223381782452,0.00012260992257317414,0.0029028850608499366,2.9900404190258034,21.40797268738287,0.447265773585976,0.011780043193471874,26.620425539650363
|
| 23 |
+
167904,2750939136,6.5040442223107675e-06,3.2981093909256742e-06,1.0491053501482384e-06,2.7934222543990472e-06,0.0009477186540607363,1.9180779645466828e-05,3.1798939143967195,1.0000065040653912,1.0000032981149394,1.0000010491059372,1.0000027934271984,1.0009482545579949,1.0000191810406815,23.9255471081278,4.275218312634278,0.8877839728488129,0.11914362778823363,0.11257685391974623,0.96875,1.0,0.914046049118042,1.0,0.31071412563323975,0.9583333333333334,0.057291666666666664,0.020833333333333332,0.932690073736012,nan,nan,0.9097012325655669,nan,nan,3.031604227259141,0.0011646146232244133,0.024763656846832703,0.02371154353299174,0.0010952853620861886,0.00010918680687180747,0.0026673659924312584,2.978092557086156,21.152848099012793,0.44870682912380816,0.010995367597369719,26.604225076613602
|
| 24 |
+
175536,2875981824,2.779091346383211e-06,1.5541176594524586e-06,1.5191381521617586e-06,4.129057202817421e-07,0.0013545864494517446,1.789644954897085e-06,3.1679066269037626,1.0000027790957613,1.0000015541188672,1.0000015191396703,1.000000412905806,1.0013558061008307,1.000001789646737,23.642758767178893,4.234874661169509,0.9001079951413022,0.11842299377979978,0.12186446510501009,0.96875,1.0,0.9391724765300751,1.0,0.32024672627449036,0.984375,0.052083333333333336,0.036458333333333336,0.9254075314189928,nan,nan,0.9173105029234042,nan,nan,3.0191079138938908,0.0010973428500846672,0.024665749433891656,0.023615098985466536,0.000988128022549162,0.00010059508498957567,0.002447167710673506,2.9661938146558526,20.908306709625435,0.45006767694352184,0.01021072727974235,26.677974034268562
|
| 25 |
+
183168,3001024512,4.582688006848912e-06,2.272635924782662e-06,1.6087581684587349e-06,8.060290213052212e-07,0.0009752878395374864,4.004312813776778e-06,3.1563791138275215,1.0000045826985664,1.000002272638738,1.0000016087596573,1.0000008060293566,1.0009758712007768,1.0000040043208331,23.371991382103626,4.196140075393079,0.8998870968970498,0.12647649969272468,0.1167714862963391,0.96875,1.0,0.9287953078746796,1.0,0.3227217197418213,0.9895833333333334,0.06770833333333333,0.03125,0.9295633141882718,nan,nan,0.9111936714034528,nan,nan,3.0077908374707567,0.0010176901371446111,0.024551302067441494,0.023493938677625992,0.0008856004814887152,9.265192743295893e-05,0.0022276832980441734,2.955521954258658,20.749531247674543,0.4507164395994173,0.009432097738825898,26.70877315331591
|
| 26 |
+
190800,3126067200,5.477061222336488e-06,2.78544541743031e-06,2.653128603924415e-06,2.727991898154869e-06,0.0008645834168419242,6.009819344399148e-06,3.1473864298895933,1.000005477078144,1.0000027854493077,1.0000026531332327,1.0000027279967978,1.0008650906970176,1.0000060098374437,23.191138736935883,4.170289068732546,0.9038150112701229,0.13056655960039545,0.12981012128340566,0.9895833432674408,1.0,0.9451248645782471,1.0,0.3408399522304535,0.9791666666666666,0.06770833333333333,0.036458333333333336,0.9399729962460697,nan,nan,0.9203242475632578,nan,nan,2.9973252246001443,0.0009684365694662885,0.024470243284903118,0.023407153905060853,0.0008046340550757826,8.663717297847073e-05,0.0020511733024462415,2.945536930574811,20.596623451954663,0.4516597698194676,0.008665408697250875,26.627528403453194
|
| 27 |
+
198432,3251109888,5.072653721072129e-06,3.4270670994374086e-06,2.3333552690019133e-06,2.4092602188829915e-06,0.0005219889135332778,3.68281132523407e-06,3.1382494320084033,1.0000050726686145,1.0000034270730143,1.0000023333581831,1.0000024092641537,1.0005221606336168,1.000003682819115,22.98556668647194,4.140872253184091,0.8994759364411807,0.12975783124141985,0.12685995494429209,0.9791666865348816,1.0,0.9524862766265869,1.0,0.33698737621307373,0.96875,0.06770833333333333,0.036458333333333336,0.9351075214411443,nan,nan,0.9220587436575443,nan,nan,2.98776761046985,0.0009134343563981759,0.024361464463904563,0.023298495098252756,0.0007310863593807145,8.212021977052995e-05,0.0018924911489608643,2.936488502610092,20.43113415102539,0.45266852449368006,0.007916498943711495,26.668188052413747
|
| 28 |
+
206064,3376152576,2.422228476461896e-06,2.8373852956065093e-06,4.21256805793746e-06,2.817708519842199e-06,0.0003055910929106176,2.9259732627906487e-06,3.132396587228868,1.0000024222314337,1.0000028373895762,1.0000042125810473,1.0000028177151306,1.0003056386767801,1.0000029259777592,22.830018851380743,4.11861995799321,0.9090148015458894,0.1287021710610959,0.12781827207793378,0.984375,1.0,0.9540402591228485,1.0,0.3372255265712738,0.9947916666666666,0.06770833333333333,0.026041666666666668,0.9359643224743195,nan,nan,0.9213959488552064,nan,nan,2.978243766791186,0.0008564481854518486,0.024254225950462286,0.023175395346376176,0.0006641232116358893,7.830257533177156e-05,0.0017489215902063275,2.9274663341099525,20.23778587817146,0.4537920006267742,0.007191071867150021,26.669218485123785
|
| 29 |
+
213696,3501195264,1.8691765149014827e-06,3.882662440446438e-06,1.0032189265984925e-05,1.971186009086523e-06,0.004007571609690785,3.3647281156845565e-06,3.1271089019913534,1.000001869178568,1.0000038826713455,1.000010032273705,1.0000019711883101,1.0040225666323392,1.0000033647352855,22.731215370083948,4.105037008109072,0.9095295399182907,0.12713142092091273,0.12126743871148923,0.9947916567325592,0.993489570915699,0.9658396542072296,1.0,0.32927635312080383,0.9947916666666666,0.0625,0.03125,0.9418765053269453,nan,nan,0.9316518482907364,nan,nan,2.969929652675143,0.0008203939411759566,0.02416363731205625,0.023082584168880416,0.000613315154005936,7.524459637759865e-05,0.0016226113360149374,2.919551850502867,20.119830509151804,0.4542526308041011,0.006494652022090374,26.66636785607069
|
| 30 |
+
221328,3626237952,1.4098611131885264e-06,2.5472703555351472e-06,4.0058824311017815e-06,6.312381088946495e-07,0.0026676767738536,1.3784972452413058e-06,3.1187627725303173,1.0000014098624357,1.0000025472740566,1.0000040058938537,1.0000006312383825,1.0026712386231291,1.0000013784983053,22.555317927139082,4.0797141626470355,0.9033582065926096,0.12949420280319524,0.118214343272879,0.9947916567325592,1.0,0.9762431681156158,1.0,0.3283592462539673,0.984375,0.0625,0.020833333333333332,0.9441060259317359,nan,nan,0.9316526284286132,nan,nan,2.962730277626457,0.0007867667790233074,0.02408332709679582,0.022988918428240926,0.000567474639892089,7.278681413504504e-05,0.0015120042213663638,2.912718984365443,19.9992873064502,0.45494863945811176,0.005832543055900948,26.66216123905836
|
| 31 |
+
228960,3751280640,2.7629206442725263e-06,5.90877675676893e-06,6.984319611547107e-06,1.8890256114900694e-06,0.0012566751902340911,3.2631069188937545e-06,3.113598438212648,1.0000027629245853,1.0000059087959636,1.0000069843525008,1.0000018890273958,1.001258170146186,1.0000032631124032,22.440052502855465,4.063047354459214,0.9047794561821173,0.1333759301376747,0.13582362135805784,0.9895833432674408,1.0,0.973280131816864,1.0,0.3499419689178467,0.9895833333333334,0.0625,0.041666666666666664,0.9392188863130286,nan,nan,0.9307735367522886,nan,nan,2.9562518996096085,0.0007570082653619923,0.023994355305969443,0.022896359340274078,0.00052765446629225,7.092482420239072e-05,0.001419991387614354,2.906585590675692,19.942878912267446,0.45529371761254167,0.005209787318395601,26.66232695776383
|
| 32 |
+
236592,3876323328,3.991698235950025e-06,4.924373797621229e-06,4.943427484249696e-06,4.709822292170429e-06,0.0007523291860707104,3.833065079561493e-06,3.110101210710127,1.0000039917069574,1.0000049243864202,1.0000049434409222,1.0000047098355234,1.0007527598490544,1.000003833072848,22.369203890471905,4.052854150394805,0.908556933796006,0.13316014414946786,0.1460624271987645,0.9895833134651184,1.0,0.9688158333301544,1.0,0.35898348689079285,0.9895833333333334,0.06770833333333333,0.052083333333333336,0.9390653570299037,nan,nan,0.9294742565446844,nan,nan,2.950285760036284,0.0007296726582631963,0.02390335836600067,0.02279931898921947,0.0004950958902890281,6.960353846952645e-05,0.0013435382696625716,2.900945156927212,19.863320534084327,0.4555874130370357,0.004631127461369502,26.671581846193643
|
| 33 |
+
244224,4001366016,2.695712964850827e-06,4.745805767925049e-06,5.012382075619826e-06,1.520975672519853e-06,0.006945899658603594,3.1340999839812866e-06,3.106132364482619,1.000002695716919,1.000004745820763,1.0000050123971345,1.0000015209768303,1.006990078843125,1.0000031341053746,22.28475154747227,4.041679819333202,0.9043498318567011,0.1359163793372386,0.13740404927751762,0.9947916567325592,0.993489570915699,0.9777312576770782,1.0,0.3539715111255646,0.9895833333333334,0.0625,0.041666666666666664,0.9441060259317359,nan,nan,0.933705319572861,nan,nan,2.9452629040439877,0.0007078477556917887,0.023834357205888997,0.022721736381125877,0.0004715931543793945,6.806218996282894e-05,0.0012778380854025855,2.8961814536832162,19.808791819171084,0.4560794442915504,0.004100970320511953,26.655792596790718
|
| 34 |
+
251856,4126408704,3.963973995269043e-06,7.274148515534762e-06,6.248221779969754e-06,2.4170811343537935e-06,0.0010367859613324981,2.7688278123605414e-06,3.104488495620899,1.0000039639834948,1.0000072741812769,1.0000062482432008,1.000002417084652,1.0010377639775436,1.0000027688318531,22.243720869876267,4.03496875802547,0.9043498318567011,0.1412035694412701,0.13869916861197518,0.9895833432674408,1.0,0.9703170955181122,1.0,0.3601062595844269,0.9895833333333334,0.078125,0.046875,0.9410303758146862,nan,nan,0.9303473456917951,nan,nan,2.94135504919882,0.0006920615355443308,0.02376198986721577,0.022653477820684935,0.00045274287915631904,6.702344788856222e-05,0.0012278517794776908,2.892499886080883,19.75302235666561,0.4564670673307945,0.003623353354756449,26.65565304750827
|
| 35 |
+
259488,4251451392,2.4138891490110836e-06,6.336734145406808e-06,5.246884256848716e-06,1.5459609699064458e-06,0.0005339328054105863,1.9253666323493235e-06,3.103261740218538,1.0000024138923407,1.000006336761896,1.0000052468988603,1.000001545962201,1.0005341831931385,1.000001925368486,22.21863939919325,4.03131300732431,0.9095295399182907,0.13851091093073575,0.1407595846955491,0.9947916567325592,1.0,0.9747682213783264,1.0,0.35981300473213196,0.9947916666666666,0.07291666666666667,0.041666666666666664,0.9438026749218503,nan,nan,0.9321589400836577,nan,nan,2.937852641764135,0.000676700562119573,0.02369729919314507,0.022580039314372476,0.00043715181854848587,6.607194828156255e-05,0.0011868966940372894,2.8892084667361506,19.700308688281716,0.45679498532595625,0.003201913898652705,26.721261211515838
|
| 36 |
+
267120,4376494080,3.502152367218514e-06,8.969139116743463e-06,6.634553301410051e-06,3.139100385851634e-06,0.0015574815661238972,4.382845190775697e-06,3.1018269226769917,1.0000035021586493,1.0000089691911769,1.0000066345783045,1.000003139105568,1.0015597983424591,1.0000043828548115,22.188519327802073,4.027157964861863,0.9039297108865192,0.14426711813955623,0.14015465290865267,0.9947916567325592,1.0,0.9747682213783264,1.0,0.3645080626010895,0.9895833333333334,0.06770833333333333,0.041666666666666664,0.9438280593603849,nan,nan,0.932390257017687,nan,nan,2.9344999648937806,0.0006657865731420883,0.023638886657698373,0.02251808516507751,0.00042550364739449715,6.543355878283367e-05,0.0011569993544843834,2.8860292535933847,19.62789918985257,0.45723576713314346,0.0028398614619212603,26.717882961133864
|
| 37 |
+
274752,4501536768,1.5626748108843458e-06,6.413127721316414e-06,4.4750825054507e-06,1.2488948186728521e-06,0.001912916744913673,1.831228757964709e-06,3.101355591924706,1.0000015626761414,1.0000064131563977,1.0000044750934087,1.000001248895733,1.0019163612379658,1.0000018312305465,22.1762293606904,4.025451607568656,0.9043498318567011,0.14418606742554285,0.15384886349555538,0.9895833432674408,0.993489570915699,0.9688421785831451,1.0,0.3765389323234558,0.9895833333333334,0.078125,0.0625,0.9410049913761517,nan,nan,0.9296425934104869,nan,nan,2.9319381458043097,0.0006564000927788382,0.023594588416909694,0.022460763877408044,0.0004178777463191514,6.453143805435659e-05,0.0011374717042738703,2.8836064965527304,19.638280746884316,0.4571669265799677,0.0025399532871463355,26.61839561993051
|
| 38 |
+
282384,4626579456,3.025069759132748e-06,7.711630814810633e-06,5.611332881017006e-06,2.9621315320582653e-06,0.0007584971644973848,2.947154598587076e-06,3.1000790342028877,1.0000030250743355,1.0000077116684067,1.0000056113498286,1.0000029621366253,1.0007590247661204,1.0000029471591971,22.15005318309044,4.0215477807492785,0.9043498318567011,0.14361698400484185,0.14926448822339772,0.9947916567325592,1.0,0.9762694835662842,1.0,0.3723841607570648,0.9895833333333334,0.078125,0.046875,0.9438280593603849,nan,nan,0.9328057609964162,nan,nan,2.929904185685867,0.0006489460119937793,0.023553929381397283,0.022419398857080813,0.0004098215499036225,6.426168821260342e-05,0.0011194905061633024,2.881688321762927,19.63588049632948,0.4571607588620493,0.002304473351749283,26.673322359741977
|
| 39 |
+
290016,4751622144,2.1337690441214363e-06,9.28062809180119e-06,5.653173616337881e-06,1.8294744563718268e-06,0.0005694631399819627,2.427211143185559e-06,3.0992553245353824,1.0000021337713316,1.0000092806903313,1.000005653192532,1.0000018294762376,1.0005697186287401,1.0000024272142674,22.132751062340798,4.019048872187748,0.9043498318567011,0.14056507106597946,0.15158740994359102,0.9895833432674408,1.0,0.9733196198940277,1.0,0.3712601661682129,0.9895833333333334,0.078125,0.057291666666666664,0.9410303758146862,nan,nan,0.9300003187575688,nan,nan,2.92861472387036,0.0006428756882911021,0.023516396018364066,0.02238112301803256,0.0004020667832583789,6.436279021733974e-05,0.0011079308064496343,2.8804999530018627,19.665745627330285,0.45704572764176493,0.0021352149741550937,26.684978243636664
|
| 40 |
+
297648,4876664832,2.6850617587115266e-06,8.303265303766239e-06,4.730275009023899e-06,2.155556387606339e-06,0.0015426845311594661,2.216489349393669e-06,3.0986975424263314,1.0000026850654087,1.0000083033146134,1.0000047302882185,1.0000021555588856,1.0015448864022576,1.0000022164918652,22.118488860735017,4.017150548265181,0.9043498318567011,0.1361857768337058,0.15294775105578065,0.9895833432674408,1.0,0.9703302979469299,1.0,0.3678796589374542,0.9895833333333334,0.052083333333333336,0.057291666666666664,0.9410049913761517,nan,nan,0.9308610849548131,nan,nan,2.9275526569767023,0.000638873290144491,0.023486962818210666,0.02235052627415327,0.0003991434302866074,6.434070490045425e-05,0.0010965205139113652,2.8795162740753804,19.62144035469561,0.45731606990864315,0.0020334671566161833,26.660585216302312
|
training_logs/run_metrics_text67/07_bittoken_medium/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: True
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 2
|
| 5 |
+
cache_base_path: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/cache
|
| 6 |
+
combine_strategy: zero_pad
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/configs/config_bittoken_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_stem
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 1024
|
| 45 |
+
n_head: 16
|
| 46 |
+
n_layer: 24
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: bittoken
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: bce
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/configs/config_bittoken_text67.py --tqdm --verbose --seed 42 --run_name bittoken_gpt2_medium_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 24 --n_head 16 --n_embd 1024 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/trained/bittoken_gpt2_medium --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group bittoken_gpt2_medium_text67 --tie_word_embeddings --tokenizer_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_repo/tokenizers/num_text/bittoken_gpt2 --model rope_stem --num_embedding_type bittoken --add_reciprocal --combine_strategy zero_pad --num_loss_type bce --dropout 0 --cache_base_path /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/cache --compile --difficulty_column difficulty --train_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Multiplication_binary_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/DivM_binary_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Division_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard standard standard --train_set_ratios 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized_number --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Multiplication_binary_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/DivM_binary_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Division_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt pretokenized_number --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Sat Oct 3 19:43:21 2026'}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: bittoken_gpt2_medium_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/trained/bittoken_gpt2_medium
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_repo/tokenizers/num_text/bittoken_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/cache/bittoken_gpt2_47200109/96841076'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/cache/bittoken_gpt2_47200109/46368368'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/cache/bittoken_gpt2_47200109/63050008'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/cache/bittoken_gpt2_47200109/4733157'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/cache/bittoken_gpt2_47200109/44316829'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/cache/bittoken_gpt2_47200109/48579485'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/cache/bittoken_gpt2_47200109/63589654'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/cache/bittoken_gpt2_47200109/28473695'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/cache/bittoken_gpt2_47200109/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized_number
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Multiplication_binary_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/DivM_binary_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'pretokenized_number']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Multiplication_binary_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/DivM_binary_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-07/native/paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: bittoken_gpt2_medium_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/07_bittoken_medium/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_binary_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_DivM_binary_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_binary_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_DivM_binary_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_binary_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_DivM_binary_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_binary_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_DivM_binary_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_binary_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_DivM_binary_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,3.9341630401221064,3.6084216708284966,4.070420869946551,3.584218148639744,4.0097183035483495,3.895331149505788,0.7450629472732544,2.0331701637651785,4.393902298761532,1.0000220402903524,1.000012305894978,1.0000337853301584,1.0000152233923028,1.0000502727397813,1.0000086677533515,2.106623814600239,1.000084571374737,79.35999541764497,9.829649566557874,0.3787127607343319,0.029081322954632904,0.03010726128889547,0.020305125590628154,0.025475633419979847,0.1354166641831398,0.0572916604578495,0.09052359871566296,0.6817179304020532,0.04589585214853287,0.10416666666666667,0.0,0.005208333333333333,0.005208333333333333,0.0,0.49023271741801483,nan,nan,0.4675813817981786,nan,nan,5.091767278107457,0.02367419444622015,0.023977156486889382,0.026957453452089733,0.022899255629500036,0.025919250462036306,0.008528314999805517,0.0029365053855930213,0.020489473507224156,4.936385643853956,748.5381695082102,0.2628578675751934,0.004997126321997158,27.36295237262605
|
| 3 |
+
15264,250085376,5.062201521071454,4.250562466866503,4.917585219233919,4.153620053757739,4.848249668280497,4.845728527265237,0.7238714396953583,2.563269025728351,3.9461562806197135,1.000016790103834,1.0000042495092205,1.000006919418075,1.0000207124187757,1.000011637310625,1.0000664127684538,2.062406428290407,1.000147154149825,50.745861819817456,6.64539356930963,0.37966062373768283,0.1513653569994715,0.12358999799112148,0.15206895447318985,0.12510575413751443,0.1458333358168602,0.0885416604578495,0.14678150042891502,0.9026892195742926,0.1552450954914093,0.09895833333333333,0.07291666666666667,0.057291666666666664,0.109375,0.026041666666666668,0.5133858868066454,nan,nan,0.525887364463415,nan,nan,4.530981670744889,0.021211782439350225,0.021601425680288933,0.024897583435969015,0.020668669057076705,0.023929706760252623,0.007151283654157342,0.002663025540796126,0.015296876887752785,4.393561295032564,55.023197127213265,0.355741710587555,0.009998084214664627,16.18696685993421
|
| 4 |
+
22896,375128064,5.204354509360087,4.31236587792345,4.936164947794017,4.139250755507115,4.723253865829179,4.940890352054035,0.7285231649875641,2.5857785004806093,3.797146486622902,1.0000029649843893,1.0000034057130873,1.0000048999570277,1.0000242794514338,1.0000029998990705,1.000012994029274,2.0720836734638537,1.0000294424178542,43.810393226980885,5.875839765210764,0.3847173269738705,0.2014259863822389,0.13563499981286725,0.18470108186626077,0.15107766327631883,0.3020833283662796,0.1145833395421505,0.25767751038074493,0.9627683359413646,0.20488528907299042,0.10416666666666667,0.109375,0.052083333333333336,0.13541666666666666,0.026041666666666668,0.6508334873942658,nan,nan,0.6478998878277101,nan,nan,4.260131431252786,0.020127346059880178,0.020360713845781413,0.0238020447373083,0.01944392482027057,0.022823687607831183,0.006438685473865569,0.002555469448125324,0.013204282540977677,4.131375256621237,42.3700070303691,0.37789340504685287,0.014999042107332095,14.544758784457914
|
| 5 |
+
30528,500170752,5.99927523297508,4.839608811151587,5.727477540571831,4.5388218925023605,5.429501614839312,5.6990440581187904,0.5411490350961685,2.8594536457451234,3.730091657528343,1.0000003830834907,1.0000003405527913,1.0000002682202838,1.0000001157943994,1.0000002800132086,1.0008742037327183,1.7201216193238305,1.0000292368208923,40.99534663203233,5.524041453285993,0.3975620639122912,0.24723012256163635,0.16603478133283545,0.21803926279003777,0.15978322649513965,0.6614583134651184,0.3749999813735485,0.44000209867954254,0.9814349710501996,0.29128193855285645,0.09895833333333333,0.140625,0.052083333333333336,0.16666666666666666,0.03125,0.8710064191642838,nan,nan,0.7699835513194557,nan,nan,4.097322775601591,0.01947554867176253,0.019602120089085004,0.02312857210972938,0.018720612058999792,0.022166246331206074,0.005711110375836355,0.002476377069959378,0.011690913473936363,3.9743512565742565,37.93570161702998,0.3875733714324248,0.020000000000000004,14.154506416104693
|
| 6 |
+
38160,625213440,6.524678697780624,5.3463233156705225,6.411416076740112,4.879817158147786,5.878786330732121,6.024556627091442,0.1670115366578102,3.0725071528542167,3.63612895195062,1.0000010238055879,1.0000002626284754,1.0000003408596587,1.0000039638884535,1.0000000847502712,1.0000648425808678,1.181814487949952,1.0000147578367748,37.316055255549344,5.055328335538821,0.39780389044046915,0.2702816342712193,0.18994234657473963,0.25521224138440085,0.19604546215380408,0.8489583134651184,0.84375,0.5985171496868134,0.989121286769338,0.352354496717453,0.109375,0.15625,0.057291666666666664,0.171875,0.036458333333333336,0.9544679538812488,nan,nan,0.8535673594257484,nan,nan,3.982619791516968,0.019005585042857397,0.018998829069482925,0.02258586777965743,0.018174952107913152,0.021662572978954738,0.004969264775817477,0.002168237803534622,0.010297126469345845,3.864757336504304,34.86286199742643,0.3957286344360818,0.019965729849129636,14.129766859848782
|
| 7 |
+
45792,750256128,6.697203317035459,5.4352968866352,6.516174143488774,4.99793699476642,6.075914142322295,6.023536794231404,0.037925779819488525,2.9626289949288402,3.5830577822828977,1.0000003961218427,1.0000113185190482,1.0000004628655126,1.000000507568674,1.0000006335894134,1.0000129867906997,1.0386560837689207,1.0000079419011403,35.39429951695107,4.82588776089737,0.4127492880406327,0.293192636697213,0.19955136385754843,0.27486815447640006,0.2144126612900754,0.953125,0.9609375,0.7688579857349396,0.9926662469721833,0.38225314021110535,0.11458333333333333,0.16666666666666666,0.0625,0.18229166666666666,0.052083333333333336,0.983077650812144,nan,nan,0.9129200130701065,nan,nan,3.894189647268075,0.018641410998649934,0.018522640670194785,0.02215987353923761,0.017721860662719677,0.021246890207475336,0.004340592498129516,0.0018537872920162402,0.009123484356932316,3.7805790885512347,32.45280425503569,0.40255475729953816,0.019863180383909898,13.995623941859327
|
| 8 |
+
53424,875298816,6.564703012497734,5.300922697718036,6.343678429409438,4.7725126186003255,5.794933802447925,5.996239651500826,0.03150581102818251,3.0033803399718138,3.5434518143495852,1.0000004221912682,1.0000003039192558,1.0000001934017786,1.000002884771884,1.0000001654647046,1.00002212290021,1.0323433635454855,1.000002879678287,34.02680684908719,4.673242131662229,0.4218437272395559,0.30359290656181,0.2112256438332893,0.2948095317680871,0.21751898792896557,0.9791666567325592,0.973958320915699,0.842920333147049,0.9958849853380555,0.39776328206062317,0.125,0.17708333333333334,0.0625,0.203125,0.057291666666666664,0.9921207557490561,nan,nan,0.9468924681035181,nan,nan,3.6130611184882917,0.017466005157131977,0.017181199414305428,0.02097153239170835,0.01644342431170862,0.020086022951102615,0.0030564304833327212,0.0013839541230528634,0.006126059843082053,3.510346473385349,30.96144901181667,0.4072052352638052,0.019693132578928373,14.052405097513665
|
| 9 |
+
61056,1000341504,6.7195265209300326,5.39940081753744,6.48329378509273,4.860655114054655,5.979082332300427,6.111586779976728,0.008195017697289586,3.101167768936344,3.5096449708022797,1.0000000710907626,1.0000002256871934,1.0000000133489553,1.0000000447034598,1.0000001452836407,1.0000000900276969,1.0082299719519061,1.000008037421607,32.914660932802825,4.5469888369242275,0.4252071446023522,0.3088268100320377,0.22166511605770434,0.30097043163799075,0.22631152469717664,0.96875,0.993489570915699,0.8873656690120697,0.9979296334156468,0.40890032052993774,0.109375,0.17708333333333334,0.06770833333333333,0.20833333333333334,0.046875,0.9845594200305641,nan,nan,0.9546549428875247,nan,nan,3.5129296733971076,0.01707794127527333,0.016603611382085683,0.02044694001785109,0.01588272632169326,0.019565623951587726,0.00220332801467639,0.0010015299521495115,0.004784315117251012,3.4153636409780685,29.871244101154478,0.4107567348031315,0.01945688144839207,14.026752629754084
|
| 10 |
+
68688,1125384192,6.880597848134576,5.603863118836607,6.741023470958071,5.241949623760981,6.375401036813853,6.459958635890377,0.008715404663234949,3.2230244419634433,3.4855159369762987,1.000000177261585,1.0000001179668656,1.000000009934105,1.0000000655029262,1.0000000167638041,1.0000028641307663,1.0087673546667426,1.0000072716926964,32.10791025172946,4.457409792183217,0.4281970901869735,0.317798151405156,0.2228408371382689,0.303656662322638,0.23661163253866835,0.9895833134651184,0.993489570915699,0.9022597968578339,1.0,0.41638776659965515,0.13020833333333334,0.18229166666666666,0.06770833333333333,0.20833333333333334,0.052083333333333336,0.9965087405095497,nan,nan,0.9616259230921665,nan,nan,3.4497968838878816,0.016802161385059217,0.016227238478658396,0.020103315531863855,0.015540626590142768,0.019242088825950836,0.0014615497616836135,0.0006251204943813609,0.0035824921377810453,3.3562122747994527,28.980084885716188,0.41380947262860945,0.019156226183831337,13.937046711491854
|
| 11 |
+
76320,1250426880,6.825999422930182,5.5802329555153785,6.746018346088618,5.228210030743984,6.3853187045701505,6.546529965228805,0.0004133740076213144,3.3188642037256013,3.4653358062108355,1.0000000040357309,1.000000017384683,1.0000000158324722,1.0000001378352603,1.000000067675928,1.0000050034520322,1.0004134794523396,1.0000028781479564,31.48488420640192,4.387256201135369,0.43736797487994034,0.31840329166637227,0.23360658288772695,0.30994800669332334,0.24382019861824994,0.9947916567325592,1.0,0.9259771406650543,1.0,0.4261225759983063,0.140625,0.1875,0.06770833333333333,0.22395833333333334,0.0625,0.9950666074291803,nan,nan,0.9750103950500488,nan,nan,3.4016573986885934,0.0165790681448029,0.01592918084669145,0.01982292834998666,0.015255302260336964,0.018976831869863278,0.0009597832498178442,0.00026324433710803554,0.0026730582586232824,3.311197985463485,28.416422455690693,0.4158451935155337,0.018793456452196412,13.96160055159904
|
| 12 |
+
83952,1375469568,6.6305844048038125,5.373198597071069,6.4859094303101275,5.067912609626783,6.2114852908998675,6.453253865105429,0.001080840127542615,3.2420078106629235,3.447025927132927,1.0000000058983753,1.000000008381902,1.0000000080714606,1.000000070780453,1.0000000242143818,1.0000048079834616,1.0010816216060356,1.0000006005219682,30.90047703705662,4.322396020501628,0.435558478198124,0.33391051778106284,0.23112055120661718,0.30401076123010695,0.2516052061890301,1.0,1.0,0.9437552690505981,1.0,0.4297809898853302,0.15104166666666666,0.19791666666666666,0.06770833333333333,0.20833333333333334,0.057291666666666664,0.9998677022134265,nan,nan,0.9746218870083491,nan,nan,3.36455319644291,0.016400737707958062,0.015731474162247957,0.019643463429722837,0.015050722978088894,0.018782955515367624,0.0007007214159557883,0.00011718936377645223,0.0021295875301003335,3.2759963290663054,27.8811426018394,0.41777614647799816,0.018371334958694578,13.912471350490122
|
| 13 |
+
91584,1500512256,6.85456159897143,5.54128363354012,6.7123882190013795,5.182660042616661,6.321598782824752,6.538047122462256,0.0007236902019940317,3.3218977860033774,3.4301269430822385,1.0000000316649122,1.0000003439487597,1.000000021420405,1.000000416609669,1.0000000940633649,1.0000269492412173,1.000724103735506,1.0000026407744924,30.385692050246305,4.265160739078293,0.4407766187863622,0.3286477665574802,0.25028964981813706,0.3159930128173474,0.252502968148015,0.9947916567325592,1.0,0.9481537044048309,1.0,0.4392009377479553,0.140625,0.18229166666666666,0.08333333333333333,0.21875,0.07291666666666667,0.9979856073235472,nan,nan,0.9726166874170303,nan,nan,3.336754953997648,0.016259412201164402,0.015597950625327723,0.019510122937957966,0.014921844559513387,0.01865446621968309,0.0005714770448144674,8.000056571175753e-05,0.0018134024308863312,3.2493462618583573,27.439246271891164,0.41947872466573677,0.017893076407163026,13.923939695433285
|
| 14 |
+
99216,1625554944,6.6403470470880475,5.4551812031616755,6.5900131100788695,5.165253135686072,6.284896554425352,6.519670186562328,0.00032197409018408507,3.2605333832417447,3.4174881786845313,1.0000000034148488,1.0000000018626451,1.000000003414849,1.0000000527748987,1.000000021730852,1.0000006935231187,1.0003220411593827,1.000012062271384,30.017075395397644,4.224156697283291,0.4440192783270206,0.32178209349568826,0.2478676337731199,0.3132551761796566,0.25175536155427325,1.0,1.0,0.9377633929252625,1.0,0.43636322021484375,0.14583333333333334,0.18229166666666666,0.078125,0.22395833333333334,0.06770833333333333,1.0,nan,nan,0.9764011115767062,nan,nan,3.3145208955904475,0.01613561200367835,0.01548130215224722,0.01939212887785134,0.014819279974889452,0.01854774808069557,0.0004973243598619269,6.935566427584053e-05,0.0016034954312479721,3.2279746333471673,27.06547338345266,0.4208531632933539,0.017362323018206012,13.932725137781276
|
| 15 |
+
106848,1750597632,6.859321203393235,5.481336916175979,6.635223676760628,5.170357066827485,6.33050563372668,6.423903128062722,0.00019340120343258604,3.2316735439355635,3.402994394806835,1.0000000065192514,1.000000004656612,1.0000000496701758,1.0000000381841962,1.0000000242143512,1.0000033980219296,1.0001934265280523,1.0000025739304084,29.583202057181705,4.175933508767409,0.45357227103731024,0.32563665101448025,0.24814547471868742,0.3244695225953932,0.25841247209173185,1.0,1.0,0.9629688262939453,1.0,0.4434433877468109,0.13020833333333334,0.19270833333333334,0.08333333333333333,0.234375,0.057291666666666664,1.0,nan,nan,0.9865781165038546,nan,nan,3.29557432854388,0.016029371588183714,0.015379083573402985,0.019280821094169016,0.014732798112876542,0.01844865735376203,0.000449296124737259,6.227340965623865e-05,0.0014572074468122555,3.2097348036735234,26.638809274702453,0.42259597237879876,0.016783116791537683,14.032197572423664
|
| 16 |
+
114480,1875640320,6.609008511838812,5.461152598261834,6.571581181138744,5.111944531400972,6.260682777812053,6.219258055706188,0.000169878636370413,3.124547411983437,3.384092877347333,1.0000000009313226,1.0000000049670534,1.0000000161429137,1.0000000546375671,1.0000000360111085,1.0000011380712723,1.0001699018295724,1.0000055162714792,29.025653859024633,4.113981169765214,0.4463920872390923,0.34691704147928465,0.253062650773836,0.31973014771326586,0.25756775252119435,0.9947916567325592,1.0,0.9540797472000122,1.0,0.44692105054855347,0.15104166666666666,0.203125,0.08333333333333333,0.21354166666666666,0.07291666666666667,0.9971261629834771,nan,nan,0.9844811282431086,nan,nan,3.278947919039648,0.0159259812574792,0.015292911318141856,0.019180424025758664,0.014657969399684665,0.018362178668968127,0.00041192388291252826,5.720231105623857e-05,0.0013382028153336894,3.1937211086956627,26.20792812508857,0.42417166376607474,0.016159868723772935,13.94016195774243
|
| 17 |
+
122112,2000683008,6.495204568219681,5.327421831587949,6.4127145232632765,4.99718470157433,6.089519277215,6.312961923268148,0.00012176562449894845,3.1941813326467137,3.375396421586629,1.000000000310441,1.0000000074505795,1.0000000021730857,1.0000000276292216,1.000000014901158,1.000000344899068,1.0001217748833813,1.0000016842033963,28.785868892852672,4.087332527700334,0.4540699006307966,0.35625904100361244,0.2564073124585781,0.32076176946297486,0.2715776011878248,1.0,1.0,0.9821955263614655,1.0,0.45622310042381287,0.140625,0.21354166666666666,0.08333333333333333,0.22916666666666666,0.09375,1.0,nan,nan,0.9909308546533188,nan,nan,3.2635239417593764,0.015827994919912453,0.0152052546054734,0.01908585030214886,0.014586476054368639,0.018271618636515694,0.0003782755707090955,5.456218067248166e-05,0.001228620989618534,3.1788852713783458,25.88610884990832,0.4255494874954786,0.015497325216091833,14.009652909074362
|
| 18 |
+
129744,2125725696,6.469633406338552,5.276339284765205,6.346958106065895,5.0790994831662974,6.161293442981876,6.271995342947207,0.000301605792628834,3.151772994976441,3.366167540953029,1.0000000257665507,1.0000000319753917,1.0000002238205554,1.0000001459063808,1.0000000555688524,1.0000012290258664,1.000301689387642,1.0000298108853332,28.51352965256687,4.05709587387816,0.45207110395764494,0.35922643336256255,0.26037578213160595,0.3266685969375365,0.2787615751954672,0.9947916567325592,1.0,0.9733064770698547,1.0,0.4611230790615082,0.15625,0.21354166666666666,0.08854166666666667,0.234375,0.09895833333333333,0.9976949250946442,nan,nan,0.9905660847822825,nan,nan,3.2495684887690612,0.01573537283262679,0.015130984212111117,0.018996934492611552,0.014517899956046536,0.018191476601939375,0.00035315254925800266,5.28310332458008e-05,0.0011447501541170866,3.165445069177825,25.62418480389273,0.4265503405712436,0.014800531927597655,13.922752846794817
|
| 19 |
+
137376,2250768384,6.37969315269341,5.262339128491774,6.346282767752807,4.982056663682039,6.067072201209629,6.172712410787446,4.886119859293103e-05,3.134050584802134,3.349448462579555,1.000000002173086,1.0000000133489522,1.00000000745058,1.0000000459452236,1.0000000481181999,1.0000009589395567,1.000048862720301,1.0000174727845161,28.047364342143155,4.005270194847063,0.45734600214923177,0.3589310489734402,0.2647713103293252,0.3322890955044062,0.290300292618252,0.9895833432674408,1.0,0.9881874322891235,1.0,0.4681151211261749,0.16145833333333334,0.21875,0.09375,0.234375,0.10416666666666667,0.9968003053218126,nan,nan,0.9959374259536465,nan,nan,3.236324425081245,0.01564107231316134,0.015057330146437781,0.018908452020195306,0.0144554037218562,0.018114623101216632,0.00032937155884222213,5.089710369484762e-05,0.001068446126535289,3.1526988107583516,25.320726038337504,0.42765539624499826,0.014074795349650518,13.939414677236144
|
| 20 |
+
145008,2375811072,6.336514951040347,5.297077698322635,6.362122495969137,4.957497727125878,6.030827961854792,6.064960720634502,0.00010436063530505635,3.0685372043236496,3.3415557700791396,1.0000000012417631,1.0000000027939673,1.0,1.0000000111758658,1.0000002011584204,1.000000956154377,1.000104366447109,1.000000400487816,27.824673715260655,3.980531072746664,0.45869171782646445,0.34901577825280233,0.2689575824321385,0.33181474522212384,0.2835213561617783,1.0,1.0,0.9925990402698517,1.0,0.46604040265083313,0.15625,0.203125,0.09895833333333333,0.22916666666666666,0.09895833333333333,1.0,nan,nan,0.9979230786363283,nan,nan,3.223160563202107,0.015548548020277976,0.014986514738711425,0.01882686803347134,0.014389574505287208,0.018035811072398968,0.00030916582996739426,4.980567406670231e-05,0.000999832623328616,3.14001442515698,24.993129939340694,0.42919819831332695,0.013325642393809753,13.920656526334943
|
| 21 |
+
152640,2500853760,6.087620854377747,5.243967771840592,6.30546013514201,4.920201709804432,6.001674339796106,5.885256014452612,2.2964705294725718e-05,2.9329232926467492,3.3304911425802857,1.0,1.000000000310441,1.0,1.0000000108654263,1.0000000055879343,1.0000004790091082,1.0000229650851167,1.0000006795938865,27.541693869577333,3.949079778892138,0.4686211330393018,0.35640901891289245,0.27310696080092545,0.33432651100668953,0.30197407750994487,1.0,1.0,0.9852375686168671,1.0,0.4757155478000641,0.15625,0.20833333333333334,0.09375,0.23958333333333334,0.11979166666666667,1.0,nan,nan,0.9936747353834411,nan,nan,3.2111318904378043,0.015461982636251061,0.014916290699749674,0.018751955250438216,0.014321530543434987,0.01795662969071288,0.00028867105508349294,4.866954260163602e-05,0.0009338688938647843,3.12845227441729,24.757273840704304,0.43009264646508033,0.012558778301146429,13.918020562818532
|
| 22 |
+
160272,2625896448,6.149323789092399,5.169162362813942,6.209848525313039,4.875380887339277,5.9179218880211,6.0423042774590385,4.683719316744828e-05,2.995573852431203,3.319775504719776,1.00000000776102,1.0000000099340989,1.0000000021730857,1.0000000931320274,1.000000027008325,1.0000044107926709,1.000046838841449,1.000000626955907,27.240145069896215,3.915577454054978,0.46745648428443654,0.3662117930413703,0.28359596832451917,0.3459847664597402,0.3077665472757576,1.0,1.0,0.9925990402698517,1.0,0.48535802960395813,0.17708333333333334,0.20833333333333334,0.09375,0.23958333333333334,0.125,1.0,nan,nan,0.9974385493745407,nan,nan,3.199957396717941,0.015377960075408866,0.01484183428064184,0.01867365203792936,0.01425515273833227,0.017872904384639544,0.0002719244566077044,4.7474123256473306e-05,0.0008779124246540691,3.1177385646087115,24.48907761716243,0.4314248857344159,0.011780043193471874,13.944095842652775
|
| 23 |
+
167904,2750939136,6.043007969856262,5.184319035460502,6.196319659550985,4.87110258017977,5.922581361296274,5.85812301587641,1.1614664458647894e-05,2.9533025831164834,3.3141599857869246,1.0,1.0000000161428757,1.0,1.0000000086923426,1.0000000065192545,1.0000011041924073,1.0000116147747986,1.0000001769871174,27.095877919870386,3.8995434274643537,0.4748582951585306,0.3648676055193483,0.2827412529713834,0.3504492571962912,0.3117119695070085,1.0,1.0,1.0,1.0,0.4879375100135803,0.171875,0.21354166666666666,0.10416666666666667,0.2604166666666667,0.125,1.0,nan,nan,0.9999197140956918,nan,nan,3.189122788074402,0.015297237194190745,0.014770079689988102,0.018597067476938144,0.014187068392159476,0.017790538562132415,0.0002564733609175853,4.6280097857504114e-05,0.0008256752445042651,3.107352350977107,24.244095197576147,0.4325660614909866,0.010995367597369719,13.95966317916186
|
| 24 |
+
175536,2875981824,6.25576376914978,5.257182717323303,6.301195221324917,4.9176643338675285,5.948964755050839,5.965646430847681,7.568631917820312e-05,2.9943012594106424,3.3068130730922953,1.0,1.0,1.0000001958809244,1.0000000046566118,1.000000000310441,1.0000007202207035,1.0000756898004393,1.0000005079517669,26.90152226891153,3.877955487525824,0.48051058504852273,0.3692498690820139,0.2846173986085869,0.3460428169652429,0.31768295042004213,1.0,1.0,0.9970238208770752,1.0,0.4906398057937622,0.18229166666666666,0.21354166666666666,0.09895833333333333,0.234375,0.13541666666666666,1.0,nan,nan,0.9984357384964824,nan,nan,3.1780884396089575,0.015219825327918352,0.014688014673445944,0.018513712271687445,0.014113975733477599,0.01770008021842293,0.00024259908095452797,4.5398574261034e-05,0.000779919252375101,3.09678489803112,23.966548503194964,0.4338967988982141,0.01021072727974235,13.94573220857092
|
| 25 |
+
183168,3001024512,6.119270523389179,5.244563427443298,6.2701511544485395,4.908083507791099,5.969036422980327,5.865463948187113,9.733496881381143e-05,2.952775951796027,3.298394371580798,1.0,1.0000000068296924,1.0000000161428757,1.00000002918138,1.0000000027939677,1.0000003337233627,1.0000973407797775,1.0000164590951988,26.668037452292207,3.852016848982051,0.4745120379837719,0.37203131027791003,0.28627638182079485,0.33612059774141645,0.31315950949054827,1.0,1.0,0.9955752193927765,1.0,0.4875054657459259,0.17708333333333334,0.22395833333333334,0.109375,0.234375,0.13020833333333334,1.0,nan,nan,0.9985663378611207,nan,nan,3.1684087939081413,0.015148591664915867,0.014609907255400621,0.018435270368423542,0.014038178259853384,0.017608576367663224,0.00023058497280990074,4.474208479170264e-05,0.0007405620434438286,3.087552365540396,23.90832713121388,0.4342493000020521,0.009432097738825898,13.868212232291645
|
| 26 |
+
190800,3126067200,6.028574706055224,5.183162814624396,6.212537686030071,4.83751660895822,5.919704492203423,5.740124849278573,7.151714817155153e-05,2.89150427404147,3.28935285822566,1.0000000009313226,1.0000001254154272,1.0,1.0000000627088323,1.0000000549475663,1.00000205417571,1.0000715217335832,1.00000064481139,26.436993248067882,3.8263408569768576,0.4794535903752606,0.38491613856288837,0.29313991538754025,0.34923129365891703,0.32876043083716144,1.0,1.0,0.9970501363277435,1.0,0.49991294741630554,0.171875,0.234375,0.11458333333333333,0.24479166666666666,0.15625,1.0,nan,nan,0.9993612586210171,nan,nan,3.1588644423397754,0.015077613374907562,0.014534031963991138,0.0183563413308181,0.013970022665551172,0.017520705639621582,0.00022129765618601654,4.394815792689389e-05,0.0007063646692730462,3.078434101365127,23.586163124135954,0.43543258192216827,0.008665408697250875,13.92123705279902
|
| 27 |
+
198432,3251109888,6.120043198578059,5.247286560945212,6.297729214032492,4.888908698533838,5.956723895544802,5.826083764589487,9.909856999001931e-06,2.952859806319758,3.282721768715419,1.000000000310441,1.0000000027939668,1.0,1.000000034148361,1.0000000068296946,1.0000031244399823,1.000009909906163,1.0000005181496687,26.260142667545527,3.80668402934709,0.4767887484886838,0.37712611146345326,0.2957325948538221,0.36347615554547474,0.3060477537705684,1.0,1.0,0.998525083065033,1.0,0.4958518147468567,0.17708333333333334,0.21354166666666666,0.11979166666666667,0.2552083333333333,0.11458333333333333,1.0,nan,nan,0.9997443991402785,nan,nan,3.150036463541305,0.015010906386323882,0.014457081448303691,0.01827438605137711,0.013904333420305766,0.017430401119275983,0.0002130918439849051,4.351566680316886e-05,0.0006783093599109931,3.07002442298362,23.48962010833702,0.43607426669831295,0.007916498943711495,13.943768815433122
|
| 28 |
+
206064,3376152576,5.939233700434368,5.1421613218252915,6.162592975422731,4.81593609166642,5.865475737179318,5.7414289961144895,1.956375535883126e-05,2.921853059104067,3.2777820351766422,1.0,1.0000000717108557,1.0000000080714548,1.000000003104408,1.000000003104408,1.0000016540189174,1.0000195639773684,1.0000002206786842,26.134171187390624,3.792688079117413,0.4846999470652849,0.37833007555490117,0.29337095869956187,0.3665395310735,0.3402548864524405,1.0,1.0,0.998525083065033,1.0,0.5060828328132629,0.1875,0.21354166666666666,0.09895833333333333,0.25,0.16145833333333334,1.0,nan,nan,0.999087743461132,nan,nan,3.1422870457823215,0.014946668340556407,0.014384721616669646,0.018196494882967994,0.013830513891973548,0.017342505750153933,0.00020589788636536608,4.3287120002269156e-05,0.0006520705834479724,3.062684870108306,23.38732852713367,0.43667628215840654,0.007191071867150021,13.935384242885098
|
| 29 |
+
213696,3501195264,6.003787318865458,5.175032019615173,6.197244131316739,4.799749255483505,5.816701455352206,5.668642867976828,9.49081550061237e-06,2.877158968036562,3.2725453709911863,1.0,1.0,1.0000000037252876,1.0000001989853335,1.0000000034148493,1.0000003470723482,1.000009490871077,1.0000003035148946,25.99790717742026,3.7775463916671166,0.4821363571386266,0.3956283074589083,0.2958498820652483,0.3718976043243258,0.3359282320269864,1.0,1.0,0.9985119104385376,1.0,0.5099425315856934,0.18229166666666666,0.22916666666666666,0.10416666666666667,0.2552083333333333,0.14583333333333334,1.0,nan,nan,0.999039945503076,nan,nan,3.1349910144895254,0.01488625851136344,0.014307192496738673,0.018113852099217422,0.013759499877511643,0.01725092733745508,0.00020051617537905538,4.3090459576712504e-05,0.0006291762444125121,3.055800485340742,23.215080605550384,0.43754168682237715,0.006494652022090374,13.851076107036784
|
| 30 |
+
221328,3626237952,6.034269332885742,5.207617283488314,6.236508292146025,4.82709568211186,5.873667803282534,5.6859016626405126,1.2693031294475077e-05,2.881602493139904,3.2674534609153247,1.0,1.0000000006208818,1.000000002173085,1.0000000145907024,1.000000006829695,1.0000008155237636,1.0000126931162123,1.0000002739670562,25.869950433123126,3.7633293599938358,0.4848927292691654,0.38371098347161436,0.3052012107314822,0.3727614988863086,0.3442671389436116,1.0,1.0,0.9985119104385376,1.0,0.5132926106452942,0.1875,0.21354166666666666,0.109375,0.2604166666666667,0.15104166666666666,1.0,nan,nan,0.999039945503076,nan,nan,3.129189935505682,0.014827008778994068,0.01423937716693139,0.018034835618573736,0.013691825103057012,0.017161849975503934,0.00019619080651392124,4.286397333070417e-05,0.0006136110119491893,3.050382357233892,23.171810628295695,0.43747278512601956,0.005832543055900948,13.920203260488005
|
| 31 |
+
228960,3751280640,5.941142837827406,5.170661012331646,6.203252674701313,4.801824227542378,5.83120306674391,5.662478840909717,5.9742988014477305e-06,2.8791182871555803,3.264644493639935,1.0000000006208818,1.0,1.0000000015522041,1.0000005320419683,1.000000003414849,1.000000235003575,1.0000059743202687,1.0000002760443154,25.802699299884452,3.7558562580980572,0.48027413976703415,0.3999409944012695,0.31118594490560536,0.3813359950740574,0.3412615467002067,1.0,1.0,1.0,1.0,0.518771231174469,0.18229166666666666,0.22916666666666666,0.11458333333333333,0.265625,0.15625,1.0,nan,nan,1.0,nan,nan,3.1227234933788255,0.014772796727359623,0.014162777196895786,0.01795179475608765,0.013619560482926408,0.01706849113428883,0.00019278499278690029,4.271628904010873e-05,0.000598024961644071,3.0443145305433736,22.991995332251054,0.43851041470886654,0.005209787318395601,13.933488468867015
|
| 32 |
+
236592,3876323328,5.93025088403374,5.162111719449362,6.199247839239735,4.807860654150601,5.852941079686085,5.573431259952283,5.348221748135984e-06,2.8237019850679257,3.260134798649233,1.0000000009313226,1.0,1.000000002173085,1.0000002002270973,1.0000000037252899,1.0000002052012247,1.0000053482386533,1.0000006260913463,25.68092021205047,3.7423251776264994,0.47830752449063774,0.40302746150513946,0.3145615602328984,0.37185337041696354,0.3462263191175873,1.0,1.0,0.9985119104385376,1.0,0.5193318724632263,0.17708333333333334,0.24479166666666666,0.11458333333333333,0.2552083333333333,0.16145833333333334,1.0,nan,nan,0.999039945503076,nan,nan,3.118083141007409,0.014722642232924259,0.01409461225865729,0.01787315220788452,0.013551298912608402,0.01698054640539574,0.00019021834853419887,4.251768917073409e-05,0.0005878641874332234,3.0400402722873734,22.960685163174034,0.4386427650536631,0.004631127461369502,13.925146221189829
|
| 33 |
+
244224,4001366016,5.9099437799304715,5.128895839055378,6.1407121034959955,4.7717297123745075,5.810279654338956,5.642930017162408,7.631651328665612e-06,2.86380706157654,3.257198757511408,1.0000000006208818,1.0,1.0000000012417631,1.000000004656612,1.0000000068296977,1.0000005426498215,1.0000076316907351,1.0000002916151844,25.6048326572166,3.7338712373912557,0.4822746967567474,0.39949262167728294,0.31766270289520593,0.36449947005859956,0.3368865417414922,1.0,1.0,1.0,1.0,0.5161579847335815,0.171875,0.234375,0.11979166666666667,0.25,0.13020833333333334,1.0,nan,nan,1.0,nan,nan,3.1136822862502456,0.014673928147055608,0.014030620644601538,0.017797523431573002,0.013483080678931185,0.01689695962631501,0.0001883047785117768,4.2211805327222456e-05,0.0005805738148186736,3.0359890665247216,22.885349926521194,0.4390823213067257,0.004100970320511953,13.89108272545458
|
| 34 |
+
251856,4126408704,5.949747761090595,5.152891476949055,6.175328096064427,4.795138837769628,5.826863649301231,5.645330237845286,1.2068712862856046e-05,2.875570435055355,3.2555591441535703,1.0,1.0,1.000000000310441,1.000000001862645,1.0000000027939673,1.0000002856053367,1.000012068834579,1.000000319466452,25.564092502671514,3.729345020171659,0.4806952656161375,0.4108467843946025,0.31622545946197544,0.37004564277414737,0.3477044419504309,1.0,1.0,0.998525083065033,1.0,0.5215435028076172,0.1875,0.23958333333333334,0.11458333333333333,0.2604166666666667,0.140625,1.0,nan,nan,0.9994726357981563,nan,nan,3.1095038303424647,0.014636866464494363,0.013970549148596903,0.017724267037140865,0.01342813255546672,0.01681915348721766,0.00018624787603944087,4.202559210410868e-05,0.0005751767543369952,3.032121394443957,22.807685547217382,0.43949354390771883,0.003623353354756449,13.87087469890314
|
| 35 |
+
259488,4251451392,5.891289314876,5.1387741168340035,6.152288913726807,4.788871410613258,5.807582739740609,5.597826877607474,8.909356211006525e-06,2.840737273639502,3.254644900288743,1.0000000012417631,1.0,1.0,1.0000000031044083,1.0000000037252899,1.0000006351607769,1.0000089094170963,1.000000453055367,25.54532994969627,3.727259995044553,0.4853522671362903,0.40762969666352256,0.3242246728450985,0.37312352539410776,0.3397087034459365,1.0,1.0,1.0,1.0,0.5225988030433655,0.1875,0.23958333333333334,0.125,0.265625,0.125,1.0,nan,nan,1.0,nan,nan,3.106639151183522,0.01460080374001996,0.013916135066591404,0.017660188253635085,0.013374389479092011,0.016749272531612402,0.00018497301026215287,4.160598643157245e-05,0.0005717312945712496,3.0295400344808976,22.82538752573341,0.4393553360203277,0.003201913898652705,13.931189495064631
|
| 36 |
+
267120,4376494080,5.920204721080758,5.167346597649157,6.1826801337301704,4.794735115952789,5.832034946729739,5.620202645967595,1.0138124366676493e-05,2.8611270949052257,3.2535508482833393,1.000000002173085,1.0000000009313226,1.0000000037252876,1.0000000021730857,1.0000000012417634,1.0000008198723502,1.0000101382018092,1.0000007174365277,25.51578048826588,3.7239769082245675,0.4871306161639585,0.4047355288185922,0.3216479529109842,0.37798295826750217,0.3562298648764155,1.0,1.0,1.0,1.0,0.526766300201416,0.19791666666666666,0.23958333333333334,0.125,0.2604166666666667,0.16145833333333334,1.0,nan,nan,1.0,nan,nan,3.104046816871893,0.014573547161694494,0.013873828550324605,0.017604310153205807,0.013329910335645461,0.016687553219274624,0.00018418894871411584,4.134648948721093e-05,0.0005673771908740569,3.0271847369416163,22.798581376453114,0.43938382519740976,0.0028398614619212603,13.925240703770452
|
| 37 |
+
274752,4501536768,5.889472961425781,5.136517087618511,6.145315968741969,4.771543942081432,5.801783205009997,5.615763015113364,1.1572785638236383e-05,2.8553478386301596,3.2529133992502466,1.0,1.0,1.0000000037252876,1.000000002173086,1.0000000009313226,1.0000003442785637,1.0000115728865995,1.0000017493146824,25.503015261245753,3.722558770506144,0.4855354927259692,0.4057149177329176,0.32309562164812244,0.3828633232217846,0.3552172054121754,1.0,1.0,1.0,1.0,0.5279675722122192,0.17708333333333334,0.234375,0.11979166666666667,0.2604166666666667,0.140625,1.0,nan,nan,1.0,nan,nan,3.10218224248567,0.014549454624174327,0.013833468615955717,0.017564944428626703,0.013294297032724353,0.016641736584914045,0.00018360564711889736,4.1195241352274966e-05,0.0005667287575283028,3.0255067934556124,22.735504859086102,0.43969228509540204,0.0025399532871463355,13.956350587814581
|
| 38 |
+
282384,4626579456,5.928016702644526,5.153622428576153,6.182789818694145,4.792786243061224,5.830863120034337,5.626378687097457,1.3445543345369515e-05,2.8617123376456712,3.2520504909528727,1.000000000310441,1.0,1.0000000161428757,1.0000000024835267,1.0000000018626451,1.000000627710448,1.0000134456750387,1.0000005857988028,25.482494878808733,3.720278839865835,0.48612663676159007,0.41011512487873725,0.3234670611794307,0.3842622203553363,0.3508208415171466,1.0,1.0,1.0,1.0,0.5281761884689331,0.1875,0.23958333333333334,0.13020833333333334,0.2708333333333333,0.14583333333333334,1.0,nan,nan,1.0,nan,nan,3.1005117310032735,0.014531050367228125,0.013798055660183288,0.01752799312259669,0.013257199700513058,0.016597772848054995,0.00018348908650417953,4.106586996657586e-05,0.0005631577839606022,3.0240119287807947,22.73853300696149,0.4398341597071546,0.002304473351749283,13.91671003243411
|
| 39 |
+
290016,4751622144,5.9022533893585205,5.145963033040365,6.15165750402957,4.780622126845022,5.804964503894248,5.583579369199214,9.33892044940876e-06,2.83760348344567,3.2510359112444958,1.0,1.0,1.000000002173085,1.000000002173086,1.0000000012417634,1.000000424993067,1.0000093389833231,1.0000007318503057,25.454404453195732,3.7171572171789293,0.48438200071945453,0.41197557682619834,0.32534307792030903,0.37884192101634895,0.3569235868287382,1.0,1.0,1.0,1.0,0.529198944568634,0.18229166666666666,0.23958333333333334,0.13541666666666666,0.2604166666666667,0.15625,1.0,nan,nan,1.0,nan,nan,3.099659815566012,0.014514121783091565,0.013773464431477271,0.017503079910831595,0.013233179935137986,0.01656609377181086,0.0001832898103598709,4.112111070484541e-05,0.0005611533986133868,3.023284294611919,22.76341563644399,0.4397392698194621,0.0021352149741550937,13.928318363293446
|
| 40 |
+
297648,4876664832,5.917801463355617,5.153205991722643,6.164626286365085,4.788152975651125,5.810362101842959,5.6047387629122145,1.2644537491723895e-05,2.851064163809132,3.250573091131325,1.0000000037252876,1.0000000009313226,1.0000000058983698,1.0000000027939673,1.0000000012417634,1.0000003684929308,1.0000126446441722,1.0000041593619107,25.4443235607219,3.7160378608679583,0.48825397948420407,0.4058976050049949,0.3250365673575388,0.3780814380527062,0.35518441099345577,1.0,1.0,1.0,1.0,0.5278983116149902,0.19270833333333334,0.234375,0.13020833333333334,0.2604166666666667,0.15104166666666666,1.0,nan,nan,1.0,nan,nan,3.0992330580475986,0.01449926423420121,0.013748312516318233,0.017485044276697143,0.013206446102690499,0.0165439613913866,0.00018365499275495818,4.105447087251556e-05,0.0005596011650958665,3.0229657029291808,22.760854084661673,0.4396269764649668,0.0020334671566161833,13.923218264024845
|
training_logs/run_metrics_text67/08_bpe_medium/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: False
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 2
|
| 5 |
+
cache_base_path: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/cache
|
| 6 |
+
combine_strategy: sum
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty_sd
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/configs/config_td_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_gpt2
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 1024
|
| 45 |
+
n_head: 16
|
| 46 |
+
n_layer: 24
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: bittoken
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: mse
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/configs/config_td_text67.py --tqdm --verbose --seed 42 --run_name bpe_gpt2_medium_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 24 --n_head 16 --n_embd 1024 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/trained/bpe_gpt2_medium --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group bpe_gpt2_medium_text67 --tie_word_embeddings --tokenizer_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_repo/tokenizers/num_text/td_gpt2 --model rope_gpt2 --dropout 0 --cache_base_path /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/cache --compile --difficulty_column difficulty_sd --train_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Division_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard --train_set_ratios 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Division_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_prompt efficient_prompt efficient_prompt efficient_prompt efficient_prompt efficient_prompt pretokenized --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Sat Oct 3 19:43:14 2026'}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: bpe_gpt2_medium_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/trained/bpe_gpt2_medium
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_repo/tokenizers/num_text/td_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/cache/td_gpt2_88525419/96841076'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/cache/td_gpt2_88525419/63050008'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/cache/td_gpt2_88525419/44316829'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/cache/td_gpt2_88525419/48579485'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/cache/td_gpt2_88525419/63589654'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/cache/td_gpt2_88525419/28473695'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/cache/td_gpt2_88525419/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'pretokenized']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-08/native/paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: bpe_gpt2_medium_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/08_bpe_medium/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,3.1315294901529946,3.9980963468551636,3.605796992778778,2.3418282767136893,0.7442610412836075,4.994163274765015,4.3641088008880615,24.080133044643816,63.18502700257775,44.56945286870124,11.675060066218999,2.1090022007539755,255.84387314525702,78.98224265109532,68.63497013989259,0.30804146359463774,0.011095121183828023,0.007408895998976373,0.4114583333333333,0.11328125,0.3258153423666954,0.6816891783563567,0.03346557915210724,0.125,0.005208333333333333,0.0,0.6614319609795226,nan,nan,0.6145238281460479,0.1413051247909607,0.11875477460368226,5.247541601209271,0.04562227168087252,0.051652409601956606,0.04116745484627752,0.01976908897915518,0.0018384852604947856,0.045450453418653565,5.042041403458178,632.7437095222233,0.2185367979373365,0.004997126321997158,15.811594587580991
|
| 3 |
+
15264,250085376,3.152265032132467,4.133291343847911,3.7453309694925943,2.2307926217714944,0.7297499080499014,3.87391804655393,3.9165384769439697,28.51193521509988,82.43718588005768,54.173612677953976,10.303337273403619,2.076998180714607,76.8670494033452,50.519274116804176,43.55562753533989,0.4045218497776988,0.12091581416911724,0.09382961421898689,0.6145833333333334,0.11848958333333333,0.4949577550093333,0.9013046588709712,0.20233295857906342,0.19270833333333334,0.052083333333333336,0.026041666666666668,0.7991009878193532,nan,nan,0.7283442610496422,0.17625444513032562,0.1524341685581021,4.602566806197291,0.037128164662600764,0.04667897402202472,0.036773520028335775,0.011214857975215518,0.001618155881414906,0.026011835738264346,4.44314127161599,56.01104428408281,0.33067358420689275,0.009998084214664627,13.709445610892349
|
| 4 |
+
22896,375128064,2.987388471762339,3.940030872821808,3.6055093804995217,1.4569271082679431,0.6690413256486257,2.860758990049362,3.778060793876648,25.827400883876862,65.46172870216277,46.233215990907844,5.611366219446629,1.954266392939813,27.87535689407324,44.01283751071542,30.996596084874657,0.4291695254904477,0.17047833759086303,0.14857142099682166,0.7395833333333334,0.17578125,0.6680738081534704,0.9643770483825796,0.28569701313972473,0.22395833333333334,0.09895833333333333,0.052083333333333336,0.8573155652231511,nan,nan,0.8348061166955935,0.155887741022525,0.13184885036510727,4.306277965964094,0.03312032972006902,0.043974864281483575,0.034328667021335045,0.008066592933636622,0.0015328357758684747,0.018834439329781023,4.16642021300866,43.47701221729474,0.35397299767259033,0.014999042107332095,13.408616544379592
|
| 5 |
+
30528,500170752,2.8062342206637063,3.94942569732666,3.7122119466463723,0.7719210954383016,0.5188988571365675,2.4547342409690223,3.7159347534179688,20.60403065317969,68.41153972939325,52.00623862503728,2.5472453850748704,1.6841143125160896,32.86148050449876,41.346697946445296,31.35162102230646,0.4680772448474719,0.15755487140768926,0.17099130842093005,0.8541666666666666,0.39453125,0.7593132605155309,0.9793749304622649,0.34355708956718445,0.2708333333333333,0.08333333333333333,0.06770833333333333,0.9299869423678805,nan,nan,0.8783940016000997,0.1608927810689238,0.1361282722403606,4.133035028099513,0.03055588079747486,0.04233482422073327,0.03288084686347196,0.006400223437295479,0.0014500264335363365,0.015049837593186618,4.0043633655807005,39.27004956074481,0.36421983529358665,0.020000000000000004,13.895988862107819
|
| 6 |
+
38160,625213440,2.677132397890091,3.98509418964386,3.599731922149658,0.3078091489927222,0.31476854781309765,1.9185920159022014,3.6306022802988687,17.659526096362715,76.01404845809215,47.36510322926185,1.4222131937297613,1.37476574606261,12.856212407126053,37.93382180822008,27.803670134122175,0.50060655235426,0.18752582955853245,0.19109560128176242,0.9270833333333334,0.6744791666666666,0.8380105445782343,0.9902940889307081,0.4039812386035919,0.296875,0.09895833333333333,0.07291666666666667,0.9656122182980956,nan,nan,0.9082986616073564,0.1666295165887277,0.14175293291918933,4.013379461629086,0.02864551092537054,0.041198476878496613,0.0318848356293159,0.005347738558221735,0.0013450373029474834,0.012675545757919657,3.8922822947240974,36.29078486824186,0.3721499013981979,0.019965729849129636,13.603187318064787
|
| 7 |
+
45792,750256128,2.4817029535770416,4.072399934132894,3.698807736237844,0.27807129942812026,0.20910199669500193,1.2823269553482533,3.580707589785258,16.144106533811485,84.30174027765244,51.740389320609296,1.7777178722348728,1.2353740069283863,5.042466331029491,36.10644970387967,28.049749149449376,0.5265367199047121,0.20499716016077177,0.21469125386528606,0.9583333333333334,0.78515625,0.8803110718727112,0.9926662469721833,0.4403725266456604,0.3385416666666667,0.11979166666666667,0.09895833333333333,0.978304214872575,nan,nan,0.9360871354341119,0.15854812473298807,0.13295382919022813,3.9223632168110685,0.02710228548787084,0.04030674996663466,0.03111261225960873,0.004617680083142681,0.0012329428448924025,0.011031083724134246,3.806959841085388,33.883110072377846,0.37923726533974994,0.019863180383909898,13.562540112212043
|
| 8 |
+
53424,875298816,2.4004264871279397,4.0648800532023115,3.6787564555803933,0.20178429594185823,0.14386717416346073,1.2057659768809874,3.5342546701431274,16.509272085329066,83.15969664103433,51.45190215530416,1.2489691220155008,1.1563581319944605,4.016792494864816,34.46617190202925,27.429880361795945,0.5415280900744123,0.205485629163818,0.23276885223848975,0.953125,0.84375,0.9022186448176702,0.9958849853380555,0.455843985080719,0.3489583333333333,0.125,0.109375,0.9719402921424868,nan,nan,0.9496498876348293,0.17047446212200157,0.1463044739405935,3.618398917007184,0.022513723579259133,0.03758250852278157,0.02870864741088319,0.0014731022691618007,0.0010137053397469811,0.0038974595115377026,3.5232097516754015,32.43411230691074,0.3836774079078336,0.019693132578928373,13.479982198815671
|
| 9 |
+
61056,1000341504,1.9200853953758876,4.053576906522115,3.7125434378782907,0.02719564051343089,0.09515802391494314,1.0094521816742297,3.506678819656372,11.744416222704578,83.04703978924412,56.169055723901245,1.0294720962189337,1.101181067539455,4.259257570970536,33.53804555087911,27.269781145922565,0.5679052542923553,0.21739912553690854,0.22018005712978872,0.984375,0.8893229166666666,0.927492822209994,0.9979296334156468,0.4628710448741913,0.390625,0.14583333333333334,0.09895833333333333,0.9880186130176298,nan,nan,0.9581553244740159,0.16689009425615117,0.14368427731096745,3.523147819445297,0.020581321649586586,0.03644038118720758,0.027716154824053332,0.001170594194111547,0.0008494158611816445,0.003217689796395905,3.433172243266548,31.241947097848296,0.3879295001685057,0.01945688144839207,13.437513850897915
|
| 10 |
+
68688,1125384192,1.7950085699558258,4.032666126887004,3.7134631077448526,0.10124635527730182,0.05522729182848707,0.5900929090954984,3.4778044621149697,11.262085687347358,83.740668975791,53.22949266287532,1.1375198986123987,1.0586545387409028,2.3362907781534537,32.58087363765297,26.477940882739055,0.5845396357172564,0.2234456568247973,0.2501757615202343,0.9791666666666666,0.9609375,0.9489154368638992,1.0,0.4895426630973816,0.3958333333333333,0.13541666666666666,0.11979166666666667,0.9823795779957436,nan,nan,0.966461376020258,0.17737942427978245,0.15272092398178452,3.4647723316266394,0.019054518749680015,0.03578572001442922,0.027181055742868495,0.0010105667737455937,0.0006756821219078387,0.002869958228403072,3.3781948116855927,30.525608180328984,0.3903379856727908,0.019156226183831337,13.444204597292293
|
| 11 |
+
76320,1250426880,1.0487916866938274,3.9857343832651773,3.669207841157913,0.17678754552374207,0.05690485729913538,0.6774039059431137,3.462933818499247,5.3087897565515085,77.67074202274098,48.632748067934415,1.3499068735917463,1.0605703685462033,2.848608582005155,32.09116875184427,24.137504917602033,0.6636266746940704,0.223806883800748,0.2230253451636149,0.9791666666666666,0.9479166666666666,0.9267725547154745,1.0,0.4789915084838867,0.4635416666666667,0.14583333333333334,0.10416666666666667,0.9880835523557229,nan,nan,0.956327381451653,0.17575158116339176,0.15114995478264368,3.4194513375341766,0.017536075185883657,0.035272178761426756,0.026765488802242547,0.0009088066532520825,0.0005193798710994439,0.0026582793369455955,3.335791112260228,29.887742042416548,0.39276212167611785,0.018793456452196412,13.455198042998816
|
| 12 |
+
83952,1375469568,0.7801323309540749,3.9975024461746216,3.8956945538520813,0.23435404115753045,0.034561397323462494,0.7282274754640335,3.4460906982421875,2.387614383124407,79.05904648575607,68.63397828975793,1.343392583205185,1.036231307792923,2.453077103484477,31.563205979098793,26.639506590317108,0.7253460394906471,0.23929860933288236,0.22927465098243915,0.96875,0.9609375,0.9459553857644399,1.0,0.49825066328048706,0.5833333333333334,0.140625,0.08854166666666667,0.9764137794459581,nan,nan,0.9626378215325531,0.1802544581197291,0.15728316906218728,3.3832681156059685,0.015765642223800457,0.034867710857665137,0.02642321621289426,0.0008488834871929475,0.0004049828038611432,0.0025247475834818,3.30243291649616,29.20899615179068,0.39540456964847176,0.018371334958694578,13.473100399426297
|
| 13 |
+
91584,1500512256,0.487063467502594,4.073953151702881,3.5700695713361106,0.09694396148885669,0.05124018251808593,0.44991602751542814,3.427091360092163,1.7344701926985195,85.98827573470636,44.41153019645994,1.1216775796676064,1.0552911646411993,1.9186923058837877,30.956165340859172,23.88372893070237,0.7860591571956177,0.24639772343432922,0.24246091760632738,0.9791666666666666,0.9674479166666666,0.9511074970165888,1.0,0.5161726474761963,0.65625,0.140625,0.109375,0.9820812501517745,nan,nan,0.9737076837336645,0.16866452212807795,0.14333447711154199,3.355114286790187,0.013923789515432716,0.03458781803528252,0.026164669454057723,0.0008137548303223739,0.0003277735226103555,0.002444917364714866,3.276851548414331,28.683505787039703,0.39760222256699074,0.017893076407163026,13.444132533660735
|
| 14 |
+
99216,1625554944,0.3679432508846124,4.019017219543457,3.8728747268517814,0.1170372080581122,0.044756371974168964,0.4654782730503939,3.4140663544336953,1.4962864554616886,79.7866501689346,61.9199714647222,1.1906759562337774,1.0475205182436929,1.951072777620614,30.5637967861278,25.422282018192053,0.8123129542613997,0.23786629408312787,0.25772679262235726,0.984375,0.9609375,0.9612645208835602,1.0,0.521920919418335,0.6875,0.14583333333333334,0.125,0.9885015065665357,nan,nan,0.9780735336147094,0.17695868694069836,0.15136680849051723,3.3319950196774877,0.01208488993451595,0.034363460746723286,0.02597335757721693,0.0007876871308940527,0.0002733172484849693,0.0023841684834094655,3.256128123031368,28.322867005751068,0.39910520052838,0.017362323018206012,13.443158891935374
|
| 15 |
+
106848,1750597632,0.39107827221353847,4.152318735917409,3.943945606549581,0.027051620701361873,0.05301891040047243,0.4574128428794211,3.401150385538737,1.8367923086619948,97.8386390562323,72.4579457436838,1.0296077327007862,1.0578489166297178,2.3044492267945755,30.16260939649241,29.526841768742226,0.8542664290402446,0.240389663303664,0.2493138555793303,0.9895833333333334,0.9674479166666666,0.9648938179016113,1.0,0.521528959274292,0.7447916666666666,0.140625,0.109375,0.9900856389237257,nan,nan,0.9763588334511345,0.1862602710617504,0.16160764762510857,3.3129071158953765,0.010292064576536423,0.03419092734929065,0.025802118021094746,0.0007688797575161025,0.0002346038940929437,0.0023457957014680854,3.2392727121773257,27.929383014857894,0.40083403662784295,0.016783116791537683,13.442537117884537
|
| 16 |
+
114480,1875640320,0.4617438856512308,4.126240253448486,3.9254454970359802,0.08622242095968129,0.024293963279584812,0.5907875106689365,3.385305960973104,1.9049440305057166,95.88541313743438,65.2254142574846,1.117460181653962,1.0253157844646954,2.9018285926476373,29.68835784304402,28.249819118176436,0.8841240749389309,0.24184972615720887,0.2684346376706842,0.984375,0.9739583333333334,0.9528302252292633,1.0,0.5350340604782104,0.7864583333333334,0.140625,0.140625,0.9869089581576796,nan,nan,0.9675825464219088,0.17569916557835086,0.15103917960853627,3.294729093864175,0.008583710201690364,0.0340289590127114,0.025643414582885996,0.0007562862850161404,0.00021017199832661727,0.002324217677272151,3.223182319839837,27.476741635761922,0.40272017900267737,0.016159868723772935,13.498685730460629
|
| 17 |
+
122112,2000683008,0.36384056881070137,4.132728139559428,3.8108752270539603,0.0050366849515436725,0.04355291363754077,0.48157548709362646,3.376530090967814,1.9885296497289742,92.25833278116488,59.25766397602208,1.0051566934287923,1.0461973332119103,2.093031813491988,29.4326075415099,26.725931398365507,0.8997802800896847,0.25172057827442984,0.2792231047118174,0.9947916666666666,0.9739583333333334,0.961900015672048,1.0,0.5495803952217102,0.8125,0.13541666666666666,0.13020833333333334,0.9948518580640666,nan,nan,0.9777456593777364,0.17607897932288097,0.15232733476053303,3.2787364287904555,0.00716104552781047,0.033916603654813714,0.025500310128922413,0.0007444921306655395,0.00019448602145977816,0.0022859056617706556,3.2089335709956037,27.197973619217144,0.40382073076922187,0.015497325216091833,13.406159310872312
|
| 18 |
+
129744,2125725696,0.288339890850087,4.1388324697812395,3.7340555091698966,0.005645900273445174,0.04546554332531135,0.29232070773165714,3.3610744078954062,1.5058314593338895,93.75319444739937,54.48463806246346,1.0058022386684768,1.0479170658828254,1.486484911417408,28.975064992475165,26.036990453948654,0.9038960223222352,0.2408860827928128,0.2853106780068972,0.9947916666666666,0.9609375,0.9669362803300222,1.0,0.5450569987297058,0.8229166666666666,0.140625,0.14583333333333334,0.9948518580640666,nan,nan,0.9778855032054707,0.16808445065599167,0.14314315091663352,3.2642635573053136,0.006238040548707495,0.03377769838115246,0.025370450068612084,0.0007331335937056765,0.00018180196784249068,0.002253715496651031,3.1957087024010247,26.815354577275443,0.40531396585862217,0.014800531927597655,13.468170120169466
|
| 19 |
+
137376,2250768384,0.2636224321710567,4.236793716748555,3.6775998373826346,0.030248488184345963,0.03176355874287159,0.49394357597702765,3.3452436526616416,1.4445878996279973,107.20870218753446,52.152460232041335,1.0346711265295114,1.0333856107792239,2.879455061562629,28.52501180026594,27.75403913119158,0.9228505199884397,0.24841776389887085,0.2731998588996051,0.9895833333333334,0.9739583333333334,0.9601268917322159,1.0,0.5448236465454102,0.8541666666666666,0.140625,0.13020833333333334,0.9900856389237257,nan,nan,0.9689616604252175,0.19345983654541973,0.16868950275238603,3.250876372915665,0.005603903257464776,0.03364857269806044,0.025253091697014247,0.0007252238012094018,0.0001727820194417417,0.002222859703976469,3.1832499256465585,26.50056368112564,0.40678236290297043,0.014074795349650518,13.417704072437397
|
| 20 |
+
145008,2375811072,0.22755557116276273,4.250533998012543,3.7559904158115387,0.06984619825561822,0.036846517305093585,0.22734102029729306,3.340205947558085,1.3217618378797542,112.86852413181508,56.48712759446915,1.1028422726258167,1.038977888397488,1.4003802024809804,28.38312389748106,28.94324826073562,0.9370192742848418,0.2580006602461515,0.2862603836506194,0.9895833333333334,0.9674479166666666,0.9744563649098078,1.0,0.559691846370697,0.890625,0.140625,0.14583333333333334,0.9916751772980206,nan,nan,0.9842250891573107,0.18163964955338466,0.15766780524669835,3.238551301989707,0.005134576533345169,0.03353042577308421,0.02512248356852311,0.0007161656005964976,0.0001648579484626348,0.002199687318425845,3.171683091961842,26.293112318033444,0.40754966109803636,0.013325642393809753,13.421979042647974
|
| 21 |
+
152640,2500853760,0.10827833452882867,4.268791437149048,3.68968336780866,0.0060214570000880485,0.047460229926097476,0.32231552054872736,3.3303264379501343,1.1183641500508188,112.46029263866971,52.5269171487895,1.0062035247295331,1.0507945934017011,1.5684433922036634,28.11163674324,28.263236027297847,0.9283996516201177,0.24268809152168416,0.2755717839415914,0.9947916666666666,0.9674479166666666,0.9708316177129745,1.0,0.5428299307823181,0.8645833333333334,0.13541666666666666,0.13541666666666666,0.9948518580640666,nan,nan,0.9745588805963052,0.18900173479030022,0.16426270270797735,3.2267190034180397,0.004754168060901687,0.033422663333828366,0.0250071782678067,0.0007099363475701632,0.00015905915623064224,0.0021682790930034365,3.1604977054801293,26.017742924282885,0.4087775350944676,0.012558778301146429,13.448470562170636
|
| 22 |
+
160272,2625896448,0.13262036776480576,4.274688442548116,3.8085408806800842,0.0067473828224062045,0.020236863289028406,0.3512594917495638,3.314492384592692,1.1516676816312392,120.21705919250847,61.51050717473614,1.006945904514626,1.0210648338082036,1.6417888418776945,27.665262907162017,30.602042362319764,0.9169411524142973,0.26309378355859714,0.28534538470555926,0.9947916666666666,0.98046875,0.9691676795482635,1.0,0.5621134042739868,0.8541666666666666,0.15625,0.15104166666666666,0.9948518580640666,nan,nan,0.9757135567245617,0.19409710814922695,0.17015720418809602,3.2158409477576644,0.004436663681049016,0.03331525987541338,0.02489819939301631,0.0007004519346714812,0.00015352362044678553,0.0021440989331171093,3.1501927373606704,25.71316440693987,0.4102532650955193,0.011780043193471874,13.448104808018304
|
| 23 |
+
167904,2750939136,0.20308692400188497,4.392155468463898,3.860958069562912,0.05114926962348439,0.03036416469224908,0.36946763294872653,3.305552085240682,1.2945352102035084,130.76361528531913,62.17725868937553,1.0579498560621463,1.0320459850967216,2.0063486970057385,27.417042982210365,32.249828100753305,0.9290473570545141,0.2754928339790692,0.2869661869263846,0.984375,0.9739583333333334,0.9753949890534083,1.0,0.5710185170173645,0.8697916666666666,0.171875,0.14583333333333334,0.9852579309178205,nan,nan,0.9828427575606232,0.17942412175074196,0.1558770607225597,3.2051086052601403,0.004166280234126868,0.03318983181791379,0.02478970850745944,0.0006958054881541207,0.00014797814412011398,0.0021224420821680844,3.1399965454869174,25.48267727584209,0.41105042852117707,0.010995367597369719,13.413570243601669
|
| 24 |
+
175536,2875981824,0.12114340245413284,4.4413320024808245,3.8399648467699685,0.06054686377266686,0.023328572979001667,0.2520571928192415,3.3003857135772705,1.1400872955007635,140.72381882152132,62.861160177523175,1.083513275907645,1.0244530095124642,1.4441421464061441,27.2800488841217,33.65103194435617,0.9563314767078926,0.2561063389978038,0.2898429948748957,0.9895833333333334,0.98046875,0.9814153909683228,1.0,0.5622572302818298,0.9010416666666666,0.14583333333333334,0.140625,0.9916751772980206,nan,nan,0.9828359485885206,0.17442976383239284,0.149655189171123,3.1958933241436274,0.003933968648731936,0.033093418278679336,0.024696179406151535,0.0006866291309722145,0.00014302665356358594,0.002103796036882325,3.131236292182316,25.33752157503204,0.4115314221154094,0.01021072727974235,13.453167211877565
|
| 25 |
+
183168,3001024512,0.16237951310661933,4.447650194168091,3.813933491706848,0.007119645853075933,0.022873728625351458,0.2985616433037042,3.290307601292928,1.2056478278342626,147.83180848518583,62.22795391321379,1.0073910361280825,1.0241935228308086,1.4963943234129573,27.009224746790366,34.54323055077087,0.9542429890999236,0.2691300562443912,0.29363770090721125,0.9947916666666666,0.98046875,0.9771108229955038,1.0,0.5729361772537231,0.9166666666666666,0.16145833333333334,0.15625,0.9948518580640666,nan,nan,0.9806947582837893,0.17254023666934395,0.14837994917373484,3.187045335061562,0.0037322222248011333,0.032975811900061394,0.024595760766901325,0.0006784802522399248,0.00013916721696689756,0.0020825776055416237,3.1228413012407197,25.08417205065801,0.4129490759887535,0.009432097738825898,13.471683309439362
|
| 26 |
+
190800,3126067200,0.14257956917087236,4.487789809703827,3.848909765481949,0.006106162139303706,0.023615562779468746,0.3633931076619774,3.283722480138143,1.1772480604688367,148.08829193083847,61.96781571684668,1.0062956544596087,1.0244323137672977,2.611445027598089,26.83368551987218,34.672744889121596,0.9600858124428413,0.27133548833442095,0.31262616659472836,0.9947916666666666,0.9739583333333334,0.9740894883871078,1.0,0.5840960144996643,0.921875,0.171875,0.171875,0.9948518580640666,nan,nan,0.9825351140849913,0.18566403644202487,0.16204761531359205,3.178350156205651,0.0035599175763905973,0.032879625239320344,0.024503723538640474,0.0006727938789391888,0.0001361985861296912,0.0020604273343300234,3.1145374552196428,24.934221856124246,0.41360003316083793,0.008665408697250875,13.474689675636583
|
| 27 |
+
198432,3251109888,0.08426560143319269,4.547043204307556,3.885298788547516,0.005276979007987848,0.014754892459298693,0.1705772057254459,3.2773483196894326,1.092616131547125,168.3529634141593,67.74071684296034,1.005418002009366,1.015208551239373,1.3064089268616963,26.659476014312244,38.16754398329849,0.9659847896692694,0.27014333470660024,0.3050207215118685,0.9947916666666666,0.9869791666666666,0.9803332587083181,1.0,0.5807245373725891,0.921875,0.16145833333333334,0.16145833333333334,0.9948518580640666,nan,nan,0.9855562542021895,0.166097904836685,0.14313405666810772,3.170138528988359,0.0034100271410412177,0.0327903633674634,0.024426427901677584,0.0006638347732372028,0.00013335603223906244,0.0020451161505301166,3.1066693893596686,24.737044796503813,0.41439200130992215,0.007916498943711495,13.475001281486952
|
| 28 |
+
206064,3376152576,0.07872221246361732,4.648836096127828,4.020289132992427,0.0055375270807568695,0.016713079835653843,0.1661295258196939,3.271859566370646,1.0849596505955454,180.9433179347582,78.74565841387805,1.0057115355920994,1.0172648453805802,1.296519168180969,26.511674447822788,41.51501514231546,0.9600478686283423,0.273342023556414,0.2948872725773739,0.9947916666666666,0.98046875,0.9831110388040543,1.0,0.5769153237342834,0.9270833333333334,0.15104166666666666,0.14583333333333334,0.9948518580640666,nan,nan,0.9868483922230856,0.18738227339584532,0.16420802392531186,3.1629681736439115,0.003279167827652522,0.032705991922755376,0.024359984688288688,0.0006570428559978397,0.00013056982728955173,0.002020658215745906,3.099814743873521,24.60484720507258,0.4149359022608843,0.007191071867150021,13.466514963747576
|
| 29 |
+
213696,3501195264,0.09478055937991788,4.709783832232158,3.974822372198105,0.006020024260370367,0.02740865091421559,0.17665045704082635,3.2680466969807944,1.1102457248103847,204.97143797277167,75.37911707449265,1.0062047641353398,1.028777385812647,1.3033759391507,26.416889688073613,44.4594355070353,0.9591734949831481,0.2635989807676033,0.314589765881052,0.9947916666666666,0.98046875,0.9831110388040543,1.0,0.5805619359016418,0.9114583333333334,0.16145833333333334,0.171875,0.9948518580640666,nan,nan,0.9869283239919847,0.1822589272201185,0.15848975155192116,3.1562863469701767,0.003165822713964627,0.03262619478349814,0.024300686063639988,0.0006501613656245502,0.0001282469828282634,0.0020073798629975195,3.0934078410439647,24.46934814660554,0.41568909711995217,0.006494652022090374,13.515372081031654
|
| 30 |
+
221328,3626237952,0.05564352573128417,4.767223954200745,4.074032445748647,0.005701976944607168,0.011767499463985587,0.26520079632488586,3.2641029357910156,1.0582494590073648,219.94390807648537,83.62622950020116,1.005875294938555,1.0120214566539258,1.488191441839044,26.307899133478816,47.77748205180061,0.9643496122933916,0.2770781563825104,0.3288266635352447,0.9947916666666666,0.9934895833333334,0.9749411046504974,1.0,0.5970319509506226,0.9166666666666666,0.16145833333333334,0.1875,0.9948518580640666,nan,nan,0.9778725310219064,0.1797712767424855,0.15593173424713314,3.1497740617827636,0.00306690446917643,0.03254774817952835,0.024232631607080734,0.0006463206149888911,0.00012620729877176818,0.0019951631627025796,3.087159071846557,24.37180788462017,0.41616718061815017,0.005832543055900948,13.43069269974224
|
| 31 |
+
228960,3751280640,0.05956284982191088,4.844610492388408,4.149490545193355,0.004791496308077588,0.02012386647443994,0.2401009318617374,3.2612521648406982,1.0630854924232687,239.52387092669304,89.0457114205501,1.004903640236854,1.0209598939769582,1.4129300889430976,26.236415759865206,51.329696746098364,0.964520929503608,0.2856921180485217,0.3245031154124908,0.9947916666666666,0.9739583333333334,0.9798430651426315,1.0,0.5997651219367981,0.9270833333333334,0.17708333333333334,0.17708333333333334,0.9948518580640666,nan,nan,0.9816245599019263,0.1700015720269524,0.1470763991237618,3.1441738605655463,0.002976346264535055,0.03249964449864797,0.02416854361732764,0.0006454707712467261,0.0001239442244904088,0.00198015184664134,3.0817797447988866,24.23289655901851,0.4168159188722227,0.005209787318395601,13.412460768747263
|
| 32 |
+
236592,3876323328,0.10216115100774914,4.862867573897044,4.121198356151581,0.005427270183351614,0.021312140464469092,0.24026298899328444,3.2573498090108237,1.1205083763014827,245.1143251221398,88.48857909311965,1.0055771308276624,1.0221369299819882,1.4160556842491812,26.13063917234331,52.04254592985186,0.9654079030882555,0.2690754163115703,0.3149667885605009,0.9947916666666666,0.98046875,0.9798430651426315,1.0,0.5846523642539978,0.9427083333333334,0.15104166666666666,0.16666666666666666,0.9948518580640666,nan,nan,0.9816245599019263,0.18565254729204148,0.16190904619482657,3.1390497799647763,0.002898253809481311,0.03242977948673525,0.02411715381200849,0.0006435843567205162,0.00012175151288769102,0.00197253051678174,3.0768667125227065,24.164539926144563,0.4171820012989222,0.004631127461369502,13.418386138169499
|
| 33 |
+
244224,4001366016,0.05439300693493957,4.881244758764903,4.113968590895335,0.0054191545071470655,0.01971897355178953,0.16209176990075017,3.254489223162333,1.0577121973080286,253.60798457586793,88.47711538482997,1.0055685975895652,1.020519453642303,1.2982611800950972,26.058268399313324,53.21791854123517,0.9750086041710007,0.27582094501509496,0.32730232339767157,0.9947916666666666,0.98046875,0.9878280212481817,1.0,0.5960526466369629,0.9583333333333334,0.16145833333333334,0.18229166666666666,0.9948518580640666,nan,nan,0.9898607616681451,0.1819965935081503,0.15932716235208014,3.1344910616577093,0.002831463345409585,0.03237489579378215,0.024065902002328575,0.0006459786906097563,0.0001196492374761094,0.001973482417983308,3.072479675921587,24.068102303540932,0.4177041126208173,0.004100970320511953,13.622003899710506
|
| 34 |
+
251856,4126408704,0.0467980173319423,4.944245477517446,4.179764896631241,0.004900098580407075,0.019034010900403093,0.24373256347992842,3.2525571982065835,1.048797291466813,278.6343132704299,92.98522420287416,1.005028062060709,1.0198500549663836,1.424939549090845,26.01028744690676,57.446919982542234,0.9706458223850776,0.2837628889363253,0.3220040491733791,0.9947916666666666,0.98046875,0.9798430651426315,1.0,0.598004937171936,0.9479166666666666,0.15625,0.18229166666666666,0.9948518580640666,nan,nan,0.9816245599019263,0.181772785055576,0.15742368035716936,3.1311959806702125,0.002784832299639255,0.03233674469558167,0.02402689875344987,0.0006437085256925218,0.0001185106376854683,0.0019680885587245642,3.06931718242056,24.123009983286167,0.41743835758425657,0.003623353354756449,13.50521380695882
|
| 35 |
+
259488,4251451392,0.050532187218777835,4.924728512763977,4.1567807992299395,0.0053336984765242805,0.016987821746776415,0.18201990276293145,3.251630266507467,1.0529276197948085,272.0605156130878,90.17843514820622,1.0054833052563554,1.017624918885698,1.3171200869370767,25.985940601325975,56.08829247049913,0.9645393733145867,0.2775517292585347,0.32574993513779454,0.9947916666666666,0.9869791666666666,0.9782090783119202,1.0,0.5957325100898743,0.9270833333333334,0.16145833333333334,0.17708333333333334,0.9948518580640666,nan,nan,0.9830443823787695,0.18253399743586365,0.15866276947781444,3.128130867465508,0.002745857320970158,0.0323062962805234,0.024005479975152826,0.0006410474922714187,0.0001173392898426783,0.0019637955229461507,3.0663510368919558,24.019956037933223,0.4178311824095699,0.003201913898652705,13.545875509432566
|
| 36 |
+
267120,4376494080,0.04953452339395881,4.951195478439331,4.1667807797590894,0.005315140678722703,0.017698499289243348,0.24398814804741656,3.2506314516067505,1.051766529978898,281.5937007233633,93.6190421018265,1.0054645540876204,1.0183821959876802,1.4267605267303878,25.961743380156964,57.953837144590196,0.9695684098411199,0.28827362236604764,0.3243687572553855,0.9947916666666666,0.98046875,0.9798430651426315,1.0,0.6019458770751953,0.9427083333333334,0.17708333333333334,0.17708333333333334,0.9948518580640666,nan,nan,0.9816245599019263,0.17888729060708405,0.15479862575496858,3.1256200939984398,0.0027130933080844297,0.03228212491421069,0.02397375441730513,0.0006411848754240784,0.00011687477629639973,0.0019591102324735786,3.063933937416616,24.006150538504,0.417903859734504,0.0028398614619212603,13.472470060490346
|
| 37 |
+
274752,4501536768,0.04982198623474687,4.961586217085521,4.142819941043854,0.005329615917152826,0.01652546056728473,0.24143616069083387,3.248838027318319,1.052128856786956,285.6595178117115,92.67689258982487,1.0054790439160477,1.0171582080783124,1.4182068487584683,25.91457343654299,58.3919938279456,0.9662063970129887,0.288860017942487,0.3381342635414702,0.9947916666666666,0.9869791666666666,0.9798430651426315,1.0,0.609052300453186,0.9375,0.171875,0.19270833333333334,0.9948518580640666,nan,nan,0.9816245599019263,0.19060324112539698,0.16681248327950016,3.1237322904330957,0.0026924959433428227,0.03224773103681052,0.023950425333051882,0.0006370699922534879,0.00011669813552302314,0.001970483099922679,3.062117373160784,23.964179316219294,0.4179919959988984,0.0025399532871463355,13.511020875335413
|
| 38 |
+
282384,4626579456,0.049017109733540565,5.003775854905446,4.205683211485545,0.005050867043649987,0.018215587791511705,0.16978333153216832,3.248801589012146,1.05120762338489,300.0364138541116,98.95076476927385,1.0051843122553883,1.0189056381816346,1.3050577794150449,25.912725447772058,61.32575134634207,0.9697929213041837,0.28912243698997214,0.3264716181594399,0.9947916666666666,0.98046875,0.9831110388040543,1.0,0.603693962097168,0.9427083333333334,0.171875,0.18229166666666666,0.9948518580640666,nan,nan,0.9867741264170036,0.183521806192554,0.1599297379143536,3.1221865799218147,0.0026724799976005553,0.03224296811307506,0.02394941518595572,0.0006353718758218504,0.00011676150291697653,0.001970211262497655,3.0605993582668582,23.933824640025133,0.4182723964285301,0.002304473351749283,13.487430405574221
|
| 39 |
+
290016,4751622144,0.05921137984842062,5.001575569311778,4.2157248655955,0.005291130145148297,0.017655314391845423,0.24584704492008314,3.2482312520345054,1.0628961005702857,299.4355131211512,98.8410667627637,1.0054404701405182,1.018301612396301,1.4314429706028202,25.898506828291747,61.24188112370238,0.966141871539696,0.2895439421552302,0.32857803847222294,0.9947916666666666,0.98046875,0.9798430651426315,1.0,0.6046003699302673,0.9270833333333334,0.16666666666666666,0.1875,0.9948518580640666,nan,nan,0.9816245599019263,0.18402540424464556,0.15958443252990642,3.121029822271679,0.0026647823779706223,0.03223439818825755,0.023930153510110894,0.0006315312826903572,0.00011651828693989148,0.001961513846485449,3.0594909110508084,23.90244233745699,0.41846266190523373,0.0021352149741550937,13.483793954031816
|
| 40 |
+
297648,4876664832,0.0593097356807751,5.006396929423015,4.163686186075211,0.0052114078862265005,0.021088758000663194,0.24294593074667623,3.246702194213867,1.0629957876922254,302.1986341974028,93.97158547076765,1.0053506314180274,1.0219054166942685,1.4225883578608955,25.857392562823154,60.93435034637986,0.96369291598825,0.2915066407580443,0.32526040698198744,0.9947916666666666,0.98046875,0.9798430651426315,1.0,0.604056715965271,0.9270833333333334,0.171875,0.17708333333333334,0.9948518580640666,nan,nan,0.9816245599019263,0.18541285935179985,0.16155579154534885,3.119845472320466,0.0026478236473909136,0.03221506190422472,0.02392004827944696,0.0006322042431458464,0.00011619742741980124,0.001961541063394972,3.058352583054789,23.944361612606848,0.4183257148802218,0.0020334671566161833,13.569662698893685
|
training_logs/run_metrics_text67/09_fone_medium/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: False
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 10
|
| 5 |
+
cache_base_path: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/cache
|
| 6 |
+
combine_strategy: sum
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty_sd
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/configs/config_FoNE_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_stem
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 1024
|
| 45 |
+
n_head: 16
|
| 46 |
+
n_layer: 24
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: fone
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: mse
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/configs/config_FoNE_text67.py --tqdm --verbose --seed 42 --run_name fone_gpt2_medium_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 24 --n_head 16 --n_embd 1024 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/trained/fone_gpt2_medium --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group fone_gpt2_medium_text67 --tie_word_embeddings --tokenizer_dir /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_repo/tokenizers/num_text/bittoken_gpt2 --model rope_stem --num_embedding_type fone --base 10 --dropout 0 --cache_base_path /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/cache --compile --difficulty_column difficulty_sd --train_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Division_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard --train_set_ratios 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized_number_pos --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Division_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos pretokenized_number_pos --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Sat Oct 3 19:43:18 2026'}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: fone_gpt2_medium_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/trained/fone_gpt2_medium
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_repo/tokenizers/num_text/bittoken_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/cache/bittoken_gpt2_47200109_pos/96841076'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/cache/bittoken_gpt2_47200109_pos/63050008'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/cache/bittoken_gpt2_47200109_pos/44316829'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/cache/bittoken_gpt2_47200109_pos/48579485'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/cache/bittoken_gpt2_47200109_pos/63589654'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/cache/bittoken_gpt2_47200109_pos/28473695'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/cache/bittoken_gpt2_47200109_pos/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized_number_pos
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'pretokenized_number_pos']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/mnt/bn/search-gec-vagcp/mlx/users/<user>/willy/yhc_v2/.formal-control/20261003-useast1b-r7/case-09/native/paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: fone_gpt2_medium_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/09_fone_medium/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,0.13235789514146745,0.00010834114800672978,2.4097164896375034e-05,1.4564536741090706e-05,0.7564137876033783,0.0006599505577469245,4.4269964865719285,1.1510463241973212,1.000108347074586,1.0000240974595997,1.0000145646487668,2.1307066007801048,1.0006602011009929,79.6377446635435,12.417186399829268,0.23876764905088704,0.03334807004813906,0.02812894747785022,0.020833333488553762,0.05598958022892475,0.035556258633732796,0.6815837897005198,0.042378634214401245,0.03125,0.010416666666666666,0.005208333333333333,0.35773611490124796,nan,nan,0.35303723497903167,nan,nan,5.299636304128595,0.05447534619063233,0.057295620751877614,0.0544299585193169,0.029091462484644868,0.004363751555090102,0.06424711316001976,5.035733021889468,6525.294929140019,0.2661259587635807,0.004997126321997158,27.27594553338048
|
| 3 |
+
15264,250085376,0.04601848137099296,0.00010089273564517498,7.849880057619885e-05,0.00028095601010136306,0.7267932891845703,0.0005899467359995469,3.9444266646169126,1.0481000523308945,1.0001008978373966,1.0000785018823084,1.000280995921064,2.0684902504572618,1.000590126163872,50.78425682382386,8.271699664059522,0.4392646885647733,0.048199991225602465,0.05346371268911047,0.0989583358168602,0.0625,0.09784555435180664,0.8989397750506463,0.0884222760796547,0.2864583333333333,0.005208333333333333,0.005208333333333333,0.43383457307936624,nan,nan,0.4427930347737856,nan,nan,4.632271345025338,0.037438711027133496,0.04566471241497526,0.04338185221154,0.021021537064668378,0.003976704394864602,0.04495540003287869,4.4358324031238165,55.731799541904245,0.36171057279098706,0.009998084214664627,16.38897675115103
|
| 4 |
+
22896,375128064,0.30714061856269836,0.0005242702609393746,0.0004230051126796752,0.0005305068334564567,0.7205077707767487,0.0019074466545134783,3.79723412093396,1.3629780646157448,1.0005244080702267,1.0004230946045687,1.0005306476194538,2.055517338253829,1.0019100334565314,43.81357286347137,7.319350921441675,0.3260646023123841,0.01929088349067301,0.016387396120191012,0.0729166679084301,0.1015625,0.08308312110602856,0.964601839923668,0.04588340222835541,0.1875,0.0,0.0,0.40944293823364813,nan,nan,0.40422191657247214,nan,nan,4.307906858930215,0.027707242390593537,0.04017614924227187,0.03820358254184658,0.01692084896459246,0.003830384629518887,0.03500738184956689,4.1460612474945275,40.99970099473649,0.38371555951953057,0.014999042107332095,15.900424494656033
|
| 5 |
+
30528,500170752,0.13537264242768288,3.18323718602187e-05,2.4125404706865083e-05,5.30377419636352e-05,0.7216843962669373,0.0007343611214309931,3.714190335245803,1.1465518700486907,1.000031832882026,1.0000241256969202,1.0000530391487596,2.057906311576362,1.0007346355869193,40.50646702419906,6.81596697701982,0.39796971319621754,0.028204756141581043,0.021130358059257552,0.0677083320915699,0.08203124068677425,0.08440002053976059,0.9845170569259669,0.055947717279195786,0.22395833333333334,0.005208333333333333,0.0,0.4277931033769467,nan,nan,0.42534557245016913,nan,nan,4.1188117487773885,0.022387019358163496,0.03725936298077512,0.03546066561433637,0.014689681379070098,0.0037570503661664063,0.02962984449252832,3.9756281035922982,36.76457992920835,0.39309984588882335,0.020000000000000004,15.623998114807103
|
| 6 |
+
38160,625213440,0.08947573858313262,5.981178219371941e-06,4.730374939754256e-06,5.9715153838624246e-05,0.7174931466579437,7.317194285860751e-05,3.623997882280188,1.0979469530855868,1.000005981196109,1.0000047303861477,1.0000597169860015,2.0493540599902103,1.0000731747714766,37.06568988417923,6.316162071513538,0.678011173308888,0.06641994161519528,0.06411919481145806,0.1822916641831398,0.0572916604578495,0.11570269614458084,0.9902940889307081,0.10815908014774323,0.5677083333333334,0.020833333333333332,0.0,0.5277094720562067,nan,nan,0.49561813004887273,nan,nan,3.991257126173258,0.019198876569173018,0.03563112935150897,0.033915659299206216,0.013298303090315116,0.00371188023909819,0.026305007197395303,3.859196248925457,34.17934918253677,0.40015760950150603,0.019965729849129636,15.594971694361686
|
| 7 |
+
45792,750256128,0.001218551944475621,1.985141079785535e-05,5.25244718119211e-06,0.00022994653409114107,0.7239018380641937,0.0001856117887655273,3.5672271854127757,1.0012193937218192,1.0000198516189367,1.0000052524610155,1.0002299748737848,2.0624867689337982,1.000185629050581,35.02915726151864,6.013329161739796,0.7482767264767657,0.07302648777252005,0.06120061951131903,0.1458333283662796,0.0494791604578495,0.13490308076143265,0.9938657362868036,0.10470211505889893,0.703125,0.036458333333333336,0.005208333333333333,0.526147716678679,nan,nan,0.5008848825164023,nan,nan,3.8931177135929205,0.016628290950059115,0.03420758551446452,0.03257408712046569,0.01224555617101864,0.0036810193392805266,0.023826087851216107,3.76995506659046,31.647773513753958,0.40701453368993673,0.019863180383909898,15.529863247165258
|
| 8 |
+
53424,875298816,0.07599371235119179,2.2332908883981872e-05,1.2555508419609396e-05,9.848238914855756e-05,0.7169036269187927,0.000241560279391706,3.526977457862813,1.0820345986678794,1.000022333160336,1.0000125555872543,1.0000984873881262,2.0480918974736086,1.0002415916304241,33.653871631487604,5.82633901362789,0.8169176217223915,0.078586354206896,0.07521586367149721,0.1875,0.07552083022892475,0.15123261883854866,0.9971087059441789,0.1310027837753296,0.8072916666666666,0.03125,0.005208333333333333,0.5485598840362703,nan,nan,0.5298285286213892,nan,nan,3.568929287005583,0.00813468358377587,0.029152227346889496,0.02779248469679374,0.008517435333358156,0.0035409353091828927,0.014973105911365114,3.4768183974172997,30.131092782290477,0.41192338740613227,0.019693132578928373,15.52993449663699
|
| 9 |
+
61056,1000341504,0.00016052171849878505,7.786671631038189e-06,5.40425639883324e-06,3.262154496042058e-05,0.7220441401004791,3.764754183066543e-05,3.4962098289591568,1.000160534823741,1.0000077867023314,1.0000054042710231,1.0000326220908173,2.058638439585067,1.0000376482632598,32.63933478882313,5.671173889222766,0.8265866818491769,0.09380101554778536,0.07594420555259015,0.2343749925494194,0.0559895895421505,0.1704198271036148,0.9991687465321138,0.12954795360565186,0.8489583333333334,0.046875,0.005208333333333333,0.5783775469074802,nan,nan,0.5416530531753475,nan,nan,3.4610757152676834,0.005241601053491277,0.027889684046446823,0.02662716319268422,0.007432790312432481,0.003529222147846526,0.012467848110378144,3.3778873895542927,29.043799084687382,0.415346096310788,0.01945688144839207,15.507833916479523
|
| 10 |
+
68688,1125384192,3.6595994970411994e-05,5.927091251578531e-06,3.6563978937920183e-06,1.5256623100867728e-05,0.5745592713356018,1.3194362509239e-05,3.471505629344999,1.0000365967028335,1.0000059271092157,1.0000036564048798,1.0000152567521918,1.77642649625345,1.0000131944496657,31.857616570868142,5.519159671220054,0.7837995189034146,0.08358977173022808,0.08026027511326786,0.2239583358168602,0.407552070915699,0.1956120952963829,0.9991687465321138,0.1807372123003006,0.7395833333333334,0.03125,0.015625,0.5593234031499984,nan,nan,0.5511425316411381,nan,nan,3.3997591392460564,0.004381464220392708,0.0274447018123127,0.02621224436202496,0.007012325775322529,0.0035025184675266906,0.011680716589412273,3.31952515119006,28.256114202225483,0.41842791254181155,0.019156226183831337,15.491068369321637
|
| 11 |
+
76320,1250426880,0.0005240773316472769,9.394301059728605e-06,9.806050911720376e-06,7.431135236402042e-05,0.5291979312896729,0.00011488162272144109,3.4493791552182906,1.0005242178939429,1.00000939435289,1.000009806108596,1.000074314162605,1.697923143664179,1.000114890365139,31.160475025799144,5.4084472560495,0.7048248756871281,0.07265071101366356,0.07033370430051944,0.3125,0.498697891831398,0.20750369131565094,1.0,0.1731574684381485,0.6822916666666666,0.015625,0.010416666666666666,0.5954090419303005,nan,nan,0.5548397189704701,nan,nan,3.3523344586074226,0.0037977090193885158,0.027098183850510466,0.025871897877111333,0.00663631022228094,0.003359260639879837,0.011090472853470172,3.2744806069935395,27.566535178907763,0.42113023713246706,0.018793456452196412,15.512256070389292
|
| 12 |
+
83952,1375469568,0.00010841689800145105,6.329817551886663e-06,3.998746251454577e-06,0.0003137629755656235,0.4551006555557251,2.3782491552992724e-05,3.4819365174043924,1.000108423746563,1.0000063298376172,1.000003998754249,1.0003138559872746,1.5764889876883457,1.0000237828251142,32.016920886177694,5.513409466430979,0.7364633649307455,0.0800184507930499,0.08056390583263798,0.3072916716337204,0.596354141831398,0.2385561540722847,0.9991687465321138,0.19235378503799438,0.6822916666666666,0.010416666666666666,0.005208333333333333,0.5995277639788886,nan,nan,0.5675265418055156,nan,nan,3.31310342867409,0.003167035850310004,0.026614517448675004,0.025427298524510047,0.006237517980691341,0.0031539441116167537,0.010495449416156355,3.238007648954081,27.01011957737385,0.4235930654651776,0.018371334958694578,15.430288054875149
|
| 13 |
+
91584,1500512256,0.0001181212137453258,4.451254994819465e-06,3.73401167053089e-06,8.066188820521347e-05,0.37610216438770294,5.59113086637808e-05,3.410620870922382,1.0001181283417782,1.0000044512652044,1.0000037340191468,1.0000806651458243,1.4567756026747478,1.0000559134471172,29.985944565996736,5.206140437270079,0.8236271547986733,0.09020345920024635,0.09055691674864588,0.40625,0.583333358168602,0.32156024873256683,1.0,0.22131846845149994,0.8333333333333334,0.03125,0.010416666666666666,0.6591369978462657,nan,nan,0.6166269177338108,nan,nan,3.2842288065225325,0.0029525596005190863,0.026425175495776995,0.025252733283362513,0.005924825713606772,0.002894339498783307,0.010094396289939791,3.2106847596529855,26.55856596273946,0.42501298867291,0.017893076407163026,15.471365076102428
|
| 14 |
+
99216,1625554944,9.315971874457318e-05,4.491029812925262e-06,4.62417528979131e-06,2.398272158643522e-06,0.2884766459465027,0.0001764593744155718,3.4301240201651426,1.000093167008564,1.0000044910400256,1.0000046241860496,1.000002398275039,1.336332323143842,1.0001764903043031,30.46881875117805,5.257918892162268,0.7392414660073133,0.08501084050652907,0.08224621860585282,0.3802083432674408,0.759114608168602,0.28309366106987,1.0,0.2073802649974823,0.703125,0.036458333333333336,0.010416666666666666,0.6200760198601832,nan,nan,0.5915316539661338,nan,nan,3.260870575529974,0.002799262828663415,0.026273901869460246,0.025114494447230608,0.005626942217850324,0.0025680530992724256,0.009698861798842306,3.188789042081244,26.16246623131964,0.4268838854885389,0.017362323018206012,15.549704497574515
|
| 15 |
+
106848,1750597632,0.00012656948456424288,2.430693029964459e-06,1.8935738239633793e-06,0.00014983761138864793,0.21138878911733627,3.889468825946096e-05,3.384137467830442,1.0001265811433948,1.0000024306959956,1.0000018935756234,1.0001498543336793,1.2359690924883404,1.0000388954863018,29.206363456063485,5.063236029112403,0.8387471644080495,0.09310968498276682,0.08811343697346051,0.4583333283662796,0.78515625,0.3408133238554001,1.0,0.22822944819927216,0.8645833333333334,0.036458333333333336,0.015625,0.6909767019290788,nan,nan,0.6230716941645369,nan,nan,3.2403530734692065,0.002672193590570122,0.02615044564264159,0.024990783306326817,0.005326955332347143,0.0021868946720968088,0.00931248187891157,3.169713301712987,25.658711731746262,0.4286707828418826,0.016783116791537683,15.479952391607743
|
| 16 |
+
114480,1875640320,0.00018307466962141916,2.9495168973880936e-06,2.77269498383248e-06,5.4327812904375605e-05,0.16199412569403648,2.9784355319861788e-05,3.3678344825554327,1.0001830986016913,1.0000029495212512,1.0000027726988736,1.0000543292893695,1.1769245401534145,1.0000297848148962,28.73557149906644,4.987538424877991,0.8557847684999462,0.09210870500733215,0.10038396936809273,0.484375,0.84375,0.3912636935710907,1.0,0.24311862885951996,0.8697916666666666,0.041666666666666664,0.015625,0.7027895390056074,nan,nan,0.6547489165095612,nan,nan,3.2217578340652393,0.002532284346048222,0.026006111111765262,0.024849911344678233,0.005027128531517666,0.0017895409387575212,0.008911124866178056,3.1526417143793894,25.272040232047594,0.4305202323607554,0.016159868723772935,15.414821759986046
|
| 17 |
+
122112,2000683008,2.892869815696031e-05,5.033624347561272e-06,5.440615723273368e-06,5.718287184208748e-06,0.13400626182556152,2.2015095964889042e-05,3.356891109627516,1.000028929160178,1.0000050336370787,1.0000054406305245,1.0000057183046036,1.144428115217024,1.0000220154176398,28.431952773797438,4.939492575166355,0.8423691826891042,0.1063199578463347,0.08802465034669363,0.5989583432674408,0.869791679084301,0.4312710762023926,1.0,0.24918882548809052,0.8802083333333334,0.041666666666666664,0.005208333333333333,0.7375979168961445,nan,nan,0.6760261316473285,nan,nan,3.204343609655033,0.0024226623598079342,0.02584863992493826,0.024716218450229685,0.004736182586446179,0.0014761558136625326,0.008505917747179518,3.1366378153712704,24.83980491276307,0.4319254577362937,0.015497325216091833,15.460938724176179
|
| 18 |
+
129744,2125725696,7.668345097044948e-05,3.011191665791557e-06,2.142680045835732e-06,0.00041150198194372933,0.13185294345021248,1.4376992567122215e-05,3.341886788587241,1.0000766868435438,1.0000030111962852,1.0000021426823418,1.0004116623607187,1.1412252444387327,1.000014377096841,27.996778932192186,4.8769302938300925,0.8469643544512302,0.09367426252877151,0.09489772992542715,0.6041666865348816,0.889322891831398,0.4564501643180847,1.0,0.24679100513458252,0.8854166666666666,0.036458333333333336,0.010416666666666666,0.7519197963313976,nan,nan,0.6983198780023182,nan,nan,3.1887180467040532,0.002331073786785948,0.02571671762178925,0.024591099120712966,0.0044358064437595105,0.0012002272761631539,0.008090370730738965,3.122352734111786,24.602072385247148,0.4330609944381554,0.014800531927597655,15.462276255303726
|
| 19 |
+
137376,2250768384,0.00010291655780747533,5.952126230113208e-06,5.100455155115924e-06,3.857173396681901e-05,0.08321518450975418,3.311949967610417e-05,3.330777797207702,1.0001029219914055,1.0000059521442735,1.0000051004681714,1.0000385724795846,1.0869152050541713,1.0000331200544958,27.69020059282941,4.825328780717358,0.8510224016335981,0.09949809326669753,0.08676415144892044,0.6822916567325592,0.90234375,0.5333438515663147,1.0,0.24822841584682465,0.9010416666666666,0.052083333333333336,0.010416666666666666,0.797535891411826,nan,nan,0.7337210381325955,nan,nan,3.1728630042254173,0.0022459816924256214,0.025608034599455272,0.024492147369186607,0.004111604553459903,0.0009556341070399843,0.007651398155799528,3.1077981852203256,24.174059481990664,0.43512401264356115,0.014074795349650518,15.453219203898572
|
| 20 |
+
145008,2375811072,0.0013516892067855224,5.634933813780663e-06,4.910161351290299e-06,0.00035787200613413006,0.06450404226779938,0.0005725589689973276,3.3152390842636428,1.0013531137201093,1.00000563494989,1.0000049101735944,1.0003579414056387,1.066974967406556,1.0005728318306368,27.266749780866707,4.762288454336162,0.8622945037222672,0.09352896660187386,0.09880692252799073,0.7083333432674408,0.947916679084301,0.5379135012626648,1.0,0.25642842054367065,0.8854166666666666,0.057291666666666664,0.020833333333333332,0.81414574990049,nan,nan,0.7292796858431151,nan,nan,3.157585430208263,0.0021290409886122828,0.025460743909987413,0.024358690010105825,0.0037645235195040705,0.0007600769794572119,0.007180930065057581,3.0939314055701055,23.885543205203252,0.43597968955537936,0.013325642393809753,15.45235480259239
|
| 21 |
+
152640,2500853760,8.16054998722393e-05,5.263493903839844e-06,4.736712526209885e-06,8.090409028227441e-05,0.036310103721916676,3.720979748322861e-05,3.30540965043474,1.000081608830922,1.0000052635079035,1.0000047367239913,1.0000809074115193,1.0371274396276324,1.0000372105303075,27.004320881693705,4.720236864046568,0.8630858533359843,0.09962818307525045,0.10712223038325436,0.7291666567325592,0.967447929084301,0.5823193192481995,1.0,0.27286797761917114,0.90625,0.052083333333333336,0.015625,0.8207198107459893,nan,nan,0.760595288205271,nan,nan,3.143330509604523,0.0020526274364104182,0.025356722917377814,0.024257047349248006,0.0034173077855364615,0.0006080716980102376,0.0066970809619739695,3.0809416320021343,23.588490820905697,0.43754158769607293,0.012558778301146429,15.493385870503511
|
| 22 |
+
160272,2625896448,8.956090459832922e-05,6.192481578182196e-06,5.000526471121702e-06,3.801667753577931e-05,0.02125491527840495,3.0776435778534506e-05,3.2946355480429097,1.0000895649179684,1.0000061925007842,1.000005000539371,1.000038017438806,1.0215482038489563,1.000030776938895,26.72625569969813,4.6782819222689875,0.8619707648882402,0.1033256856524134,0.10872633361380482,0.7708333134651184,0.986979179084301,0.5957516729831696,1.0,0.27972349524497986,0.8958333333333334,0.041666666666666664,0.03125,0.8274255217596268,nan,nan,0.7566469986146936,nan,nan,3.129855700279765,0.0019481109651205847,0.02523830175150191,0.024157039362645648,0.0030692008221646443,0.0004894861910734895,0.00619566392084535,3.068757879370101,23.32244530345159,0.43860634635017104,0.011780043193471874,15.521733483762835
|
| 23 |
+
167904,2750939136,0.0001213115465361625,5.93851746089058e-06,5.064514198238612e-06,4.813975465367548e-05,0.031804488971829414,2.7570063139137346e-05,3.284940153826028,1.0001213199061776,1.0000059385352977,1.0000050645275365,1.0000481409136377,1.032349904235394,1.0000275704566348,26.460304554656304,4.641837499032997,0.885191857235914,0.10772514618822408,0.09702766623149603,0.8020833432674408,0.954427108168602,0.6385113894939423,1.0,0.2736559212207794,0.9583333333333334,0.041666666666666664,0.020833333333333332,0.8592174919322133,nan,nan,0.7816293054881195,nan,nan,3.11727042886696,0.0018014302558592987,0.025137962964051002,0.024061677631613577,0.0027558980671835184,0.0003966731134111621,0.005714644888721167,3.0574021235441595,23.06562030240425,0.43989325700175463,0.010995367597369719,15.466896637111137
|
| 24 |
+
175536,2875981824,0.0003371396305738017,5.3677736104873475e-06,5.136557774676476e-06,8.841161979944445e-05,0.016749456524848938,5.4841701057739556e-05,3.275648351661706,1.0003372010044558,1.000005367788066,1.0000051365712839,1.000088415701938,1.0168906514292289,1.0000548432054865,26.229449731367904,4.606690192438338,0.8976392185027535,0.10972213168264712,0.10156546449200887,0.859375,0.98046875,0.7023546099662781,1.0,0.2836170494556427,0.9583333333333334,0.052083333333333336,0.020833333333333332,0.8776962847138444,nan,nan,0.8074304353600988,nan,nan,3.104740551711274,0.0016630458428174334,0.025038578377266703,0.023970061133288312,0.002488071599754641,0.00032593547889387076,0.0052713448287852295,3.0459834965795887,22.829031526230168,0.44108035800045264,0.01021072727974235,15.453477244896412
|
| 25 |
+
183168,3001024512,9.500277337792795e-05,6.08412983638118e-06,5.898023118788842e-06,5.429562406789046e-05,0.012107837479561567,2.5095803721342236e-05,3.2659425454912707,1.000095008322505,1.0000060841483616,1.0000058980412048,1.0000542970994148,1.012215145734972,1.0000250961213792,25.970005026747344,4.568915222316455,0.87517743084228,0.10310032147454762,0.10601863803688483,0.8333333432674408,0.986979179084301,0.7007611691951752,1.0,0.28096166253089905,0.9166666666666666,0.041666666666666664,0.03125,0.864552647418653,nan,nan,0.8098344183526933,nan,nan,3.0938915261318014,0.0015272080142515788,0.024941132758281328,0.02387195378561471,0.0022581171987338974,0.0002747320818813844,0.004870695457532428,3.036147669587703,22.665216773873354,0.44170030127562565,0.009432097738825898,15.425022972886309
|
| 26 |
+
190800,3126067200,5.85161178605631e-05,7.355433808697853e-06,7.970891601871699e-06,4.075733886566013e-05,0.010348414769396186,2.6902451281785034e-05,3.259714267294233,1.0000585178723038,1.0000073554609272,1.0000079709240755,1.000040758169905,1.010420480345524,1.0000269028148097,25.813154261979335,4.54624517822384,0.8898652414284597,0.10620778870926709,0.10226200006933911,0.859375,0.986979179084301,0.7422434687614441,1.0,0.28177890181541443,0.9583333333333334,0.041666666666666664,0.020833333333333332,0.8775800758351883,nan,nan,0.83576201259469,nan,nan,3.083878840350401,0.0013964876584945587,0.02484954582700723,0.023781316230726997,0.0020539795520348432,0.00023863614469403445,0.0044990362813133716,3.0270598215642592,22.50965128179866,0.44254688179780355,0.008665408697250875,15.423181106926302
|
| 27 |
+
198432,3251109888,8.621569213573821e-05,7.625417310919147e-06,7.158338348745019e-06,4.21473214373691e-05,0.008773281006142497,3.056052446481772e-05,3.2530833885927377,1.0000862200563545,1.0000076254463846,1.000007158365011,1.0000421482206652,1.0088191322791955,1.0000305610347497,25.64404053953865,4.521861912134431,0.8940074005495449,0.11558819934307481,0.11283175759903244,0.9114583432674408,0.993489570915699,0.7451143264770508,1.0,0.3033128082752228,0.9635416666666666,0.0625,0.020833333333333332,0.8935772029993435,nan,nan,0.8371002726101627,nan,nan,3.0748575210212037,0.0012691442611362112,0.024745641806820873,0.02368493280352362,0.0018770527234833827,0.00021232286597823484,0.004175589615137024,3.0188928204142784,22.350418293501097,0.4433520127408035,0.007916498943711495,15.492516824482248
|
| 28 |
+
206064,3376152576,0.0001209895417559892,8.370514478883706e-06,7.76157889958995e-06,8.174216418410651e-05,0.005860900739207864,3.223916883143829e-05,3.2472575296997093,1.0001209975886698,1.0000083705495122,1.000007761611244,1.000081745706848,1.0058781157159156,1.000032239690042,25.488261854772656,4.499198726519269,0.8879062039410397,0.11467393425986305,0.10027457091840583,0.8854166865348816,0.993489570915699,0.776351124048233,1.0,0.28873980045318604,0.9635416666666666,0.0625,0.005208333333333333,0.8906735383595029,nan,nan,0.848174340557307,nan,nan,3.0659129366039144,0.0011670527138232234,0.024658040248900837,0.023585316036996422,0.0017248116479231198,0.0001896070629223806,0.0038934213746792054,3.0106946701453356,22.14261500184391,0.4444848624579374,0.007191071867150021,15.44618881557396
|
| 29 |
+
213696,3501195264,9.392480569658801e-05,1.151747983385576e-05,1.058245834428817e-05,4.615994112100452e-05,0.004784784454386681,3.0451359634753317e-05,3.2419642617266313,1.0000939292266326,1.0000115175461604,1.0000105825168162,1.0000461610085325,1.0048006377006278,1.000030451833359,25.359703398719976,4.480670954078872,0.8930180487884487,0.12717908415065837,0.10813967591561897,0.921875,1.0,0.801582932472229,1.0,0.31025204062461853,0.9583333333333334,0.057291666666666664,0.020833333333333332,0.9085853916282455,nan,nan,0.8552957110417386,nan,nan,3.0581585884843983,0.0010980147513268632,0.024566938060044247,0.02349469707732087,0.0015869306701529035,0.00017221405349236032,0.00363230112699819,3.0036074754714983,22.031510181209576,0.44484147198295193,0.006494652022090374,15.464262347675122
|
| 30 |
+
221328,3626237952,0.00011143562733195722,1.2010079444735311e-05,1.0949680472549517e-05,5.0582790208864026e-05,0.004179078969173133,3.3319251087959856e-05,3.237017939973157,1.0001114418375532,1.000012010151592,1.0000109497425476,1.0000505840706213,1.0041886831528275,1.0000333198181297,25.240974338964154,4.463625903962489,0.8991732206746146,0.12163500506688578,0.10353226762493989,0.9270833134651184,1.0,0.8177017271518707,1.0,0.3005082905292511,0.9635416666666666,0.046875,0.020833333333333332,0.9145613939811786,nan,nan,0.863688493690764,nan,nan,3.0513711587443253,0.0010407025766277914,0.02447476978856318,0.02339602505505426,0.0014656270853651383,0.0001569476083064549,0.003401292391739015,2.997435776941129,21.907736434626628,0.4454936405030749,0.005832543055900948,15.482421057673163
|
| 31 |
+
228960,3751280640,0.00014658071449957788,1.4425974313780898e-05,1.3122927612130297e-05,5.5992009947658516e-05,0.003809649031609297,4.143882688367739e-05,3.2333174396383884,1.0001465917168249,1.000014426078467,1.0000131230176805,1.0000559935881186,1.0038179566723038,1.0000414397024993,25.1488719156095,4.4504230637693425,0.8980652565905414,0.12401206331838323,0.12076074142790304,0.9479166865348816,1.0,0.842959851026535,1.0,0.32259079813957214,0.9739583333333334,0.052083333333333336,0.036458333333333336,0.9193261338708302,nan,nan,0.8710707284820577,nan,nan,3.0452857334544245,0.0009975336725311732,0.024387237474899598,0.023307198756859278,0.001356046971282149,0.0001442358350257471,0.0031949349271405986,2.991898528313893,21.848918722610552,0.4458059674593575,0.005209787318395601,15.383750875532336
|
| 32 |
+
236592,3876323328,0.00016292083455482498,1.594581044628285e-05,1.5691320641053608e-05,5.1412545872153714e-05,0.0017264880007132888,3.906251185981091e-05,3.229500343906693,1.0001629375712633,1.0000159459376345,1.00001569144837,1.0000514138694943,1.0017280817511578,1.0000390633090852,25.052892861739384,4.436415142232341,0.9011053817143839,0.12558149595359153,0.10898724849315257,0.9479166567325592,1.0,0.836954802274704,1.0,0.31114092469215393,0.984375,0.052083333333333336,0.020833333333333332,0.9175620031698296,nan,nan,0.8723430555934707,nan,nan,3.0396475359058144,0.0009613516358805512,0.024304185678728787,0.023218838228882502,0.001263412424141521,0.00013363592320008428,0.003016735580720726,2.986749358601467,21.762329977263445,0.4460749391981828,0.004631127461369502,15.369732486674968
|
| 33 |
+
244224,4001366016,0.00014760980047867633,1.4491358797386056e-05,1.3631962701765588e-05,5.302381578076165e-05,0.0023262082831934094,3.469756120466627e-05,3.2270277866434,1.0001476227095585,1.0000144914638132,1.0000136320592725,1.0000530252308582,1.0023293799450679,1.0000346981756545,25.000981414692927,4.429082037753878,0.8998331025519313,0.12828931011238479,0.11532029919964501,0.9375,1.0,0.8444479405879974,1.0,0.3206805884838104,0.984375,0.046875,0.026041666666666668,0.9120646593316147,nan,nan,0.8790902308343599,nan,nan,3.0348351626578203,0.0009292339808743027,0.024238903457894374,0.02314599579195918,0.0011841711971028088,0.00012438565180200088,0.002858949519763021,2.9823535050660537,21.70811381011389,0.4465356112713584,0.004100970320511953,15.406928972387613
|
| 34 |
+
251856,4126408704,0.00011024392733816057,2.1544259652728215e-05,2.0020900592498947e-05,5.650969433190767e-05,0.0027293130406178534,3.385556556168012e-05,3.225530812846652,1.000110250529505,1.0000215444917557,1.0000200211091461,1.0000565112920503,1.0027341333763127,1.0000338561406505,24.95872060355278,4.423099560070314,0.904698899346857,0.12650656130633703,0.12636424291501863,0.9479166567325592,1.0,0.8489385843276978,1.0,0.3308662474155426,0.9895833333333334,0.052083333333333336,0.036458333333333336,0.9221359106401602,nan,nan,0.8768742843531072,nan,nan,3.0311444591025345,0.0009037566149581797,0.024170264628065645,0.023080126904611746,0.0011138767819024549,0.00011673401182866463,0.00272098100975544,2.979038700953407,21.650463461376088,0.4468657399217288,0.003623353354756449,15.389824280947538
|
| 35 |
+
259488,4251451392,0.00019577400234993547,2.3261341084435116e-05,2.2065235498303082e-05,7.465852468158118e-05,0.000775831431383267,4.68358975922456e-05,3.223463814414572,1.0001957996723263,1.0000232616116862,1.000022065488419,1.0000746613297467,1.0007761332666045,1.0000468370035718,24.909876782860735,4.415859363033299,0.9022142731301519,0.12983218358774662,0.1260153247670261,0.9739583432674408,1.0,0.8593288958072662,1.0,0.33434247970581055,0.984375,0.057291666666666664,0.036458333333333336,0.932111734058708,nan,nan,0.8814051655742029,nan,nan,3.027750677449315,0.0008801761110662177,0.02410905617853831,0.02301098847489057,0.0010596485540537153,0.00011059008111480173,0.0026068667129814684,2.97597333310129,21.595334708565687,0.44716359872519346,0.003201913898652705,15.36557590140472
|
| 36 |
+
267120,4376494080,0.00024787029178696685,2.4407293494732585e-05,2.235699230368482e-05,9.157403837889433e-05,0.0014619894791394472,5.413766484707594e-05,3.222423088659222,1.000247912194267,1.0000244075913551,1.000022357252225,1.0000915782818027,1.0014630913388638,1.0000541391462958,24.88272488186223,4.4120897668095775,0.9003516581489275,0.1288731814962187,0.12325847636647613,0.96875,1.0,0.8637141585350037,1.0,0.3306046724319458,0.984375,0.057291666666666664,0.03125,0.9301365983361999,nan,nan,0.8818646380677819,nan,nan,3.0244746481873492,0.0008619210640513718,0.024054277847146133,0.02295072213958124,0.0010139523568119752,0.00010529296945972922,0.002512772467799086,2.9729756908237435,21.519570263176846,0.44757895667957187,0.0028398614619212603,15.338790569332646
|
| 37 |
+
274752,4501536768,0.00015536356659140438,2.1948106223135255e-05,2.0498032426985446e-05,9.132499326369725e-05,0.0012239489878993481,5.508920185093302e-05,3.221649707566636,1.000155378460084,1.0000219483471122,1.0000204982498544,1.0000913291719808,1.0012249077037516,1.0000550907233305,24.863532097112813,4.409300178538418,0.8992364742917589,0.1248681129311297,0.11932756555275031,0.9479166865348816,1.0,0.8755794167518616,1.0,0.3226127028465271,0.984375,0.052083333333333336,0.03125,0.9224605706209937,nan,nan,0.8876915153426429,nan,nan,3.0219719284230346,0.0008463852230018745,0.024012766446307128,0.022894979532024483,0.0009805858309864168,0.00010121883277508619,0.0024406656264813957,2.9706953091483115,21.52855689239452,0.4475412916477996,0.0025399532871463355,15.457380422443714
|
| 38 |
+
282384,4626579456,0.00010602111797197722,2.1223517251200974e-05,1.9871913536917418e-05,5.842255995958112e-05,0.001172262622276321,4.046698086312972e-05,3.2198327848648964,1.0001060276533433,1.0000212237424717,1.0000198721186853,1.0000584242689294,1.0011731393875034,1.0000404678111483,24.818108395561552,4.402789650077662,0.9002728066492807,0.12762927626142723,0.1231724610212224,0.96875,1.0,0.8696138858795166,1.0,0.3294604420661926,0.984375,0.057291666666666664,0.03125,0.930175639068087,nan,nan,0.8861412744348248,nan,nan,3.019998613661669,0.0008343192839102876,0.023975143303631288,0.022857025171812966,0.000949592396861776,9.793945440664557e-05,0.0023813091068345792,2.96890326725237,21.52480518949107,0.4475787418388783,0.002304473351749283,15.610716178320985
|
| 39 |
+
290016,4751622144,0.00017040180318872444,2.5034300051629543e-05,2.304173722222913e-05,6.462168312282301e-05,0.0011625253246165812,4.076598270330578e-05,3.219664359213008,1.000170420682858,1.0000250346134416,1.0000230420129612,1.0000646237804551,1.0011632114943088,1.000040766822098,24.813968692892143,4.402207970328324,0.8960406900145115,0.1292330407065388,0.12082050932948664,0.9791666567325592,1.0,0.8756189346313477,1.0,0.32873037457466125,0.9739583333333334,0.057291666666666664,0.03125,0.9343691103470823,nan,nan,0.886327225714922,nan,nan,3.0187185307656192,0.0008243210902096498,0.02393928842402989,0.022819818803229634,0.0009265313343669006,9.535875865854281e-05,0.0023388607453368247,2.9677743338397296,21.55560659637491,0.44744297244295134,0.0021352149741550937,15.521372324751447
|
| 40 |
+
297648,4876664832,0.00016192912153201178,2.7053464691562112e-05,2.510720423742896e-05,5.6957671404234134e-05,0.001159197388915345,4.0181248550652526e-05,3.2191054770567766,1.0001619460399382,1.0000270538306402,1.0000251075304858,1.0000569593116557,1.0011599044620743,1.0000401820740406,24.801450205886212,4.4004173370192925,0.8978763291449829,0.12563597651776692,0.12031605525550068,0.9739583432674408,1.0,0.8844948410987854,1.0,0.32494643330574036,0.9791666666666666,0.052083333333333336,0.026041666666666668,0.9325509943688909,nan,nan,0.8927051702824732,nan,nan,3.0176490953154365,0.0008173127910504327,0.023910893778467016,0.022790145526941443,0.0009080286543929468,9.393004842160538e-05,0.0023012987712171724,2.966827468273629,21.50914680732871,0.4476902670728773,0.0020334671566161833,15.538875964040422
|
training_logs/run_metrics_text67/10_bittoken_small/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: True
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 2
|
| 5 |
+
cache_base_path: /work8/<user>/byte_v2/math_bittoken/mix_text67/cache
|
| 6 |
+
combine_strategy: zero_pad
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /work8/<user>/byte_v2/math_bittoken/mix_text67/configs/config_bittoken_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_stem
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 768
|
| 45 |
+
n_head: 12
|
| 46 |
+
n_layer: 12
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: bittoken
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: bce
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /work8/<user>/byte_v2/math_bittoken/mix_text67/configs/config_bittoken_text67.py --tqdm --verbose --seed 42 --run_name bittoken_gpt2_small_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 12 --n_head 12 --n_embd 768 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /work8/<user>/byte_v2/math_bittoken/mix_text67/trained/bittoken_gpt2_small --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group bittoken_gpt2_small_text67 --tie_word_embeddings --tokenizer_dir /work8/<user>/byte_v2/math_bittoken/_paper_repo/tokenizers/num_text/bittoken_gpt2 --model rope_stem --num_embedding_type bittoken --add_reciprocal --combine_strategy zero_pad --num_loss_type bce --dropout 0 --cache_base_path /work8/<user>/byte_v2/math_bittoken/mix_text67/cache --compile --difficulty_column difficulty --train_set_paths /work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_binary_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/DivM_binary_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/mix_text67/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard standard standard --train_set_ratios 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.017857142857142863 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized_number --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_binary_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/DivM_binary_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt efficient_number_prompt pretokenized_number --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Thu Oct 1 20:17:25 2026', 'git_root': '/work8/<user>/byte_v2/math_bittoken/_paper_repo', 'git_url': 'https://github.com/KreitnerL/BitTokens/tree/192363b53b04652aec75c4cdf84d1a330beb25d9', 'git_has_uncommitted_changes': True}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: bittoken_gpt2_small_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /work8/<user>/byte_v2/math_bittoken/mix_text67/trained/bittoken_gpt2_small
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /work8/<user>/byte_v2/math_bittoken/_paper_repo/tokenizers/num_text/bittoken_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109/96841076'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109/46368368'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109/63050008'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109/4733157'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109/44316829'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109/48579485'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109/63589654'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109/28473695'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized_number
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_binary_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/DivM_binary_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.017857142857142863, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'efficient_number_prompt', 'pretokenized_number']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_binary_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/DivM_binary_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: bittoken_gpt2_small_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/10_bittoken_small/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_binary_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_DivM_binary_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_binary_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_DivM_binary_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_binary_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_DivM_binary_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_binary_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_DivM_binary_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_binary_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_DivM_binary_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,4.0145562344672125,3.418238025006758,4.057341285674435,3.53539207241632,3.9893743204108123,3.763315366187877,0.7491487264633179,1.8750839572066547,4.390372975496575,1.0000109528151233,1.0000421823060193,1.000117695009214,1.0000127024449175,1.0000416057093893,1.000033862582841,2.1152765664336326,1.0001886521674135,79.0413013361557,9.795225061736026,0.3788468486920749,0.032080347387827994,0.028945355530273983,0.022133297355935243,0.03180719200441248,0.1302083358168602,0.05078125,0.08899599686264992,0.6874458973928453,0.04863739013671875,0.10416666666666667,0.005208333333333333,0.0,0.010416666666666666,0.0,0.48905658439896627,nan,nan,0.4629652100702515,nan,nan,5.065284846776186,0.024134033013025857,0.024274323713053915,0.027259788563791312,0.02324844031707936,0.026332267745060214,0.00878469973587308,0.003043832199853519,0.021438283611243236,4.906769148427486,775.8009994624796,0.26398943450426077,0.004997126321997158,36.936289473581134
|
| 3 |
+
15264,250085376,4.865599150970411,4.087476504420617,4.7802830726507946,3.923017770649494,4.580966442205863,4.863782097287185,0.7268396615982056,2.371003034769923,4.008021984404574,1.000046369326365,1.0000339480017646,1.0000267221739148,1.0000284026877944,1.0000216659186005,1.000131307119449,2.068537613180867,1.0002818025023124,53.993080157167576,7.006909776453183,0.3812928398564084,0.1287590716501808,0.11354604323675876,0.12719273848634505,0.12172736153450406,0.1302083358168602,0.062499990686774254,0.12157606706023216,0.8752054362717229,0.13173078000545502,0.109375,0.057291666666666664,0.036458333333333336,0.08333333333333333,0.026041666666666668,0.4979130360831429,nan,nan,0.4996724912149754,nan,nan,4.5330231604117515,0.02154605515717101,0.02188077443513621,0.025174131281615334,0.02093582169813703,0.024226401418796985,0.007314762653916219,0.0027172794481724757,0.015918117575811134,4.393309789438356,56.61684268925425,0.35292359890974667,0.009998084214664627,22.29576939899063
|
| 4 |
+
22896,375128064,5.029228231153866,4.1849289448136915,4.655799464445787,3.8287122415910444,4.2371535757342444,4.924067076481758,0.7274458408355713,2.3721769662231615,3.8758117709852136,1.0000118225372339,1.000002792589446,1.000005161842042,1.0000062950173376,1.0000069995257443,1.0003052700905233,2.070004521193775,1.002575092461599,47.390758914233025,6.273741874387858,0.38236805676122443,0.1859515041754669,0.13530557515380978,0.1608089744967215,0.14013085002207912,0.203125,0.16015625,0.19704751670360565,0.9304590036921856,0.19541814923286438,0.10416666666666667,0.10416666666666667,0.046875,0.10416666666666667,0.03125,0.5765294012041219,nan,nan,0.5900683224899694,nan,nan,4.283564652443181,0.020439622465392236,0.020672169433786927,0.02408758312747518,0.019699553760897125,0.023082665914076084,0.006704881450012951,0.002593775111501795,0.013900272976600575,4.152384103323345,45.28250021342212,0.37185099575692004,0.014999042107332095,22.28996581224365
|
| 5 |
+
30528,500170752,5.42750779643271,4.294844641542037,4.840465271195484,4.0883580242915185,4.6702598736873515,5.3324729893605145,0.6610038578510284,2.7042960992063554,3.8082433555003563,1.0000993892049475,1.0000023586149505,1.0000012752629228,1.0000029201707457,1.0000007158710567,1.0000517276916097,1.9368082756646423,1.0000341611473085,44.31683156417375,5.917092487533548,0.4038851290169425,0.1904467836004332,0.14336358400555343,0.1751174122864536,0.15616157775506856,0.3958333432674408,0.08203125,0.2961572855710983,0.9623066817982266,0.1958708018064499,0.11979166666666667,0.09895833333333333,0.046875,0.125,0.03125,0.7242357706806312,nan,nan,0.6817348243203014,nan,nan,4.134588188618672,0.019743334394230448,0.01999402560798385,0.023477851027701894,0.019005439096922153,0.0224432528468749,0.006175991741730978,0.0025161853344849625,0.012581043299390061,4.008651042649956,41.04805524631616,0.38055263052009186,0.020000000000000004,22.365658659427805
|
| 6 |
+
38160,625213440,6.1026488685359155,4.4165149914598025,5.133057436594232,4.328773492325832,5.097252326395394,6.124975673424464,0.5580717027187347,3.088087348708595,3.7290496569088036,1.0000000937531022,1.0000003563738362,1.0000001999228125,1.0000000732635395,1.000000195887006,1.0000135026005228,1.7476325598243223,1.0000103258522917,40.94477620367629,5.521381501239302,0.401529020881945,0.21290767340460165,0.14026565522632872,0.17470832640273343,0.150776179290731,0.515625,0.342447929084301,0.3570375144481659,0.9801958579337328,0.25457218289375305,0.11458333333333333,0.11979166666666667,0.052083333333333336,0.109375,0.026041666666666668,0.7944930879942452,nan,nan,0.716586498504815,nan,nan,4.029980907437687,0.01924566508444569,0.019531919929321927,0.023064501145054084,0.018546665709493076,0.022018058544446557,0.0056302955076005705,0.0023923056565683465,0.011410523417495596,3.908140950895205,38.02854495053521,0.3876288319611824,0.019965729849129636,22.286700072267045
|
| 7 |
+
45792,750256128,6.420084881285791,4.622681967293161,5.300861181807079,4.477990161316238,5.2222330154226295,6.418571307253217,0.3442319631576538,3.352509028954695,3.6779351776931435,1.0000000471869765,1.0000000316649074,1.0000180624950232,1.0000000903374437,1.0000000776098281,1.0000022192736657,1.4119351107050548,1.0000539594052822,38.90835855226268,5.257818683437874,0.4166283527298693,0.22912718848073177,0.15931047530319592,0.2115502237129215,0.16256887067337028,0.65625,0.641927108168602,0.45764854550361633,0.9868019063094674,0.2980133891105652,0.13020833333333334,0.13020833333333334,0.052083333333333336,0.15625,0.03125,0.8717767341295257,nan,nan,0.7895715357735753,nan,nan,3.949333308747521,0.018872008627299465,0.01916806736108759,0.022742352453040925,0.01819067300503304,0.021694965172782903,0.0051357327555560895,0.0022294077755827648,0.010410847404679988,3.830889232178133,35.61563462168915,0.39385850599086386,0.019863180383909898,22.26633434657977
|
| 8 |
+
53424,875298816,6.006184712350553,4.532384978606828,5.201353520997353,4.3283114007968155,5.083164810823805,6.191573498125223,0.16576816886663437,3.2988330831772363,3.642894680146128,1.0000002933544447,1.0000001061703105,1.0000006862962827,1.0000001164136885,1.0000000394258985,1.0000009511667902,1.1811105609710673,1.0000189001985733,37.57579967641045,5.084103481156389,0.41900599631437635,0.24495684564555303,0.166989091014221,0.23040031736406125,0.1717097603857266,0.7447916567325592,0.856770820915699,0.6012694835662842,0.989121286769338,0.323678195476532,0.11979166666666667,0.13541666666666666,0.052083333333333336,0.17708333333333334,0.03125,0.9150054533480821,nan,nan,0.8613777885523936,nan,nan,3.6890440502163,0.017659301645445276,0.017975904758997484,0.021686162001936248,0.0170162855954203,0.020620986746538204,0.0040202454237902654,0.0018679066899741099,0.0075930565354050805,3.5806041801443347,34.141524395977676,0.39821233523841676,0.019693132578928373,22.28170704740928
|
| 9 |
+
61056,1000341504,6.581323170723978,4.587910383629297,5.237416308682622,4.573643198492757,5.434026531875123,6.5238550951036505,0.07779223099350929,3.4001675728467746,3.619197883119341,1.0000000239039109,1.000000009623661,1.0000000810240377,1.0000005869894402,1.000000012417623,1.000008905980019,1.0809262229196412,1.0000155028071205,36.69831722928159,4.975474286105228,0.41322091712199627,0.25288820128896666,0.16499827518705268,0.24543334706897166,0.17289676667248347,0.8333333134651184,0.908854179084301,0.657672256231308,0.9902940889307081,0.3318116068840027,0.11979166666666667,0.15104166666666666,0.057291666666666664,0.1875,0.036458333333333336,0.9524990921684852,nan,nan,0.8898425769681731,nan,nan,3.600984372708824,0.017286877090255857,0.017560003944923207,0.02131165407147652,0.01659189092470782,0.020233131913812527,0.003331815843955824,0.0015421598557576533,0.00650064341065493,3.4966261762081743,33.094064353522015,0.4013494695390358,0.01945688144839207,22.346534392101745
|
| 10 |
+
68688,1125384192,6.3361005708574965,4.560882352515031,5.219136856752716,4.426202463296316,5.254456672196474,6.208044037166701,0.054827190935611725,3.257464521303551,3.5982394500946002,1.0000000720222522,1.000001492824266,1.0000005789184452,1.0000000866127685,1.000000072643052,1.0000051110553891,1.056404271156572,1.000002905336817,35.94753583151802,4.889327824676397,0.41650301809499435,0.265899562092473,0.17343019902200513,0.2442523721763269,0.18415753461136183,0.859375,0.967447929084301,0.724333643913269,0.9914757201759669,0.3456421494483948,0.109375,0.15104166666666666,0.0625,0.1875,0.041666666666666664,0.9558863045725351,nan,nan,0.9053433143223325,nan,nan,3.5447089119881374,0.017023746412999766,0.01732733781312174,0.02111224233088878,0.016380719853136206,0.020046972047721743,0.0026572503347073798,0.0011900462085226687,0.0054142661317031,3.44355631146416,32.256743525808965,0.40388592584078165,0.019156226183831337,22.261990960249925
|
| 11 |
+
76320,1250426880,6.414132996772688,4.660344164818502,5.459490847152936,4.536190306107917,5.421297592109756,6.359517799888181,0.02317613549530506,3.333576999592088,3.5804219786271765,1.0000000838189982,1.000000040978171,1.0000000313544937,1.0000048428595956,1.0000002405905168,1.0000038018641542,1.02352937512088,1.0000173958755907,35.332489423019,4.817338359497933,0.4218604717082228,0.25324918674229646,0.17339473740774758,0.2412076453368629,0.18410096131596584,0.921875,0.986979179084301,0.7451538145542145,0.9926662469721833,0.3446788787841797,0.11979166666666667,0.13541666666666666,0.052083333333333336,0.16666666666666666,0.036458333333333336,0.9754491792215655,nan,nan,0.9090623701922596,nan,nan,3.5019690666548198,0.016844443678559775,0.01713454408947298,0.02094426606687412,0.016203520728023314,0.01989656863740634,0.0021001425452090595,0.0008347933328359539,0.004483949752123591,3.403526818191022,31.75396018907959,0.40551564666741297,0.018793456452196412,22.28381926019271
|
| 12 |
+
83952,1375469568,6.279249698234018,4.570612036311232,5.394646967450696,4.6462651478734305,5.671722301281182,6.364482331153454,0.009251852054148912,3.2900926093421194,3.5654701334812366,1.000000109585545,1.0000001223123203,1.0000000447034023,1.000000435857741,1.0000004057454812,1.0000044583163397,1.0093007595929184,1.0000052273583608,34.76984738225122,4.753239882858148,0.422780841405347,0.26359849926738416,0.1791676630957754,0.2430821923160665,0.18054422132664316,0.9322916865348816,0.993489570915699,0.7733749449253082,0.9958849853380555,0.3492528796195984,0.13020833333333334,0.15104166666666666,0.0625,0.171875,0.026041666666666668,0.9777240608042727,nan,nan,0.9223757330328226,nan,nan,3.4693455654349385,0.01671923877946983,0.016970166365992197,0.020806409414433943,0.016050437870851943,0.019765923077199585,0.0017107683563165282,0.0005379795652587596,0.003824573205523228,3.37296004903683,31.266163822489965,0.40705054950367353,0.018371334958694578,22.288042097480577
|
| 13 |
+
91584,1500512256,6.414417066921764,4.6644533835341235,5.525597404998919,4.661134196132744,5.662659297695896,6.388558666954983,0.017152966000139713,3.3202748392838917,3.551246310720065,1.0000000384946413,1.000000115482985,1.0000000710908408,1.0000005094314184,1.0000004870804773,1.0000050873273172,1.0173012111449915,1.000001280985989,34.271353478302736,4.698740253260155,0.4272290219726356,0.2583098831556724,0.17593194142570515,0.247747999682753,0.18844569968739186,0.9166666567325592,0.986979141831398,0.8133559823036194,0.9946704083052177,0.35186678171157837,0.13020833333333334,0.14583333333333334,0.06770833333333333,0.18229166666666666,0.041666666666666664,0.9747948943792532,nan,nan,0.9344306471757591,nan,nan,3.4455304621588008,0.01662899603099763,0.016855115185076517,0.020704125402694416,0.015933593593871025,0.01966365073522153,0.0014351180996477972,0.00031861716550249413,0.0033530410738637235,3.3506381861542716,30.875274949478655,0.40838895251181767,0.017893076407163026,22.317286625051644
|
| 14 |
+
99216,1625554944,6.278733315558499,4.617665622383353,5.504249540157615,4.569801647790763,5.520281711592052,6.225559834067289,0.007852468406781554,3.253591049205603,3.5400137172740265,1.0000003004981535,1.0000000136593945,1.0000000071401385,1.0000007543254654,1.0000003172693925,1.0000011160223752,1.0079032409434086,1.0001119439598658,33.91745299637457,4.6583856322436406,0.4208267488698513,0.26552792595855673,0.18311213890508526,0.25626822412792927,0.18733963742102402,0.9322916567325592,0.993489570915699,0.8355193734169006,0.9958849853380555,0.3582271933555603,0.13541666666666666,0.14583333333333334,0.057291666666666664,0.1875,0.03125,0.9736322103611504,nan,nan,0.9458015022488931,nan,nan,3.426911958691792,0.01655568924671004,0.016762248578511964,0.020625826061259407,0.015842798071770893,0.019586336443862592,0.001239340260435253,0.0001879898705390052,0.003000713958022236,3.3331109987953216,30.525092061085772,0.40952212391987414,0.017362323018206012,22.340758408614303
|
| 15 |
+
106848,1750597632,6.649602374372117,4.669354957217971,5.562253198275962,4.815541401505164,5.749582739236281,6.4935432090186005,0.014352409751154482,3.3733506406483467,3.528056632960215,1.0000001201403534,1.0000000018626451,1.0000000012417631,1.0000000943737188,1.000000090959003,1.0000073725081151,1.014518655239801,1.0000041983907382,33.50744786796959,4.613553155853969,0.3869795726828541,0.24988904323877556,0.16553234318297183,0.21002013179924972,0.179836977541237,0.9322916567325592,0.986979179084301,0.8295933604240417,0.9979296334156468,0.33036795258522034,0.11458333333333333,0.14583333333333334,0.052083333333333336,0.16145833333333334,0.041666666666666664,0.9692483202767713,nan,nan,0.9481840149189035,nan,nan,3.4111871788711667,0.016502596665555147,0.016682489042542614,0.020552395913990086,0.015767959658685188,0.019518619079134998,0.0010959251663457798,0.00012886483045187252,0.0027365556158893615,3.3182017555255974,30.106341992284268,0.4109548230692588,0.016783116791537683,22.28271299431018
|
| 16 |
+
114480,1875640320,6.439409320553068,4.685258815682283,5.64021231296159,4.728163976843016,5.730932202190151,6.296399654446873,0.02945464290678501,3.309794612566879,3.514463917410467,1.0000000645716252,1.0000001490080086,1.000000015522015,1.0000000987200557,1.0000000459452374,1.000002206863358,1.0299955861208994,1.0000022756071112,33.05001725786617,4.564446411136054,0.42048081498661505,0.25906941063833105,0.1908559107139512,0.2604034946395636,0.19638920371622692,0.96875,0.98046875,0.8623445928096771,0.9979296334156468,0.36550062894821167,0.13020833333333334,0.14583333333333334,0.057291666666666664,0.17708333333333334,0.036458333333333336,0.9880452178573856,nan,nan,0.9560746133017043,nan,nan,3.397292911261841,0.01645558795831176,0.01662141587998481,0.02049123353962959,0.015704074283494924,0.019461609062281192,0.0009847109045634523,0.0001022203507714555,0.0025291698596191397,3.3049428721852103,29.661603897372633,0.41237520862690935,0.016159868723772935,22.29217837260918
|
| 17 |
+
122112,2000683008,6.393317561903428,4.638761927373636,5.534845015965383,4.836985411425445,5.823614444893732,6.358268542368677,0.0006160214252304286,3.2817793144732983,3.5010656391386874,1.0000001803625174,1.0000000493600267,1.0000000214203995,1.0000005386076003,1.000000244937507,1.0000014735070004,1.0006162126707385,1.000377095548986,32.614835449434956,4.51287014064997,0.4231594011047194,0.2693567648491746,0.19169199454858704,0.26238600751346725,0.20198483419177038,0.984375,1.0,0.882848709821701,0.9991687465321138,0.37184134125709534,0.13020833333333334,0.15625,0.06770833333333333,0.17708333333333334,0.03125,0.9957600313549241,nan,nan,0.961150235341241,nan,nan,3.384177116894539,0.016400811201333087,0.016551438264274135,0.020431327905195456,0.015641385869248,0.019400465461294668,0.0008930246552705166,8.885325019831035e-05,0.0023443029543869284,3.292425490048884,29.336819720593137,0.41355880799720873,0.015497325216091833,22.38718150071292
|
| 18 |
+
129744,2125725696,6.5467347819358075,4.6866770607108865,5.609237952468295,4.8598671372358915,5.817062126162103,6.445315745968529,0.0017486443975940347,3.331471544397676,3.49450646622184,1.0000000117967498,1.0000000021730855,1.0000000034148488,1.0000000412886267,1.0000000661238555,1.0000014578311152,1.0017503060518724,1.0000021902246936,32.420512154223694,4.4913629147920595,0.4250017101710402,0.2639981861339323,0.1830247305967485,0.26027662667263113,0.19976794782801804,0.96875,1.0,0.8934365808963776,0.9991687465321138,0.3657820224761963,0.11979166666666667,0.14583333333333334,0.07291666666666667,0.19270833333333334,0.036458333333333336,0.9921382291552922,nan,nan,0.9650733686673144,nan,nan,3.3721588544136245,0.016348598099943018,0.016492447496608557,0.02037518296921515,0.015580683835807492,0.01934896666926913,0.0008134837371522648,8.052347298699197e-05,0.0021795617026578867,3.280939389467968,29.077550264399505,0.4143588383812887,0.014800531927597655,22.315774436620803
|
| 19 |
+
137376,2250768384,6.466845283284782,4.650021594638625,5.557113900780658,4.909826250125787,5.939020645494086,6.090175866455208,0.0004485569224925712,3.0899188093898564,3.4794459156886055,1.0000000093132235,1.0000000018626451,1.0000000149011417,1.0000001303849912,1.0000001310058755,1.000000555684703,1.0004486600836358,1.0000114131054552,31.933350382685678,4.437090144336372,0.43175033379890215,0.27680116799058685,0.19910753331519376,0.2762094387184826,0.21009186165602506,0.984375,1.0,0.9022466242313385,1.0,0.3837012052536011,0.13020833333333334,0.16145833333333334,0.057291666666666664,0.19270833333333334,0.057291666666666664,0.9962449396649996,nan,nan,0.9680083633090059,nan,nan,3.360470513783911,0.01629599102998607,0.01643152394189462,0.02031483656286173,0.01552599055723793,0.019299056456319515,0.0007431446499517183,7.432728084209759e-05,0.002027715605704451,3.269757911326021,28.76351534383102,0.4153773719191051,0.014074795349650518,22.303895194555583
|
| 20 |
+
145008,2375811072,6.568623792690532,4.675853452334802,5.654537091652544,4.9179000547155205,5.948211670231981,6.5498129116985435,0.0011527525493875146,3.3495169615188325,3.4704802753791832,1.0000000114863088,1.0000000012417631,1.0000000099340989,1.0000000884755953,1.0000001195194033,1.0000001577033029,1.0011535370836762,1.0000060193562312,31.649021711579746,4.405575739597792,0.4291008088244005,0.2746181455233811,0.1991009285160405,0.26905063750285835,0.2055052723362378,0.9739583432674408,1.0,0.8991650938987732,1.0,0.379488080739975,0.11458333333333333,0.15104166666666666,0.057291666666666664,0.17708333333333334,0.036458333333333336,0.991220852981011,nan,nan,0.9654449743781394,nan,nan,3.348617389537836,0.01623826968614767,0.01636818486150643,0.020255126818073465,0.015466476304526229,0.019244705501274783,0.0006805687049394797,6.915923989043699e-05,0.0018829708219003333,3.2584119108326575,28.416553945911257,0.41684120622477194,0.013325642393809753,22.312849286203082
|
| 21 |
+
152640,2500853760,6.521830958314239,4.758585095716019,5.663814745222529,4.8851738410691405,5.857713621109725,6.478928149422545,0.0004036722093587741,3.2746071180563376,3.461503879419373,1.0000000021730857,1.000000000310441,1.0000000018626447,1.0000000037252899,1.0000000012417636,1.0000155179921455,1.0004038056927587,1.000548121860696,31.357275581942247,4.373138115200119,0.4328864448424333,0.2851265665719222,0.20142370362002765,0.27354286522795357,0.2078072761435884,0.9895833432674408,1.0,0.9259112775325775,1.0,0.3855978846549988,0.13541666666666666,0.16666666666666666,0.06770833333333333,0.19270833333333334,0.041666666666666664,0.9979429263621569,nan,nan,0.9758094271334509,nan,nan,3.3376431760048217,0.016179391263986095,0.01629960006783026,0.020194907209238493,0.01539864965688807,0.019183820936826246,0.0006207078020847495,6.508648085547025e-05,0.0017450270585101437,3.247955968732354,28.16283094408127,0.41761962379049705,0.012558778301146429,22.31444113454828
|
| 22 |
+
160272,2625896448,6.684433224921425,4.682714941290517,5.612199707267186,4.869472805721054,5.832940358358121,6.571265409992048,0.00024194782236008905,3.3354356885506995,3.452925055132558,1.0000000031044078,1.0000000021730857,1.0000000034148488,1.0000000239039373,1.000000018316006,1.0000007456604054,1.000241986439908,1.000380777240975,31.096051492784678,4.344075005893139,0.4344733512212766,0.2950098206586832,0.20285604875858326,0.28128152219075425,0.20655474561599,0.9895833134651184,1.0,0.9347608387470245,1.0,0.38964807987213135,0.13541666666666666,0.16666666666666666,0.06770833333333333,0.20833333333333334,0.036458333333333336,0.9983444207658371,nan,nan,0.9788119324172536,nan,nan,3.3273967613839,0.016117897886209515,0.016224443941304862,0.020130841388346945,0.01533291623971583,0.01912091548528892,0.0005697842210752492,6.146079225667428e-05,0.0016202014466757365,3.2382182829592256,27.876206783878477,0.4188263758204768,0.011780043193471874,22.309162730464088
|
| 23 |
+
167904,2750939136,6.61283413724353,4.6605704339841845,5.5469453843929175,4.904282368098551,5.888100304951231,6.5444252539988454,0.0002404759325145278,3.3452941394697886,3.4455019833597667,1.0000000015522041,1.0000000111758556,1.0000000509120848,1.0000000763683083,1.000000038494628,1.0000420875463245,1.0002405189394605,1.0000010490133318,30.86547140246046,4.318417248495852,0.4356597165366828,0.29610725478895167,0.20503956102662518,0.2800303797028355,0.21645916947611715,0.9791666567325592,1.0,0.9214996695518494,1.0,0.3939257562160492,0.13541666666666666,0.171875,0.057291666666666664,0.19270833333333334,0.057291666666666664,0.9940035742086669,nan,nan,0.9776188305889567,nan,nan,3.3173601168287874,0.016067871957975696,0.01615591790035086,0.020067966298918046,0.015268070054218395,0.01906284923840296,0.0005246613328221973,5.81298356762144e-05,0.0015099953441226935,3.2286446378400364,27.61024112402768,0.4199326811651201,0.010995367597369719,22.324746979379892
|
| 24 |
+
175536,2875981824,6.425735355975727,4.684940586177004,5.589460749489538,4.9787314959491145,5.973571763684326,6.5547062788410635,0.00013408425547822844,3.295927957718073,3.437004646519199,1.0000000015522041,1.0000000096236596,1.000000018936865,1.0000000226621686,1.000000026077008,1.000001970334962,1.0001340965331322,1.0000063595950879,30.612873456774963,4.290335106898895,0.43619980734876646,0.28824436355348393,0.2127184380152914,0.28955347240213775,0.2267251983167603,0.9947916567325592,1.0,0.9511430561542511,1.0,0.40207868814468384,0.13541666666666666,0.16145833333333334,0.0625,0.1875,0.041666666666666664,0.9988690251484513,nan,nan,0.9851903064797322,nan,nan,3.3070360013367743,0.016016674326286295,0.01607308307256495,0.019995699496223873,0.015198941409322977,0.01899813785574677,0.00048542302352899013,5.561125019853854e-05,0.0014098280541960041,3.218802586104135,27.3141624428941,0.42109344051897024,0.01021072727974235,22.39787066991422
|
| 25 |
+
183168,3001024512,6.638397575045626,4.776049863236626,5.652045248386622,4.976238026283674,6.006342185040313,6.651468471071839,0.00046262654359452426,3.332281211687219,3.430272226415885,1.0000000006208816,1.00000001086541,1.000000038184087,1.000000013969831,1.0000000124176323,1.0000011875064097,1.000462740764913,1.000723480790711,30.400083218763175,4.266807855987006,0.4440270175010569,0.30931725478465927,0.215922190408745,0.2945489887212607,0.23272430091565427,0.984375,1.0,0.9363279342651367,1.0,0.4111059606075287,0.140625,0.18229166666666666,0.0625,0.203125,0.057291666666666664,0.9960520422707001,nan,nan,0.9797162863736352,nan,nan,3.2979670759683555,0.015968156631926633,0.01599225420630091,0.019926623483256786,0.01512834101906899,0.018933160143226237,0.0004487660574440918,5.3420870954203276e-05,0.0013189268616702909,3.210197409834897,27.26135195476204,0.42140066670546744,0.009432097738825898,22.399310527775857
|
| 26 |
+
190800,3126067200,6.5254445883135,4.717930162325499,5.55042625342805,4.945635410957037,5.943177372527565,6.522475784607813,0.00017419774667359889,3.244319826812557,3.4248389955610037,1.0000000012417631,1.0000000043461688,1.0000000086923322,1.0000000127280733,1.000000030112709,1.000002091717917,1.0001742134188505,1.0014286418237308,30.23656349094063,4.248685388335797,0.44478214181195497,0.3106620411837957,0.2187014622580417,0.30949668056594865,0.2333355216056398,0.984375,1.0,0.9496286511421204,1.0,0.41617903113365173,0.14583333333333334,0.17708333333333334,0.07291666666666667,0.234375,0.057291666666666664,0.9960622504974405,nan,nan,0.9836453297175467,nan,nan,3.288933477165981,0.015922123512658627,0.015914668858664372,0.019855160389247187,0.015063784630438098,0.018867821500904776,0.0004178422030726763,5.135354882283934e-05,0.001236223891251719,3.201604481284057,26.900068884188276,0.4225405586114157,0.008665408697250875,22.376562883132074
|
| 27 |
+
198432,3251109888,6.492553362001961,4.66057837040474,5.449106300560133,4.97454783009986,5.985786324677367,6.432407726621895,0.0002748485640040599,3.2138363481784125,3.4173411506344564,1.0000000086923295,1.000000000310441,1.0000000046566107,1.0000000068296973,1.0000000058983751,1.0000022150884427,1.0002748904215086,1.0000066945987593,30.013875066315297,4.223795432534607,0.4422240312496165,0.3126059460839645,0.21843750505701315,0.2887006243728354,0.2274262746026371,1.0,1.0,0.96295565366745,1.0,0.41025689244270325,0.16145833333333334,0.17708333333333334,0.06770833333333333,0.19791666666666666,0.052083333333333336,0.9996150645116965,nan,nan,0.9909561692426602,nan,nan,3.2805720539171324,0.015876071179737314,0.01583330357451497,0.01977771518157781,0.014999475271784675,0.018798552926012804,0.0003897400108978709,4.977912908262231e-05,0.0011639427713669534,3.193683457046039,26.798556698443257,0.42313859834329887,0.007916498943711495,22.380736863203502
|
| 28 |
+
206064,3376152576,6.5971843401590995,4.821841494795256,5.627336107194422,5.062549925409225,6.129368748826284,6.622914081908917,0.00023758564930176362,3.3328992792252667,3.412948752656424,1.0,1.0000001753932275,1.0000000024835263,1.000000016453335,1.0000000065192567,1.0000029862978967,1.0002376203601435,1.0016605360543713,29.878662547528474,4.208951543454471,0.4424762371632841,0.30307148006026163,0.22460493484438293,0.3015993264222786,0.23479947971168363,0.9947916567325592,1.0,0.9614807367324829,1.0,0.41605547070503235,0.13541666666666666,0.17708333333333334,0.06770833333333333,0.21354166666666666,0.0625,0.9984957361593843,nan,nan,0.987958587706089,nan,nan,3.2732691864797405,0.01582991890096744,0.01575243504154011,0.019698938299367457,0.014922477639131222,0.018727224041866615,0.00036392521702092903,4.871498437415879e-05,0.0010973252007393697,3.18682821084056,26.697403259747183,0.42362021624385937,0.007191071867150021,22.38973785960921
|
| 29 |
+
213696,3501195264,6.574414571126303,4.777683815297982,5.605700612999499,5.0113564139818285,6.03102903782079,6.545060125952606,8.57602008181857e-05,3.231073205972085,3.4068325040861964,1.0,1.0000000009313226,1.0000000009313226,1.000000139694942,1.0000000121071864,1.0000083923692866,1.0000857643842596,1.0007179130430348,29.696787208584965,4.188622159116258,0.44378478416835127,0.31326937353554457,0.22345648522246317,0.30134492281735525,0.23361009420510825,1.0,1.0,0.9688685238361359,1.0,0.4175974726676941,0.15625,0.17708333333333334,0.06770833333333333,0.234375,0.046875,0.9996150645116965,nan,nan,0.9904277492314577,nan,nan,3.266428248359347,0.0157859611312275,0.01566534374670043,0.019614663689591222,0.01484808490571262,0.01865177634214445,0.0003424916622378263,4.796317488196041e-05,0.0010383017603912107,3.180433646016276,26.512914548317593,0.4243638616336404,0.006494652022090374,22.39859790149642
|
| 30 |
+
221328,3626237952,6.663774450620014,4.882147312785189,5.6958162834246915,5.060288592550872,6.058135637082155,6.614071479365435,0.0001483404848841019,3.2518928129375952,3.403136579009394,1.0,1.0000000006208816,1.0000000049670499,1.000000083817766,1.0000000083818983,1.0000003486129752,1.0001483516266059,1.0001724596049653,29.590008383305868,4.176703293437557,0.4458600341145548,0.32080777214477096,0.2273708836605896,0.30290938513307974,0.23032539401288096,0.9895833432674408,1.0,0.956990122795105,1.0,0.41945216059684753,0.16145833333333334,0.1875,0.0625,0.203125,0.052083333333333336,0.9984518491352598,nan,nan,0.989993616938591,nan,nan,3.261000167174951,0.015745350310206675,0.01559064194687286,0.01953615990064187,0.014777917130983673,0.018579286731588235,0.0003241634948119657,4.704018344792854e-05,0.0009895508000367444,3.1754100402487815,26.46930650188988,0.424370980046424,0.005832543055900948,22.41240294043837
|
| 31 |
+
228960,3751280640,6.785440842310589,4.907916189171374,5.7337756982694055,5.050782774264582,6.068793149975424,6.737223342598693,9.28965127968695e-05,3.3678146928897754,3.400559618661646,1.0,1.0000000009313226,1.0000000031044067,1.000000014280249,1.0000000121071806,1.0000610741729097,1.0000929008353057,1.0000265831306805,29.51801193861064,4.168688058574744,0.4433692454799556,0.3181525005721373,0.22980759262930955,0.3070985982855805,0.24116302528955647,1.0,1.0,0.973280131816864,1.0,0.42492157220840454,0.16145833333333334,0.17708333333333334,0.07291666666666667,0.22395833333333334,0.046875,0.9995871788511673,nan,nan,0.9931029779836535,nan,nan,3.2548898486127644,0.01570605950193788,0.015506989768169986,0.019453765900176247,0.014703249605014596,0.018500943127599794,0.00030912520217617087,4.646385839360063e-05,0.000943387938694135,3.1697198471808083,26.273925595063584,0.42529096509831754,0.005209787318395601,23.437130752488052
|
| 32 |
+
236592,3876323328,6.694504022598267,4.833180910597239,5.700432857498526,5.08899908481785,6.0967010299985605,6.6705849672905195,5.738256004406139e-05,3.344342238161264,3.397124358957323,1.0,1.0000000062088117,1.0000000006208816,1.0000000121071762,1.0000000040357309,1.0000006860053599,1.0000573843995573,1.0007188208341549,29.41246902458107,4.157027326532527,0.44388852288955666,0.320406982627594,0.22427232761219254,0.30649235151573995,0.23771819719768983,1.0,1.0,0.9584913551807404,1.0,0.4216364026069641,0.15625,0.18229166666666666,0.0625,0.21875,0.0625,0.9998674945284923,nan,nan,0.9886374209697047,nan,nan,3.250608651960629,0.015668988749394642,0.01543379044604431,0.01937875977873862,0.01463471220587891,0.01843184791269415,0.0002950079883706616,4.591735199651307e-05,0.000906415832067769,3.165813195237344,26.244175841848307,0.4254422453339495,0.004631127461369502,24.6878539627281
|
| 33 |
+
244224,4001366016,6.834506988525391,4.979441765074928,5.8490810816486665,5.119343410556506,6.14874463031689,6.7815947329354564,0.0001311930282099638,3.385657494520948,3.395953873618661,1.0,1.000000003104408,1.0000000024835258,1.00000001055498,1.0000000074505757,1.0000025625443025,1.0001312039179417,1.0007170432145684,29.380411133031615,4.153473551811325,0.4456770993467257,0.32784772377535015,0.22484933833027432,0.30445901739518066,0.2388849933333424,1.0,1.0,0.9673804342746735,1.0,0.4236142337322235,0.15625,0.19791666666666666,0.057291666666666664,0.21354166666666666,0.057291666666666664,0.9999286374077201,nan,nan,0.9913471235583226,nan,nan,3.2465870292462893,0.01563500739192456,0.015369837273857254,0.019309097958828247,0.014571724979452277,0.018372813459833305,0.00028470520115981526,4.542641617428367e-05,0.0008763057075983993,3.162122094299537,26.168362333339715,0.4258117037639803,0.004100970320511953,24.196664490016442
|
| 34 |
+
251856,4126408704,6.8032272656758614,4.8548205730815726,5.684010982823868,5.185618206858635,6.199797197555501,6.853581425272113,0.0001471948708058335,3.4331736281589587,3.394103472916565,1.0,1.0000000037252892,1.000000000310441,1.000000004967052,1.0000000043461714,1.000002539882023,1.0001472059059513,1.0003767467322793,29.325711503825545,4.147359778854972,0.4477940403075516,0.3212111821703697,0.23121725210742813,0.3082240994520181,0.24793506494885797,0.9947916567325592,1.0,0.9659055173397064,1.0,0.4287845492362976,0.15104166666666666,0.1875,0.0625,0.22916666666666666,0.057291666666666664,0.9991694493219256,nan,nan,0.9916485954696933,nan,nan,3.2427396292737694,0.01560924011595016,0.015311228380744184,0.019245543441677285,0.014525735991597592,0.018318246531343854,0.0002761940347508765,4.524301918177354e-05,0.0008522494122129149,3.158555931122069,26.085198341050738,0.4261391245617494,0.003623353354756449,22.54486247334962
|
| 35 |
+
259488,4251451392,6.788088877995808,4.895876054031152,5.747449000676474,5.176556909767285,6.176036005529263,6.856380806490318,0.0002492130188329611,3.4226243107487715,3.3923015357189192,1.0,1.00000000403573,1.0,1.0000000046566098,1.000000005277493,1.000008450146217,1.0002492594452743,1.0011575950366165,29.272900099094997,4.141590601965882,0.44801834198140233,0.3260411730509294,0.22691756502012034,0.316052794766527,0.24925371138701857,1.0,1.0,0.9673804342746735,1.0,0.43029752373695374,0.15625,0.19270833333333334,0.0625,0.22395833333333334,0.0625,1.0,nan,nan,0.990538659815987,nan,nan,3.2401897510673763,0.015585130151357322,0.015259969322034304,0.019191755675486206,0.014482440197003054,0.018271251734958575,0.0002697079053730673,4.470265526084327e-05,0.0008325501975238658,3.15625222610436,26.116029162576865,0.4259698952136237,0.003201913898652705,22.496637756141478
|
| 36 |
+
267120,4376494080,6.743900775909424,4.876834949478507,5.647645990053814,5.137568913089733,6.18086994346231,6.883372805981335,7.766766793793067e-05,3.4331763209141606,3.391401880517757,1.0,1.0000000006208816,1.0,1.0000000021730857,1.0000000065192554,1.0000138507882903,1.0000776711798558,1.0003768283223513,29.250452888279696,4.138991249764825,0.4462263511069902,0.333105829005327,0.23271285440354036,0.3053305403572957,0.2513492416414589,1.0,1.0,0.9703170955181122,1.0,0.43218401074409485,0.15625,0.19791666666666666,0.0625,0.21875,0.0625,1.0,nan,nan,0.9938710468510786,nan,nan,3.2378579329134363,0.015567454869584276,0.01521848100933925,0.019142842562127258,0.014447707612681509,0.01822944873855088,0.00026369983519465085,4.448762981236867e-05,0.0008156969411692927,3.1541280969170358,26.083289956271773,0.42598336588677244,0.0028398614619212603,22.469675997350844
|
| 37 |
+
274752,4501536768,6.851834456125896,4.9472729380552956,5.807050863901775,5.197985571498672,6.229471176241829,6.881632425674425,9.509366282145493e-05,3.4327962799370675,3.389909790828824,1.0,1.0000000015522041,1.0,1.0000000018626447,1.0000000093132213,1.0000027997258596,1.0000950984310022,1.0001702658560658,29.204824207431788,4.1338991537969765,0.44581341812164876,0.32851188637421,0.2357028769694577,0.3044061280159925,0.25438252198690303,0.9947916567325592,1.0,0.9674199223518372,1.0,0.43300798535346985,0.16145833333333334,0.19270833333333334,0.0625,0.20833333333333334,0.078125,0.9991861712187529,nan,nan,0.9914206222941478,nan,nan,3.2362262820922605,0.015551991860224762,0.015181158837692449,0.019109390674557233,0.014422275209651348,0.018203559182395548,0.0002595540585869732,4.426190569229373e-05,0.000805459173596123,3.1526486139997036,26.01509426487317,0.42633598736356515,0.0025399532871463355,22.466421522142998
|
| 38 |
+
282384,4626579456,6.866350173950195,4.9416532128428425,5.833171208699543,5.22801319944362,6.28294885593156,6.904430032776418,7.211345473479014e-05,3.4461170395216505,3.390267030976247,1.0,1.0000000009313226,1.0,1.0000000018626447,1.0000000043461714,1.0000012328307317,1.0000721164967896,1.0001702388182292,29.214580188303813,4.134980420398856,0.4483932349363754,0.324540534790129,0.23112233671947494,0.3077826951992946,0.2447363953568501,0.9947916567325592,1.0,0.971857875585556,1.0,0.4284195303916931,0.15625,0.19270833333333334,0.057291666666666664,0.21875,0.06770833333333333,0.998789983180662,nan,nan,0.993079579124848,nan,nan,3.2347809850732236,0.015540201356374168,0.015148007556123127,0.01907752240877169,0.014393462351196954,0.018177859425762114,0.00025683171018650336,4.413350427542302e-05,0.000794638846954526,3.1513483106626614,26.023370078899575,0.4264307184773546,0.002304473351749283,22.372110387945177
|
| 39 |
+
290016,4751622144,6.853945255279541,4.948685965500771,5.7925929628933455,5.217599078081546,6.252890433184801,6.8998892831824765,4.615610214386834e-05,3.443041516602592,3.3900818652085336,1.0,1.0000000015522041,1.000000000310441,1.00000000403573,1.0000000052774922,1.0000075229584184,1.0000461574310475,1.0001702020342125,29.2076856665184,4.134212173346438,0.44806546527594593,0.3332003830925957,0.23039204824044537,0.3019559909114067,0.2523431878925091,1.0,1.0,0.9703434407711029,1.0,0.43106338381767273,0.15625,0.19791666666666666,0.0625,0.21875,0.078125,0.9995871788511673,nan,nan,0.99279566667974,nan,nan,3.2340649294476256,0.01552764927764351,0.01512479158774068,0.019058382162608852,0.014376303037017736,0.01815803834651405,0.00025445819083876167,4.402105162852252e-05,0.0007888228791856231,3.1507324452930443,26.04860484100238,0.42640574312372526,0.0021352149741550937,22.34527492688945
|
| 40 |
+
297648,4876664832,6.916515429814655,4.928607543930411,5.782756011312206,5.219060783274468,6.276449166548748,6.962778138892503,7.946595906105358e-05,3.480580538949625,3.3889551509637386,1.0,1.0000000006208816,1.0000000006208816,1.0000000046566098,1.0000000027939673,1.0000019548141443,1.0000794697844801,1.0001702191033994,29.175807308713228,4.130673217900844,0.4471092298956607,0.3234614121398668,0.23255429853507165,0.3116824864279711,0.24872831401979828,0.9947916567325592,1.0,0.9688816964626312,1.0,0.43073707818984985,0.16145833333333334,0.1875,0.06770833333333333,0.21875,0.0625,0.9991950650388995,nan,nan,0.9924822840839624,nan,nan,3.2337390210070787,0.015517341901088078,0.015103038141141422,0.01904400444692385,0.014355756810146815,0.01814726819936707,0.0002532462514570533,4.3885220645013644e-05,0.0007835451983740762,3.150490918239041,26.047593043472784,0.42627852705680225,0.0020334671566161833,22.37840418661632
|
training_logs/run_metrics_text67/11_bpe_small/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: False
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 2
|
| 5 |
+
cache_base_path: /work8/<user>/byte_v2/math_bittoken/mix_text67/cache
|
| 6 |
+
combine_strategy: sum
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty_sd
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /work8/<user>/byte_v2/math_bittoken/mix_text67/configs/config_td_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_gpt2
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 768
|
| 45 |
+
n_head: 12
|
| 46 |
+
n_layer: 12
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: bittoken
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: mse
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /work8/<user>/byte_v2/math_bittoken/mix_text67/configs/config_td_text67.py --tqdm --verbose --seed 42 --run_name bpe_gpt2_small_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 12 --n_head 12 --n_embd 768 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /work8/<user>/byte_v2/math_bittoken/mix_text67/trained/bpe_gpt2_small --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group bpe_gpt2_small_text67 --tie_word_embeddings --tokenizer_dir /work8/<user>/byte_v2/math_bittoken/_paper_repo/tokenizers/num_text/td_gpt2 --model rope_gpt2 --dropout 0 --cache_base_path /work8/<user>/byte_v2/math_bittoken/mix_text67/cache --compile --difficulty_column difficulty_sd --train_set_paths /work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/mix_text67/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard --train_set_ratios 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_prompt efficient_prompt efficient_prompt efficient_prompt efficient_prompt efficient_prompt pretokenized --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Thu Oct 1 21:32:37 2026', 'git_root': '/work8/<user>/byte_v2/math_bittoken/_paper_repo', 'git_url': 'https://github.com/KreitnerL/BitTokens/tree/192363b53b04652aec75c4cdf84d1a330beb25d9', 'git_has_uncommitted_changes': True}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: bpe_gpt2_small_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /work8/<user>/byte_v2/math_bittoken/mix_text67/trained/bpe_gpt2_small
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /work8/<user>/byte_v2/math_bittoken/_paper_repo/tokenizers/num_text/td_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td_gpt2_88525419/96841076'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td_gpt2_88525419/63050008'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td_gpt2_88525419/44316829'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td_gpt2_88525419/48579485'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td_gpt2_88525419/63589654'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td_gpt2_88525419/28473695'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/td_gpt2_88525419/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'efficient_prompt', 'pretokenized']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: bpe_gpt2_small_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/11_bpe_small/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,3.2898910442988076,4.094149847825368,3.708510478337606,2.4817408521970115,0.7517572740713755,5.128674437602361,4.394307295481364,29.538852314078312,68.95360440847212,48.730494445771235,14.040556408013126,2.1251978160740177,332.1157313819008,81.42178445322683,82.41803160393377,0.29555656137539993,0.005032084741527801,0.007596578472906716,0.3854166666666667,0.11328125,0.3316403304537137,0.6621379358220792,0.03191520646214485,0.08333333333333333,0.0,0.0,0.6365707376389764,nan,nan,0.6151792207407804,0.16776870349652503,0.14752590662101284,5.213380801384055,0.04519540493267215,0.05164468295938596,0.04113681184418843,0.01864674924532966,0.001871569530794468,0.04298775207223976,5.011897804211645,630.394804681747,0.22069223635874283,0.004997126321997158,21.223435282275965
|
| 3 |
+
15264,250085376,3.2347826759020486,4.243520458539327,3.872768680254618,2.551441709200541,0.7324318438768387,4.410140494505565,4.014823993047078,29.50578355037953,88.63835921068748,62.06538560725826,14.812318104470586,2.082489750471695,177.3213567313926,55.777230568885365,61.457560503363645,0.36387159048589934,0.07290349621154736,0.06412515864275974,0.4895833333333333,0.13020833333333334,0.4038187712430954,0.858331464380958,0.15410542488098145,0.171875,0.041666666666666664,0.015625,0.6996479666365,nan,nan,0.6707021390733038,0.1396405033871512,0.11775644562051941,4.6312995468826665,0.03763216515305985,0.047363247241350416,0.037405989855475164,0.010817084619309875,0.0016412751805931728,0.025165331066220965,4.471274430435971,61.32125631068487,0.3218349813809716,0.009998084214664627,21.218803896917354
|
| 4 |
+
22896,375128064,3.0809680620829263,4.086171468098958,3.61897936463356,2.269520918528239,0.749254067738851,4.106431384881337,3.8807287216186523,24.70167526493125,76.60218191961201,46.57854217789281,12.519492052631284,2.117840856283985,96.7197059327428,48.741675233702615,43.99730191968525,0.4040190121559126,0.08586911610903068,0.09920157631265607,0.640625,0.05078125,0.5521376008788744,0.9249780297490385,0.14478732645511627,0.203125,0.03125,0.015625,0.8075925921293674,nan,nan,0.7600612017849926,0.153518097159728,0.1306132138512718,4.3621642360376525,0.03407616682928563,0.04479628885789644,0.03510950194723165,0.007952096097032886,0.0015553416614703414,0.018653794325927855,4.2200210261217865,48.59939777226318,0.34258368557447905,0.014999042107332095,21.202711753364884
|
| 5 |
+
30528,500170752,2.9541413386662803,3.985538641611735,3.54899009068807,1.5925665174921353,0.7149810890356699,2.679948012034098,3.8057785034179688,23.1263852568064,70.2054436733177,43.8888121624974,7.607962255718114,2.0468772292555077,22.60927339401293,45.22697546454097,30.67310420516415,0.42021773904617116,0.1661070413764193,0.17700504784283314,0.7395833333333334,0.12109375,0.706902931133906,0.9529187808615284,0.2679559886455536,0.20833333333333334,0.09375,0.078125,0.8658296033900115,nan,nan,0.8447805863397662,0.14784686504314473,0.12323620376021911,4.201618364272128,0.031828451711797705,0.04318163634130957,0.03368102357929291,0.006410166634156574,0.0015098931569169141,0.0151193543428868,4.069887817604955,43.740268461234415,0.3530211153770143,0.020000000000000004,21.263397707259383
|
| 6 |
+
38160,625213440,2.894266347090403,4.060056924819946,3.5916071434815726,0.6354379820016524,0.5973219648003578,1.9044342835744221,3.722352147102356,22.598564786653938,84.56520753702053,46.820794258798806,2.038989337377076,1.8244636439258965,10.27528792380613,41.57559389401888,29.95698591165732,0.444592933286487,0.15497459237275513,0.1838740059338743,0.875,0.234375,0.7863140106201172,0.9781512098561413,0.3204534947872162,0.22916666666666666,0.078125,0.07291666666666667,0.9404064607903516,nan,nan,0.8878421017337436,0.16390947299972444,0.14115963164173687,4.089905185344584,0.03017707061988011,0.04205961240420364,0.03269216295749542,0.0054185100776270026,0.0014614533985015817,0.012862898830035939,3.965233456676111,40.48209629956032,0.3606381965273956,0.019965729849129636,21.23471288882922
|
| 7 |
+
45792,750256128,2.797185162703196,4.038316051165263,3.730538159608841,0.3803557501717781,0.49259312202533084,1.9017755786577861,3.6869576374689736,22.435225336368912,77.30830343026965,52.182947300704576,1.677059624012381,1.645283881744095,10.36738701507648,40.1414526533284,29.393951320214928,0.48896083988750344,0.16936803882325746,0.1705674095879074,0.9114583333333334,0.4596354166666667,0.8075173050165176,0.9824466903416139,0.36192071437835693,0.2708333333333333,0.09895833333333333,0.052083333333333336,0.9523504936951213,nan,nan,0.9020563395073017,0.14446187529983945,0.12054167175665498,4.004983922283104,0.02885661946108811,0.04119952896306946,0.03191906831233426,0.004718849357811103,0.0014081400296321594,0.01128473845156092,3.8855969585025334,37.98067072065121,0.3670746704536164,0.019863180383909898,21.21947801172093
|
| 8 |
+
53424,875298816,2.87561363975207,4.0440687139829,3.756049563487371,0.3427889240750422,0.3700614720582962,1.6224887172381084,3.6423136790593467,24.083702934558932,85.4217037841678,56.932704352840354,1.5226554308651055,1.4537728188560834,5.854137306070072,38.40341479737291,30.524584489247324,0.4978711517593188,0.1809030937108552,0.19704552866894157,0.953125,0.5638020833333334,0.8240018486976624,0.9879572477214255,0.3963765501976013,0.3072916666666667,0.11458333333333333,0.078125,0.9758445309513869,nan,nan,0.9011219076104074,0.15165623124570024,0.1274029408232309,3.7263055322919585,0.024857125094207295,0.038677947841155795,0.029668150041340234,0.001794220466296858,0.0012531077135127205,0.004647506114846657,3.6254074576901103,36.51872433256553,0.371021322359329,0.019693132578928373,21.300051588814295
|
| 9 |
+
61056,1000341504,2.801801383495331,4.196020444234212,3.671190162499746,0.1906712989633282,0.3057662198940913,1.6689687067021926,3.6171950101852417,21.30460332815053,98.690996412606,50.984827160196545,1.2327589262346523,1.3638132996176833,12.894081938337612,37.468194752260494,31.991325116771936,0.49688968740206585,0.16960495490125807,0.1794418472140771,0.953125,0.6809895833333334,0.8622725854317347,0.9902940889307081,0.3852807283401489,0.296875,0.09895833333333333,0.078125,0.9781398124565991,nan,nan,0.9205944829397291,0.18649413510608306,0.16350224266837662,3.6361072173070066,0.023248878726930957,0.037517543682004606,0.028617209294998978,0.0014686391154107412,0.0011533373754226484,0.003916218204683331,3.540185374052249,35.32975584499741,0.3746752699005341,0.01945688144839207,21.316002798821938
|
| 10 |
+
68688,1125384192,2.6378870805104575,4.140258868535359,3.729151487350464,0.11309822093850623,0.25948169579108554,1.1028772785017888,3.595131238301595,20.24851979693987,96.11054528049283,55.98048595949362,1.149234258355982,1.299835647881318,4.144365305413742,36.65716671281307,30.798593280198634,0.5294996914340101,0.1824377831291638,0.211357717642052,0.9739583333333334,0.7265625,0.8907749305168787,0.9914757201759669,0.42078664898872375,0.3333333333333333,0.10416666666666667,0.08333333333333333,0.9897990540484898,nan,nan,0.9418518120801309,0.16975057494937573,0.14669256502141556,3.5803113883318267,0.022066200786181694,0.0368812264798757,0.028062639970402023,0.0012606997449557983,0.0010444448100460548,0.0034475861598242522,3.4875485735452734,34.64481946825981,0.37673037379308444,0.019156226183831337,21.283029226665054
|
| 11 |
+
76320,1250426880,2.5404701630274453,4.11439863840739,3.749622960885366,0.10488054140781362,0.26692171519001323,1.523236578485618,3.5822856426239014,17.651538371558072,87.94288655069822,55.68155344610051,1.1253044408398754,1.311224934114407,7.864524923205825,36.16804063439388,29.677867614415828,0.547819570047151,0.21115119336966634,0.21230681603805582,0.96875,0.7135416666666666,0.8772930304209391,0.9938657362868036,0.44160452485084534,0.3541666666666667,0.125,0.08854166666666667,0.9826517323381267,nan,nan,0.9341799043807745,0.15806499107490404,0.13332738752554482,3.5389511686737394,0.021089312327510903,0.03640299606806346,0.027669107801281564,0.0011198475762607333,0.0009255203922325509,0.003166026777913719,3.448578342011785,34.06049932780006,0.37858899494933757,0.018793456452196412,21.329958979767063
|
| 12 |
+
83952,1375469568,2.5330074429512024,4.0616951783498125,3.632497638463974,0.19964019922190346,0.17540684963266054,0.5910310795685897,3.5662246545155845,18.473655407119274,86.35048148789143,49.73051905420232,1.4125871717997243,1.1938123011033073,1.976692402039867,35.59022300815021,27.818281547472306,0.5452661287801032,0.21155147438085983,0.21263685969018806,0.9791666666666666,0.8372395833333334,0.930408184727033,0.9946704083052177,0.44988200068473816,0.3489583333333333,0.13541666666666666,0.08333333333333333,0.9852161135252876,nan,nan,0.9607536712719593,0.16579264797794244,0.1415187481131094,3.5072943377461487,0.020286618986915132,0.0360139503021211,0.027347473916662743,0.0010302035168538635,0.0008130295809907304,0.002983306605358592,3.4188197395841735,33.41100806678866,0.3805547633581254,0.018371334958694578,21.31487159333626
|
| 13 |
+
91584,1500512256,2.335506101449331,4.147157231966655,3.741237550973892,0.07991884429065976,0.16540519753471017,0.77493313808615,3.546985308329264,15.414573108365458,98.31529835807775,56.121883384364054,1.1255114019403683,1.1848337101842108,2.6500245124941237,34.89818444742257,29.958615560406937,0.549394439079325,0.22518245458254812,0.24791908062540727,0.9895833333333334,0.8307291666666666,0.9250567009051641,0.9946704083052177,0.47960764169692993,0.3697916666666667,0.140625,0.125,0.9920392868613513,nan,nan,0.9567368186641639,0.16985266614998787,0.14414639472185323,3.483570627160084,0.01963313542235484,0.03571411068520329,0.027117970191133628,0.0009749471998079085,0.000710826943405447,0.002865282408659449,3.396554338542066,32.92172071112777,0.38212793113456833,0.017893076407163026,21.319916318296418
|
| 14 |
+
99216,1625554944,2.3384832243124642,4.190994342168172,3.7691539426644645,0.11109121864986567,0.1037444385389487,0.7410328813906139,3.5419938564300537,16.58113157317297,102.51202891219152,59.14099571579945,1.1317645229689073,1.11076879721436,2.612592819699173,34.747466198412326,31.119535505636954,0.5732731652501915,0.22053503029379917,0.24457063077062605,0.96875,0.90234375,0.9462486406167349,0.9946704083052177,0.48043128848075867,0.390625,0.13020833333333334,0.11979166666666667,0.9817740161088295,nan,nan,0.9680956392451966,0.15822998107493427,0.13365544964714596,3.4646621510948776,0.019062856921000675,0.035437453639707225,0.026942185896489793,0.0009357811442801474,0.0006287515201646467,0.002774987078862039,3.378880120176044,32.60506074002454,0.3831884477920127,0.017362323018206012,21.35081007055349
|
| 15 |
+
106848,1750597632,2.2225136309862137,4.180933972199758,3.678124566872915,0.006869892148339811,0.05962980445474386,0.8338688105965654,3.530012607574463,14.804052642949602,103.16032027542961,52.284897274921434,1.0069807041316243,1.0626166115829008,2.711791509907154,34.31664687107105,29.90675798428477,0.5646603838980465,0.23261528305831025,0.22856812212098818,0.9947916666666666,0.9479166666666666,0.9141829013824463,0.9958849853380555,0.4793480336666107,0.3802083333333333,0.140625,0.09375,0.9948518580640666,nan,nan,0.9509785928579125,0.18757094389790532,0.1628123401897028,3.4495104079757595,0.018571119371034946,0.03520140559033437,0.026776800744629584,0.0009026702775347436,0.0005580757062268742,0.00271441458006143,3.364785907653393,32.23475310818704,0.38446298417727026,0.016783116791537683,21.261560800619545
|
| 16 |
+
114480,1875640320,2.0993715276320777,4.212909797827403,3.7413635353247323,0.194057560993618,0.08876976068131626,0.5231778513795385,3.524051626523336,9.696777636453438,102.08864544222895,56.7090277663714,1.712844432618667,1.0954811118884218,1.9946163204510803,34.11398622412268,29.630196990590665,0.573281291433622,0.2305215151827367,0.23298752040476256,0.96875,0.9283854166666666,0.959252322713534,0.9971087059441789,0.48178890347480774,0.3854166666666667,0.13541666666666666,0.09375,0.9783198667575638,nan,nan,0.9688983257850244,0.15708876027686833,0.13278547255322337,3.435015493125286,0.01810582630128185,0.03498400875767803,0.026613296926482684,0.0008798804846847803,0.0004928467850079569,0.002674394451064882,3.351265225244476,31.782523059307653,0.3860266384643266,0.016159868723772935,21.220885531993172
|
| 17 |
+
122112,2000683008,2.189241354664167,4.14881310860316,3.6922565400600433,0.00511267125178468,0.052615560280779995,0.4036053440892526,3.505212744077047,14.116269345979392,96.00977583692229,53.70275509931009,1.0052274464895197,1.0551977367652632,1.7116143592354323,33.479062350706464,28.725700310772634,0.5669083936564534,0.21525444909028588,0.2516793526687118,0.9947916666666666,0.9544270833333334,0.9598873307307562,0.9979296334156468,0.48343291878700256,0.3645833333333333,0.125,0.125,0.9948518580640666,nan,nan,0.9747069263927793,0.1916668458232683,0.16826139041222632,3.422231385417954,0.017690932590624736,0.034829893814817006,0.026453622048946874,0.0008595453024659155,0.00043468968393012866,0.0026199113375614484,3.339342777110287,31.5257152583864,0.38686758016218553,0.015497325216091833,21.219805998009416
|
| 18 |
+
129744,2125725696,1.9999119838078816,4.188648124535878,3.6776176194349923,0.055802006145616666,0.0490923422233512,0.5533522174324995,3.504185914993286,13.116260896209853,105.5509691384318,54.270215368439644,1.0620268692614958,1.0517273221077597,2.172281150447151,33.45688056641339,30.097194473044443,0.5979365240508584,0.23498124773363574,0.24682128080649215,0.96875,0.9544270833333334,0.9483079214890798,0.9979296334156468,0.4958847761154175,0.4270833333333333,0.15104166666666666,0.109375,0.9801110466651153,nan,nan,0.971524106770327,0.1825173974084464,0.1575428566429764,3.410418636766266,0.017280266535164793,0.03466954697209855,0.026310181074977835,0.00084167126488686,0.0003838466683267287,0.0025759530795100814,3.328357158513814,31.13853904361245,0.3880490680894887,0.014800531927597655,21.2133332767527
|
| 19 |
+
137376,2250768384,2.0239011694987616,4.3285539746284485,3.63360133767128,0.004880704849711037,0.050602754267553486,0.3942298675052977,3.4879727363586426,13.55038489674508,129.56943885226246,50.965589559761845,1.0049620034572404,1.0539494291672007,1.5592104235661501,32.91410845979762,32.945377660679654,0.5889954141902413,0.2473752327109162,0.25296498499873915,0.9895833333333334,0.9544270833333334,0.9475401441256205,1.0,0.506989061832428,0.40625,0.14583333333333334,0.11979166666666667,0.992846321023535,nan,nan,0.9706845948821865,0.16422408541075129,0.13988368498394266,3.3992700891101966,0.0168209816885225,0.03453494650672973,0.026175527602646,0.0008260954293167816,0.0003401263704243264,0.002534127140602001,3.3180382727176094,30.833206468345733,0.38935643937087283,0.014074795349650518,21.216197266035095
|
| 20 |
+
145008,2375811072,1.8761765609184902,4.315222362677257,3.734495590130488,0.020400590679249337,0.04791282392883053,0.2722421249976226,3.479599118232727,10.85711739750815,128.17672708022374,57.380242404497956,1.0222071983518284,1.0512913575093317,1.4131709047013947,32.63833397117907,33.21987004485307,0.6217752589075212,0.24557116313906788,0.25599460983034367,0.9895833333333334,0.9609375,0.9682110548019409,0.9979296334156468,0.5119633674621582,0.4322916666666667,0.140625,0.11458333333333333,0.9905405653989874,nan,nan,0.9784646747090543,0.17482894573549856,0.15109233657130972,3.3887807958135236,0.01631847410424561,0.034418523785655575,0.026027715439408048,0.0008131282846033956,0.00030670351121449394,0.002502081447125886,3.308394156633287,30.624923485255092,0.38997714346044343,0.013325642393809753,21.23004581581372
|
| 21 |
+
152640,2500853760,1.5842304627100627,4.342227001984914,3.7448973059654236,0.005251769135914704,0.04236956902605016,0.23201708066820478,3.472426732381185,8.069340947193377,135.0055552305922,57.35682905482188,1.0053589850174103,1.0458240789614628,1.3676022696378805,32.40664397668178,33.751022077558,0.6463696746528029,0.24219982831440834,0.2683424632782743,0.9947916666666666,0.9739583333333334,0.9755956729253134,1.0,0.5200789570808411,0.4479166666666667,0.14583333333333334,0.13020833333333334,0.9948518580640666,nan,nan,0.9834931300138123,0.1795916697725867,0.1556473548989743,3.3784326179217286,0.015671373930410563,0.03431794993915536,0.025896534776906687,0.00080359237949724,0.00028073942177386807,0.002463458800395947,3.298998955837669,30.33942786465151,0.39101551899948084,0.012558778301146429,21.237211503392082
|
| 22 |
+
160272,2625896448,1.0649423003196716,4.294697185357411,3.7400262157122293,0.05533310932272192,0.03241839299638135,0.41826140143287677,3.460714062054952,3.977384542098068,130.71789744101156,56.30229252097771,1.0732342488061082,1.0347746554631385,1.656992840821908,32.02708679578247,32.398523292137284,0.6776431605494486,0.23495216580129608,0.25022658638717177,0.9895833333333334,0.9739583333333334,0.9558951457341512,1.0,0.5068177580833435,0.4947916666666667,0.13541666666666666,0.109375,0.9916751772980206,nan,nan,0.9664374087151373,0.17196517629887845,0.14729520858963951,3.3687707982634554,0.014895604342124755,0.034223784990742113,0.025781564838469356,0.0007899839298340123,0.0002614621270568773,0.002433579319891377,3.290384806531197,30.020413349821883,0.39228476309442045,0.011780043193471874,21.237373507850844
|
| 23 |
+
167904,2750939136,0.9513081138332685,4.3895487785339355,3.8529321750005088,0.03727622353744664,0.0394522280015129,0.3170532941245862,3.4507569471995034,3.3664102727294463,140.10391291424182,64.35146511505663,1.0445132713386973,1.0416183368267886,1.495294620725755,31.70438094596373,34.72965649669755,0.6937642225640533,0.2551022735490855,0.27222891703127794,0.9895833333333334,0.9674479166666666,0.9711558471123377,1.0,0.5339980125427246,0.5052083333333334,0.14583333333333334,0.13541666666666666,0.9926100994343869,nan,nan,0.9808539630127294,0.17909520379953392,0.15517932324049374,3.3589830152940325,0.014014639167891519,0.0341091763784954,0.025673980876348798,0.00078136256141114,0.0002455109577702125,0.002406491058924326,3.28175184114268,29.775208831895572,0.3931312669396588,0.010995367597369719,21.217761482520295
|
| 24 |
+
175536,2875981824,0.9722831373413404,4.438896715641022,3.7088447511196136,0.004934357835585009,0.033175755744499234,0.6641273960121907,3.446475863456726,4.134065772723799,145.74838740138415,54.98291865159191,1.0050535624457708,1.0358762649214572,2.788759726666383,31.57483080378786,34.46712745478876,0.7044310203034626,0.2552393559930553,0.2932870770310377,0.9947916666666666,0.9674479166666666,0.9489108622074127,1.0,0.5451786518096924,0.515625,0.15104166666666666,0.16145833333333334,0.9948518580640666,nan,nan,0.961787061188564,0.1733138302157992,0.14987195015419275,3.3504400543743587,0.013057564484017493,0.034026646265577186,0.025582494591611712,0.0007710647164589138,0.00023171747614797287,0.002383714932574969,3.2743868387110955,29.61916140021768,0.3935786663228548,0.01021072727974235,21.30858952085467
|
| 25 |
+
183168,3001024512,0.8052500672638416,4.399833758672078,3.76443882783254,0.005883031244896604,0.04032735140208388,0.6723945640211847,3.4406604766845703,2.9376682783990735,144.6987573213816,59.38829051729676,1.0060656306683964,1.0435708888718211,2.2994117429589145,31.390785088397962,34.68064992399636,0.7326574953737213,0.23847950682494579,0.2971215307805863,0.9947916666666666,0.9674479166666666,0.9502726594607035,1.0,0.5378423929214478,0.5572916666666666,0.15104166666666666,0.15625,0.9948518580640666,nan,nan,0.9631359133685086,0.17984663577514462,0.15508988464716822,3.342103147594184,0.01213231127345749,0.03391769659179272,0.025486890105753526,0.0007603803880278331,0.0002212792934273905,0.0023575075486850275,3.267227069012024,29.354147777429915,0.3948982448260489,0.009432097738825898,21.30205826589177
|
| 26 |
+
190800,3126067200,0.7116760698457559,4.405689438184102,3.8075674970944724,0.005692248865367826,0.03287801196468839,0.5338496831755037,3.432884454727173,2.8759703453832195,138.226688978258,61.154670789397066,1.005863515681485,1.035057465486824,2.1595996266965973,31.147465735838182,33.94361663667734,0.7458756640122428,0.2366192788528887,0.2798157483233051,0.9947916666666666,0.9674479166666666,0.958133190870285,1.0,0.5293700098991394,0.5885416666666666,0.15625,0.13541666666666666,0.9948518580640666,nan,nan,0.9692114158679033,0.17516970974368937,0.15134628639013195,3.3339126502026386,0.011252979561409917,0.03383144277755163,0.025399165981902158,0.0007504325901707627,0.00021223429407131258,0.002330485602421449,3.2601358960535585,29.206693313094306,0.3954573927732259,0.008665408697250875,21.368072569811158
|
| 27 |
+
198432,3251109888,0.6478693149983883,4.507584035396576,3.820859839518865,0.054255820842778725,0.03137802917384155,0.3082882957072191,3.4253733158111572,2.710755242985332,164.1207295611015,62.870074233975565,1.0718020194400821,1.0336753082239931,1.4918644150733673,30.912790992430818,37.744527396175805,0.7615900301908646,0.25140064208427376,0.3027965178480654,0.9895833333333334,0.9739583333333334,0.9668447375297546,1.0,0.5527103543281555,0.6145833333333334,0.16145833333333334,0.16145833333333334,0.9916751772980206,nan,nan,0.9748266251893559,0.1782371735492537,0.15329080957841748,3.3261889185285254,0.010511921129786071,0.03374618548782091,0.025325170394960936,0.0007392035138762478,0.00020571002238476186,0.002311017884165122,3.253349696382995,28.99628242325483,0.3961388400953021,0.007916498943711495,21.399711648209593
|
| 28 |
+
206064,3376152576,0.6426326297223568,4.447596947352092,3.7492132584253945,0.004947830567592367,0.02677107164102684,0.34418545517352567,3.4229305585225425,2.233456617946978,147.40892753804965,58.19102774108558,1.005069856003945,1.0288031746340285,2.091047975504008,30.83973136766996,34.68543775298488,0.7612313380779853,0.24609206454544383,0.29232528982310996,0.9947916666666666,0.98046875,0.9734304398298264,1.0,0.5447267889976501,0.6041666666666666,0.140625,0.15625,0.9948518580640666,nan,nan,0.9790315319454143,0.18575149815903083,0.161447609832976,3.3195191367440176,0.009913282863443744,0.03366310938037242,0.025259740472369997,0.0007297044097131786,0.0002000820786223657,0.002279777252898317,3.247473426717695,28.865797506563318,0.3966474795647637,0.007191071867150021,21.4056147226181
|
| 29 |
+
213696,3501195264,0.7307951487600803,4.601826409498851,3.9125577211380005,0.005208386874604305,0.03779364544122169,0.30248180582808953,3.4186966816584268,2.8978279420095183,190.37511102112816,69.25349245784537,1.005345783693448,1.0405626250535045,1.4775088469506787,30.709256304880203,42.3941578545087,0.7587395337875776,0.260765798837325,0.2833945573022936,0.9947916666666666,0.9674479166666666,0.9712564994891485,1.0,0.549021303653717,0.5885416666666666,0.14583333333333334,0.140625,0.9948518580640666,nan,nan,0.9776129927389169,0.16392508886929136,0.139738538962168,3.3133642568398116,0.00943498459777952,0.03358545748173285,0.025203562805705373,0.0007215739586568095,0.00019491858144401303,0.0022629390210874905,3.2419608075882405,28.720299520145172,0.39731704516324606,0.006494652022090374,21.407096245633248
|
| 30 |
+
221328,3626237952,0.6567615630726019,4.621777911980947,3.903665761152903,0.004017453359968688,0.02966129904295182,0.29800829039110494,3.412910223007202,2.910222282312921,191.03728616112383,70.23235555387966,1.0040970051400462,1.032537856459639,1.4796127747378478,30.528158330013444,42.603467137666776,0.7478242752299721,0.2569195862313915,0.29638891255404437,0.9947916666666666,0.98046875,0.9739717890818914,1.0,0.5531277060508728,0.5729166666666666,0.15104166666666666,0.15625,0.9948518580640666,nan,nan,0.9805551422662878,0.1842705767806668,0.15810167773937187,3.3074619220680366,0.009078195547546079,0.03350722800730711,0.0251424988779089,0.0007142802683858203,0.00019048732038948684,0.002245490150081756,3.236583728181845,28.625907256418305,0.39769382805678705,0.005832543055900948,21.369588743239756
|
| 31 |
+
228960,3751280640,0.739518745491902,4.60032441218694,3.9006323417027793,0.003989977861541168,0.027658060867906897,0.3578172349007218,3.410144488016764,3.1301595939519675,193.48742635392503,70.26063017086967,1.0040535722795136,1.029691132180075,1.615257311074271,30.4485312169529,42.99653562160478,0.764743911196161,0.2522311608286453,0.2917859786133543,0.9947916666666666,0.9674479166666666,0.9721162468194962,1.0,0.5482893586158752,0.59375,0.16145833333333334,0.15104166666666666,0.9948518580640666,nan,nan,0.974634764449244,0.18875269170530373,0.1640919929680725,3.302355211491063,0.008782734965850553,0.03346124197539283,0.02508042659049178,0.0007125740850472389,0.00018588081745833532,0.002225792921890728,3.2319065454220084,28.46713097484107,0.3983312129654162,0.005209787318395601,21.360826499680098
|
| 32 |
+
236592,3876323328,0.6355546986063322,4.6469219128290815,3.9309119085470834,0.004709245214144175,0.024650046413929278,0.26100496509025106,3.406663497289022,2.1620418173880203,194.7466638426342,72.68707153905207,1.0048158038550128,1.0264683217654378,1.4306688499334916,30.33845149821238,43.34231166754866,0.7417715353197427,0.25506638799340625,0.2954349059221491,0.9947916666666666,0.98046875,0.9765750914812088,1.0,0.5510658621788025,0.5729166666666666,0.15625,0.15104166666666666,0.9948518580640666,nan,nan,0.9805228408561865,0.1839931625721268,0.15798790151408562,3.2977045688369446,0.008563707289326226,0.03339078395969445,0.025031825170289613,0.0007090223816653406,0.0001827651126283662,0.00221367192059459,3.2276127790026026,28.401570746296358,0.39867677222117315,0.004631127461369502,21.44392732039911
|
| 33 |
+
244224,4001366016,0.6313810559610525,4.643890619277954,3.895863503217697,0.0040787508628833775,0.02542715778933295,0.32589311225456186,3.4047884146372476,2.152345297628275,200.42648934531826,68.76743553022358,1.0041622514092776,1.026912109996761,1.577460524286744,30.283048238756667,43.60540761394566,0.7423706135768868,0.2562321773751191,0.28875512235232753,0.9947916666666666,0.98046875,0.9749411046504974,1.0,0.5484282374382019,0.5729166666666666,0.15625,0.140625,0.9948518580640666,nan,nan,0.9779081795180294,0.17462118791687922,0.1512018924501414,3.2935128039638117,0.008384305795328919,0.03333903661249615,0.024981539094162317,0.0007095719516897535,0.00017988798402590839,0.002209959765017058,3.223708488992113,28.288883217725616,0.39919540884687843,0.004100970320511953,21.392100897396887
|
| 34 |
+
251856,4126408704,0.5312994346022606,4.651203056176503,3.9029992719491324,0.0043512086686708544,0.0263772896341834,0.3706957995988584,3.4025846322377524,1.9123358970387991,194.76055860044536,67.98628181889374,1.004437337227825,1.0281614203341576,1.6415761247690976,30.217776278339034,42.650161068149714,0.7473880167331272,0.2528816494425052,0.2932399984732665,0.9947916666666666,0.9739583333333334,0.9675599932670593,1.0,0.5482417941093445,0.5885416666666666,0.14583333333333334,0.15104166666666666,0.9948518580640666,nan,nan,0.9713128104388792,0.17860772738883193,0.15380605189905813,3.290493187430145,0.008246931141726187,0.03330206309261671,0.02494520084874849,0.0007067870137424914,0.00017751204309843273,0.0022025097740916754,3.2209121690793605,28.357569764557624,0.39890017625983154,0.003623353354756449,21.40016477296092
|
| 35 |
+
259488,4251451392,0.5453573899964491,4.718830247720082,3.9439911345640817,0.004557043427818523,0.03182940340290467,0.21492863755944805,3.402722120285034,1.9524489191594372,219.70212957446265,72.55159195534569,1.0046648384001478,1.0335627625058337,1.3641754151356513,30.221046752037324,46.832802888149544,0.7622300129152365,0.2564537355999762,0.30979121844080226,0.9947916666666666,0.9674479166666666,0.9765750914812088,1.0,0.5599046349525452,0.6145833333333334,0.15104166666666666,0.16666666666666666,0.9948518580640666,nan,nan,0.9805662223758796,0.17697300394562868,0.1529616404635211,3.2876587194018976,0.008137865496366881,0.03327252901521082,0.024926107767712296,0.0007044197115706769,0.0001753235457337127,0.0021931825702566216,3.2182492770952287,28.241706064599615,0.3992600184425993,0.003201913898652705,21.354634345145342
|
| 36 |
+
267120,4376494080,0.625839085628589,4.69343235095342,3.939215431610743,0.0043235685772439565,0.026706643310414318,0.26230291560447466,3.4012238581975303,2.1491364874639682,207.01943969616983,72.41225580172353,1.004419898352553,1.0284436047015435,1.4324487207523129,30.17488434161122,45.03157550725357,0.7560473840445949,0.25173816702344803,0.30306145669468526,0.9947916666666666,0.9739583333333334,0.9765750914812088,1.0,0.5533303618431091,0.5885416666666666,0.15104166666666666,0.16145833333333334,0.9948518580640666,nan,nan,0.9817632277942417,0.17477800095669713,0.149063314369414,3.2852935596342525,0.008049106810478348,0.03325056499932716,0.02489206472545658,0.0007034738653807422,0.00017458319637586275,0.002186843965205855,3.2160369083936153,28.22884131702487,0.3993476243920046,0.0028398614619212603,21.365828711006362
|
| 37 |
+
274752,4501536768,0.530282681186994,4.684607605139415,3.931171099344889,0.004439213067219801,0.028653874271791818,0.26585118376776035,3.4011953671773276,1.9144915321322078,204.46746489730256,71.5487149159059,1.004540421033003,1.030745895719125,1.4410711757939805,30.17717673373993,44.51202936737525,0.7556439749357198,0.26629328715904665,0.3061526846170219,0.9947916666666666,0.9739583333333334,0.9765750914812088,1.0,0.5644734501838684,0.59375,0.16145833333333334,0.16145833333333334,0.9948518580640666,nan,nan,0.9817632277942417,0.18882637193019036,0.16409427732772505,3.283548249031992,0.007986510447172772,0.033219324757919694,0.024871037657139012,0.0006984845782705284,0.00017369750740742538,0.002197789245391613,3.2144013918613448,28.177116884875346,0.39936952432324674,0.0025399532871463355,21.342784245044534
|
| 38 |
+
282384,4626579456,0.4769271748761336,4.715627908706665,3.958729843298594,0.004372289141050108,0.02802514169403973,0.26057962553265196,3.3998217980066934,1.786580474608175,220.70841648509864,73.4035206072998,1.004467981030698,1.0297382972277977,1.4285063151208484,30.13489468322504,47.07087497765871,0.7541813742531099,0.26173477972349035,0.3027352483332962,0.9947916666666666,0.9739583333333334,0.9782090783119202,1.0,0.5598177909851074,0.5989583333333334,0.15625,0.16145833333333334,0.9948518580640666,nan,nan,0.9818173590950513,0.1823323600231864,0.15794647556807226,3.282095317081252,0.007931835917134184,0.03321714978811548,0.02487182589271415,0.000697547875056179,0.00017274451173085277,0.0021966097117792284,3.2130075894598145,28.15014369401542,0.39967954859715465,0.002304473351749283,21.31673398108133
|
| 39 |
+
290016,4751622144,0.588445791353782,4.711307148138682,3.9543557465076447,0.0042940340833865775,0.028882809396236553,0.3020820443877407,3.399265011151632,2.0123720218492758,221.02800843288153,74.5777617735891,1.0043875280705892,1.0307954857030868,1.4840636578403636,30.118803234505847,47.32231316206283,0.7591585101314315,0.2632110922152157,0.30705948073807404,0.9947916666666666,0.9674479166666666,0.9724619537591934,1.0,0.5626838803291321,0.609375,0.15625,0.16145833333333334,0.9948518580640666,nan,nan,0.9789972821308766,0.18715272354105159,0.16276489671630165,3.281057253558056,0.007899074706892923,0.03320705619670586,0.024853478394379444,0.0006942339510481541,0.00017062831874531053,0.0021863722256720264,3.212046395516079,28.116308478501857,0.3998102289966192,0.0021352149741550937,21.28892909019499
|
| 40 |
+
297648,4876664832,0.5883388922860225,4.687101463476817,3.903294583161672,0.0043488396944818914,0.03152593231000841,0.3005760496198491,3.3992961645126343,2.0164473812872727,205.22849393439142,70.91271270653327,1.0044448068885992,1.0335736636817825,1.4797613567576657,30.118751683131034,44.5420265046673,0.7694147313251459,0.2628096316384955,0.3148728162067105,0.9947916666666666,0.9739583333333334,0.9724619537591934,1.0,0.5672194957733154,0.6145833333333334,0.16145833333333334,0.16666666666666666,0.9948518580640666,nan,nan,0.9789972821308766,0.18132276693116486,0.15727369452361017,3.2799113437788496,0.007863165491456972,0.033189395899887,0.024843473300172808,0.0006946952066314804,0.00016896472826396925,0.002184905380562548,3.210966729905891,28.155594451934785,0.3997506744831613,0.0020334671566161833,21.296396032163063
|
training_logs/run_metrics_text67/12_fone_small/args.yml
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
adamW_lr: 0.02
|
| 2 |
+
add_reciprocal: False
|
| 3 |
+
attn_implementation: flash_attention_2
|
| 4 |
+
base: 10
|
| 5 |
+
cache_base_path: /work8/<user>/byte_v2/math_bittoken/mix_text67/cache
|
| 6 |
+
combine_strategy: sum
|
| 7 |
+
compile: True
|
| 8 |
+
context_length: 1024
|
| 9 |
+
curriculum_advancement_difficulty_window_size: 3
|
| 10 |
+
curriculum_advancement_performance_window_size: 1
|
| 11 |
+
curriculum_advancement_threshold: 0.9
|
| 12 |
+
curriculum_generalized_mean_power: -1.0
|
| 13 |
+
curriculum_max_difficulty_fraction: 0.15
|
| 14 |
+
curriculum_new_difficulty_ratio: 0.3
|
| 15 |
+
curriculum_performance_history_size: 3
|
| 16 |
+
curriculum_step_size: 0.02
|
| 17 |
+
data_type: torch.bfloat16
|
| 18 |
+
default_vram: 200000000000
|
| 19 |
+
deterministic: False
|
| 20 |
+
device: cuda:0
|
| 21 |
+
device_batch_size: 16
|
| 22 |
+
difficulty_column: difficulty_sd
|
| 23 |
+
dropout: 0.0
|
| 24 |
+
effective_batch_size: 96
|
| 25 |
+
endgame_switch_lr_fraction: 0.5
|
| 26 |
+
endgame_switch_step_fraction: 0.5
|
| 27 |
+
eval_every_k_steps: 7632
|
| 28 |
+
eval_every_k_tokens: 125042688
|
| 29 |
+
float_type: float64
|
| 30 |
+
frequency_weight_slope: 0.0
|
| 31 |
+
grad_clip: -1.0
|
| 32 |
+
load_config_from: /work8/<user>/byte_v2/math_bittoken/mix_text67/configs/config_FoNE_text67.py
|
| 33 |
+
loss_norm_magnitude: 10
|
| 34 |
+
loss_weight_momentum: 1.0
|
| 35 |
+
lr: 0.02
|
| 36 |
+
lr_scheduler_type: cosine
|
| 37 |
+
max_eval_steps: 2
|
| 38 |
+
max_milestone_eval_steps: 50
|
| 39 |
+
max_milestone_eval_tokens: 0
|
| 40 |
+
max_train_steps: 305176
|
| 41 |
+
min_difficulty_ratio: 0.5
|
| 42 |
+
model: rope_stem
|
| 43 |
+
muon_momentum_warmup: 300
|
| 44 |
+
n_embd: 768
|
| 45 |
+
n_head: 12
|
| 46 |
+
n_layer: 12
|
| 47 |
+
no_save_latest: True
|
| 48 |
+
norm: rms
|
| 49 |
+
normalize_num_embedding: False
|
| 50 |
+
num_embedding_type: fone
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
num_head_type: linear
|
| 53 |
+
num_loss_type: mse
|
| 54 |
+
num_loss_weight: 10.0
|
| 55 |
+
num_warmup_steps: 30518
|
| 56 |
+
num_warmup_tokens: 500000000
|
| 57 |
+
num_workers: 4
|
| 58 |
+
online_weighting_warmup_tokens: 75497472
|
| 59 |
+
optimize_last: True
|
| 60 |
+
optimizer: Muon
|
| 61 |
+
position_embedding: rope
|
| 62 |
+
precision_type: torch.float64
|
| 63 |
+
preview_difficulty_ratio: 0.2
|
| 64 |
+
preview_exponential_decay: 0.8
|
| 65 |
+
reproducibility: {'command_line': 'python train.py --load_config_from /work8/<user>/byte_v2/math_bittoken/mix_text67/configs/config_FoNE_text67.py --tqdm --verbose --seed 42 --run_name fone_gpt2_small_text67 --device_batch_size 16 --effective_batch_size 96 --num_workers 4 --n_layer 12 --n_head 12 --n_embd 768 --context_length 1024 --train_token_budget 5000000000 --num_warmup_tokens 500000000 --eval_every_k_tokens 125042688 --max_eval_steps 2 --save_dir /work8/<user>/byte_v2/math_bittoken/mix_text67/trained/fone_gpt2_small --default_vram 200000000000 --no_save_latest --wandb --wandb_project sne_text67 --wandb_group fone_gpt2_small_text67 --tie_word_embeddings --tokenizer_dir /work8/<user>/byte_v2/math_bittoken/_paper_repo/tokenizers/num_text/bittoken_gpt2 --model rope_stem --num_embedding_type fone --base 10 --dropout 0 --cache_base_path /work8/<user>/byte_v2/math_bittoken/mix_text67/cache --compile --difficulty_column difficulty_sd --train_set_paths /work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_train_30M.csv.gz /work8/<user>/byte_v2/math_bittoken/mix_text67/data/fineweb_000_011_train.txt --train_dataset_curriculum_types standard standard standard standard standard standard standard --train_set_ratios 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.02380952380952382 0.8571428571428571 --num_loss_weight 10 --train_dataset_type pretokenized_number_pos --optimize_last --weight_decay 0 --lr 0.02 --lr_scheduler_type cosine --val_set_paths /work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_val_10k.csv.gz /work8/<user>/byte_v2/math_bittoken/_paper_data/val_text.txt --val_set_metrics logSMAPE logSMAPE logSMAPE exact_number_acc normalized_quint_class_acc exact_number_acc scaled_ppl --val_dataset_types efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos efficient_number_prompt_pos pretokenized_number_pos --val_additional_metrics logSMAPE exact_number_acc --loss_weight_momentum 1 --online_weighting_warmup_tokens 75497472 --grad_clip -1', 'time': 'Thu Oct 1 23:25:22 2026', 'git_root': '/work8/<user>/byte_v2/math_bittoken/_paper_repo', 'git_url': 'https://github.com/KreitnerL/BitTokens/tree/192363b53b04652aec75c4cdf84d1a330beb25d9', 'git_has_uncommitted_changes': True}
|
| 66 |
+
reset_loss_after_warmup: False
|
| 67 |
+
run_name: fone_gpt2_small_text67
|
| 68 |
+
save_checkpoint_steps: 9223372036854775807
|
| 69 |
+
save_checkpoint_tokens: 151115727451828646821888
|
| 70 |
+
save_dir: /work8/<user>/byte_v2/math_bittoken/mix_text67/trained/fone_gpt2_small
|
| 71 |
+
seed: 42
|
| 72 |
+
stop_after: 9223372036854775807
|
| 73 |
+
task_loss_exponent: -1
|
| 74 |
+
task_ratio_cap: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 75 |
+
tie_word_embeddings: True
|
| 76 |
+
tokenizer_dir: /work8/<user>/byte_v2/math_bittoken/_paper_repo/tokenizers/num_text/bittoken_gpt2
|
| 77 |
+
tqdm: True
|
| 78 |
+
train_cache_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109_pos/96841076'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109_pos/63050008'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109_pos/44316829'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109_pos/48579485'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109_pos/63589654'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109_pos/28473695'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/cache/bittoken_gpt2_47200109_pos/96369308')]
|
| 79 |
+
train_dataset_curriculum_types: ['standard', 'standard', 'standard', 'standard', 'standard', 'standard', 'standard']
|
| 80 |
+
train_dataset_type: pretokenized_number_pos
|
| 81 |
+
train_set_loss_weights: tensor([1., 1., 1., 1., 1., 1., 1.])
|
| 82 |
+
train_set_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_train_30M.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/mix_text67/data/fineweb_000_011_train.txt')]
|
| 83 |
+
train_set_ratios: [0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.02380952380952382, 0.8571428571428571]
|
| 84 |
+
train_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 85 |
+
train_token_budget: 5000000000
|
| 86 |
+
unique_samples: 9223372036854775807
|
| 87 |
+
use_curriculum: False
|
| 88 |
+
val_additional_metrics: [logSMAPE, exact_number_acc]
|
| 89 |
+
val_cache_paths: [None, None, None, None, None, None, None]
|
| 90 |
+
val_dataset_types: ['efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'efficient_number_prompt_pos', 'pretokenized_number_pos']
|
| 91 |
+
val_set_metrics: [logSMAPE, logSMAPE, logSMAPE, exact_number_acc, normalized_quint_class_acc, exact_number_acc, scaled_ppl]
|
| 92 |
+
val_set_paths: [PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Addition_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Multiplication_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Division_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/MinMax_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Interval_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/Sorting_decimal_uniform_val_10k.csv.gz'), PosixPath('/work8/<user>/byte_v2/math_bittoken/_paper_data/val_text.txt')]
|
| 93 |
+
val_set_tokens: [-1, -1, -1, -1, -1, -1, -1]
|
| 94 |
+
verbose: True
|
| 95 |
+
wandb: True
|
| 96 |
+
wandb_group: fone_gpt2_small_text67
|
| 97 |
+
wandb_project: sne_text67
|
| 98 |
+
wandb_sweep_id: False
|
| 99 |
+
weight_decay: 0.0
|
training_logs/run_metrics_text67/12_fone_small/metrics.csv
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
step,num_tokens,val_gen_loss_Addition_decimal_uniform_val_10k.csv.gz,val_gen_loss_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_loss_Division_decimal_uniform_val_10k.csv.gz,val_gen_loss_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_loss_Interval_decimal_uniform_val_10k.csv.gz,val_gen_loss_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_loss_val_text.txt,val_gen_perplexity_Addition_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Multiplication_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Division_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_MinMax_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Interval_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_Sorting_decimal_uniform_val_10k.csv.gz,val_gen_perplexity_val_text.txt,val_gen_perplexity_total,val_gen_acc_Addition_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Multiplication_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_Division_decimal_uniform_val_10k.csv.gz logSMAPE,val_gen_acc_MinMax_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_Interval_decimal_uniform_val_10k.csv.gz normalized_quint_class_acc,val_gen_acc_Sorting_decimal_uniform_val_10k.csv.gz exact_number_acc,val_gen_acc_val_text.txt scaled_ppl,val_gen_acc_total,additional_metrics_Addition_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Multiplication_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Division_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_MinMax_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_Interval_decimal_uniform_val_10k.csv.gz_exact_number_acc,additional_metrics_Sorting_decimal_uniform_val_10k.csv.gz_logSMAPE,additional_metrics_val_text.txt_logSMAPE,additional_metrics_val_text.txt_exact_number_acc,train_loss_train_total,train_loss_Addition_decimal_uniform_train_30M,train_loss_Multiplication_decimal_uniform_train_30M,train_loss_Division_decimal_uniform_train_30M,train_loss_MinMax_decimal_uniform_train_30M,train_loss_Interval_decimal_uniform_train_30M,train_loss_Sorting_decimal_uniform_train_30M,train_loss_fineweb_000_011_train.txt,train_perplexity,train_token_accs,lr,duration
|
| 2 |
+
7632,125042688,0.14886308612767607,2.2823960989626357e-05,3.4724818760878406e-05,7.224569435493322e-05,0.7346813082695007,0.051717799447942525,4.423058742579694,1.1730106243212122,1.0000228242579174,1.0000347254220636,1.0000722497865127,2.084832937877792,1.0544247543411114,78.96368622798981,12.325154906285203,0.196966802828417,0.030291158358896467,0.027920175286871823,0.0572916679084301,0.0755208395421505,0.04295722767710686,0.6876807453804017,0.05412285774946213,0.015625,0.005208333333333333,0.0,0.386734513839959,nan,nan,0.37449777817770763,nan,nan,5.283447265718718,0.05777319199142029,0.06012471452593585,0.05732952406145711,0.031023782447802465,0.00438835393135637,0.06824644369686285,5.004561220075099,7045.102427024762,0.26827678359354146,0.004997126321997158,22.230065450128055
|
| 3 |
+
15264,250085376,0.0486602985765785,1.66659469869046e-05,1.69896434272232e-05,0.00011521522174007259,0.7227656841278076,0.0014976041857153177,4.003637867708069,1.0510329591561969,1.0000166660871783,1.0000169897955475,1.0001152236534034,2.060128946838029,1.001498797941771,53.6995154684646,8.687475007419533,0.3569469437462678,0.04249474220654165,0.044192338781078144,0.1145833320915699,0.0690104104578495,0.06966392882168293,0.876335199376562,0.07983958721160889,0.13541666666666666,0.010416666666666666,0.005208333333333333,0.45767467660577194,nan,nan,0.4325986869322757,nan,nan,4.649874846687232,0.041553243401653644,0.0479482811401194,0.045635021669961966,0.022631491564003858,0.0039831038846451765,0.048246053350719374,4.439877623328997,58.305393284221864,0.3585848066198251,0.009998084214664627,22.24421870160545
|
| 4 |
+
22896,375128064,0.004510861588642001,3.772670061152894e-05,5.631553358398378e-05,9.880980724119581e-05,0.7219274342060089,0.0009392354550072923,3.8718408890999854,1.0045235583429233,1.000037727412276,1.000056317120772,1.0000988148656011,2.058480269767553,1.0009399971176611,47.115173670420965,7.739901479292536,0.5221237587922892,0.04488588517992649,0.045543344259849045,0.1354166679084301,0.07552083022892475,0.10826221853494644,0.9343798840114292,0.09080986678600311,0.375,0.015625,0.010416666666666666,0.4848554638350227,nan,nan,0.4732105342779202,nan,nan,4.34488831189735,0.031647300470609326,0.0422406782512309,0.04016088612705005,0.01824093458993109,0.003835904185872838,0.03778017238013519,4.170982410015758,44.12529610712823,0.37774246323861543,0.014999042107332095,22.231213920759483
|
| 5 |
+
30528,500170752,0.08268192037940025,0.0005289936671033502,0.028092128690332174,0.0002624478584039025,0.7126763164997101,0.0006626505783060566,3.7952471257109814,1.0862593672941274,1.0005291359521795,1.0286156223906133,1.000262483142981,2.0395005385885963,1.0006628955421268,43.880452315381895,7.290897479756074,0.23946985551146466,0.020762391771125988,0.022530521262043288,0.031250000931322575,0.12109375,0.02524494379758835,0.9694075749381911,0.03940872102975845,0.109375,0.010416666666666666,0.0,0.30206949419941037,nan,nan,0.3126485598941902,nan,nan,4.1652624214720175,0.025787962495552645,0.03897588959293718,0.037051544020750976,0.015784180443105488,0.0037612522390408224,0.031914947852524064,4.0119866208023,39.63766266852305,0.38634919051463623,0.020000000000000004,22.233683200781496
|
| 6 |
+
38160,625213440,0.00022482336316898,3.3139345759991556e-05,1.3172766557545401e-05,0.00011177502710779663,0.7236945033073425,8.259342030214611e-05,3.7030189173528925,1.000224864501381,1.000033139894946,1.0000131728534147,1.0001117825894492,2.0623101304583997,1.0000825977781145,40.06868114993348,6.733065262572741,0.6126049213242168,0.06466320852270087,0.05944198243769752,0.1666666641831398,0.09505208022892475,0.15567056089639664,0.9845170569259669,0.12093915790319443,0.484375,0.020833333333333332,0.0,0.501223832150572,nan,nan,0.5083301693181662,nan,nan,4.041465336331157,0.021805901444747467,0.03674433830367944,0.03495863244870767,0.014130708902724776,0.0037130784120096337,0.02801712768953352,3.9020955270535045,36.62078467555016,0.3932592163616069,0.019965729849129636,22.257867324618967
|
| 7 |
+
45792,750256128,0.0002442920522298664,6.517879910461488e-06,4.751821961690439e-06,1.9204296677344246e-05,0.7206586599349976,1.8529217413743027e-05,3.659319717087783,1.0002443225539952,1.0000065179011546,1.0000047518338948,1.0000192045071499,2.0558144450707045,1.0000185293903,38.35245492975234,6.486937528715649,0.6614524750075086,0.06378057106365656,0.06273082225659957,0.234375,0.1080729104578495,0.18535345792770386,0.989121286769338,0.13193237781524658,0.5729166666666666,0.03125,0.0,0.5335766475861116,nan,nan,0.5284603229471637,nan,nan,3.949105542319729,0.01922095148714492,0.03538707714434453,0.03365384299645837,0.012994801960293277,0.003680584936585049,0.025420976497553767,3.8187472852979147,34.43642380714916,0.3990450359927783,0.019863180383909898,22.202747062639265
|
| 8 |
+
53424,875298816,9.10063499759417e-05,8.90034789335914e-06,2.5983547402574914e-06,1.7944766568689374e-05,0.7195237278938293,4.526145221461775e-05,3.6233608224817244,1.000091010513345,1.0000089003876353,1.0000025983582113,1.000017944942502,2.0534631885090664,1.0000452626560334,36.99961814067448,6.293321006577324,0.693445654846233,0.07853499630588393,0.0676699063362915,0.2395833358168602,0.0950520895421505,0.20905762910842896,0.9914757201759669,0.1415877789258957,0.6354166666666666,0.026041666666666668,0.0,0.5494884764775634,nan,nan,0.5410174640807478,nan,nan,3.6413723269254907,0.010296816754918028,0.02984176026439199,0.028373515536028508,0.008893106123817315,0.0035349694974002133,0.015853286639175162,3.544578853049618,32.80495645977416,0.4034609749319668,0.019693132578928373,22.243252574174885
|
| 9 |
+
61056,1000341504,0.00024975316773634404,1.6064986994024366e-05,8.231996048380097e-06,0.00018236998221254908,0.7195720970630646,9.597731423127698e-05,3.5990468692422533,1.0002498006903608,1.0000160651249916,1.0000082320413979,1.0001823973542434,2.0535726856272722,1.000095985089938,36.09889580498492,6.164717281559018,0.7064175892476836,0.08268584567032594,0.06829149483836822,0.296875,0.1015625,0.25055309385061264,0.9938657362868036,0.15097728371620178,0.671875,0.020833333333333332,0.0,0.5759917964460328,nan,nan,0.5692628142229902,nan,nan,3.539594180832515,0.006762965544756839,0.028346948132747397,0.02699438072327761,0.007537427303782494,0.003523142505318768,0.012875333717326354,3.453553965144708,31.779641475942398,0.4064071798615111,0.01945688144839207,22.262797584808826
|
| 10 |
+
68688,1125384192,0.0006132980051916093,5.025627388022258e-06,1.0432467547616397e-06,2.5184571768477326e-05,0.720997542142868,0.0013945875689387321,3.576013615626531,1.0006135915119976,1.0000050256404927,1.0000010432473194,1.0000251849469999,2.056508427162319,1.0013958051430034,35.28832010978226,6.049552741062056,0.7294992432562871,0.07773029592979984,0.07202750811982593,0.4270833432674408,0.08203124068677425,0.288967028260231,0.9938657362868036,0.14856287837028503,0.6927083333333334,0.036458333333333336,0.010416666666666666,0.6272840566787939,nan,nan,0.590298281769113,nan,nan,3.480310497736489,0.005435419939361968,0.027641429876775894,0.02635871746579275,0.006909034969915205,0.0035170667039983467,0.01170218687219471,3.398746624154219,30.935512288061577,0.40905961052239315,0.019156226183831337,22.27754955455103
|
| 11 |
+
76320,1250426880,7.628436924278503e-05,2.5047716007975396e-06,1.2774908100254834e-06,9.93720386759378e-05,0.6181865036487579,0.00020886447600787506,3.553338468850901,1.0000762884337895,1.0000025047747378,1.0000012774916631,1.0000993785863344,1.855639400363766,1.0002089021640237,34.506837046546245,5.90898068548008,0.7361449941922931,0.08546487257138496,0.07080848938806103,0.4166666567325592,0.2578125,0.2934444844722748,0.9958849853380555,0.18483085930347443,0.6875,0.03125,0.005208333333333333,0.6522147457047444,nan,nan,0.5939148870529607,nan,nan,3.436016615987824,0.004682249322788322,0.027182030524966743,0.025937051006532937,0.0063880285799973865,0.0034677254294139155,0.010870815007090744,3.357488697789125,30.35633676094579,0.41099856969992193,0.018793456452196412,22.289446511742607
|
| 12 |
+
83952,1375469568,8.204453433791059e-05,2.92577010441164e-06,3.790984379747897e-07,2.9285662094480358e-05,0.157439973205328,0.08085972552771636,3.5377191707181432,1.0000820510468862,1.0000029257745617,1.0000003790985104,1.000029286292957,1.1737016363689654,1.0877639378569581,33.98326423757803,5.7492634934309805,0.7287562321968282,0.07629643290423091,0.07650440321070232,0.5156250149011612,0.869791679084301,0.32900072634220123,0.9958849853380555,0.20182883739471436,0.671875,0.026041666666666668,0.005208333333333333,0.7014964858147626,nan,nan,0.6260903510216546,nan,nan,3.4015414772579966,0.004284891947824622,0.026918673652984396,0.02568866265628547,0.005938151536177803,0.003157293834079892,0.010265352026463254,3.3252884323606535,29.764288040764427,0.413880734722007,0.018371334958694578,22.266333984307245
|
| 13 |
+
91584,1500512256,0.0004888020321232034,6.820278940722346e-06,3.079824637097772e-06,9.173168047027502e-05,0.08884561061859131,1.0929376003332436e-05,3.527204519836232,1.0004890291245352,1.0000068203043626,1.0000030798294512,1.0000917399816136,1.0931606149124442,1.0000109294358985,33.62093472299828,5.673528133798084,0.672536911638248,0.0773261285779099,0.06636449861152889,0.5416666865348816,0.928385429084301,0.38677307963371277,0.9979296334156468,0.19445358216762543,0.6145833333333334,0.026041666666666668,0.010416666666666666,0.7164335670822766,nan,nan,0.6513165628227094,nan,nan,3.3748716346914334,0.003808043006017471,0.026507379493176948,0.025331516139281512,0.005438267180966139,0.0026796661067133203,0.009557486603857858,3.3015492573109726,29.410632582205647,0.4152614249178136,0.017893076407163026,22.252602019546522
|
| 14 |
+
99216,1625554944,1.642487177377916e-05,4.49748722530785e-05,2.125569267263927e-06,3.3087451356550446e-06,0.042242628522217274,0.0008485643854498903,3.5112342639282965,1.000016425009247,1.0000449761795007,1.0000021255715654,1.000003308750676,1.0432519768463262,1.0008492844066317,33.08553332879504,5.5899573465084265,0.7514689398011705,0.08708478727992025,0.07447695217258679,0.5989583432674408,0.954427070915699,0.4163901209831238,0.9979296334156468,0.21630477905273438,0.7083333333333334,0.046875,0.010416666666666666,0.7358636180773223,nan,nan,0.6824344780567723,nan,nan,3.3542149885812065,0.003657364017266818,0.026372523096410895,0.02520032743375051,0.005015276257115645,0.0021694055393501766,0.009020689881497351,3.2827793825352676,29.00585931983134,0.4168033879015993,0.017362323018206012,22.26420841510791
|
| 15 |
+
106848,1750597632,0.1267898779365879,1.454300502246042e-06,2.4584153379692e-07,4.283956940298594e-06,0.04907071776688099,1.5334496765717631e-06,3.5018215164697417,1.144315021090677,1.00000145430156,1.0000002458415798,1.00000428397421,1.0504046859263103,1.0000015334509373,32.79259140999914,5.5696169477977735,0.7641030158969363,0.08546875793672985,0.08114023775183657,0.6666666567325592,0.947916641831398,0.489096075296402,0.9979296334156468,0.22628195583820343,0.734375,0.036458333333333336,0.005208333333333333,0.7602678678619365,nan,nan,0.6979304681687305,nan,nan,3.3372637133975074,0.0036162095565938994,0.026348153457802462,0.02515857115275325,0.004617008918311068,0.0016480221395495697,0.00855622330729423,3.2673195040857017,28.712245249148435,0.41795299224852267,0.016783116791537683,22.265724392146662
|
| 16 |
+
114480,1875640320,0.00044793425331590697,3.1174556625046534e-06,1.306630224462424e-06,1.6979038477415997e-06,0.03086694050580263,4.145180446357699e-05,3.4897628002800047,1.0004481076474812,1.000003117461103,1.0000013066313311,1.000001697906469,1.0313646282629332,1.000041453129398,32.38957931910017,5.48877709001984,0.7515297260515192,0.08657480510001636,0.08147018061326025,0.703125,0.9609375,0.5379266738891602,0.9991687465321138,0.22967049479484558,0.75,0.041666666666666664,0.010416666666666666,0.7852769740081081,nan,nan,0.7335352183047993,nan,nan,3.3217517651206214,0.0034986101566870584,0.026233514129645617,0.02504564146714638,0.004208821444323119,0.0011127405814534046,0.008053180345487137,3.253599237288777,28.233087387974656,0.41961771714456175,0.016159868723772935,22.264078431726755
|
| 17 |
+
122112,2000683008,6.679449802504678e-05,2.6092428839774584e-07,4.685610974775045e-07,1.9196059184878322e-05,0.009602019563317299,5.867016398042324e-06,3.477481558045838,1.000066798692592,1.000000260924323,1.000000468561225,1.0000191964161387,1.0096487789657909,1.0000058670406886,32.006862052059795,5.430943346094365,0.7807801807903458,0.09205703673409747,0.09401853271399663,0.7447916865348816,0.993489570915699,0.5409291982650757,1.0,0.25019943714141846,0.7916666666666666,0.03125,0.015625,0.8118651252589189,nan,nan,0.731064250091246,nan,nan,3.3072699832539305,0.0033932767954536138,0.026123722395120976,0.02495138858465164,0.0038216123704434145,0.0006125174139099039,0.007559412562401505,3.2408080341217613,27.886363295076777,0.420619115202182,0.015497325216091833,22.26569127853139
|
| 18 |
+
129744,2125725696,0.134786532697035,3.2942132293101167e-06,2.7901734256374766e-07,4.650435482034254e-06,0.038823094218969345,4.120392134154827e-06,3.4687597507533305,1.1546328497725915,1.0000032942186552,1.0000002790173896,1.0000046504568458,1.0395894913513326,1.0000041204060897,31.720037517308352,5.416324600361608,0.7011594957234741,0.08016971039379077,0.07738498912181047,0.6458333134651184,0.954427070915699,0.5023967623710632,1.0,0.2159910798072815,0.6197916666666666,0.020833333333333332,0.010416666666666666,0.7744126434748372,nan,nan,0.7246725898197232,nan,nan,3.2949098781594692,0.0033107864973279696,0.026040255870301714,0.024864239586942852,0.0034784420116371825,0.0003700274684987854,0.007106303146965544,3.229739804057747,27.66782851251666,0.4214852483166026,0.014800531927597655,22.263099784920488
|
| 19 |
+
137376,2250768384,0.001354638319753576,4.3863261112164764e-07,3.0714043219859377e-07,2.8811784886784153e-06,0.014750462956726551,2.3049350375003996e-06,3.4546275719379387,1.0013561869013756,1.0000004386327157,1.000000307140482,1.0000028811835069,1.0148738157285848,1.0000023049383813,31.27464543121685,5.327268766534557,0.7995298000246917,0.08968590672538207,0.08847133529956935,0.78125,0.98046875,0.5749841928482056,1.0,0.24353797733783722,0.8020833333333334,0.036458333333333336,0.015625,0.8314194701961242,nan,nan,0.7514612469822168,nan,nan,3.282283141074024,0.0032215291536487572,0.025960547938535493,0.024786020215837624,0.0031870312243600374,0.0002926223751425679,0.00669650996456871,3.2181388603576746,27.298905266593337,0.4231978776733893,0.014074795349650518,22.298020695822327
|
| 20 |
+
145008,2375811072,0.00019307991534489588,1.6735824459601645e-07,6.283480935564967e-08,3.857242845128894e-07,0.016635286156088114,0.022580783875326915,3.443021598039195,1.0001931171829057,1.000000167358262,1.0000000628348116,1.0000003857243878,1.0167810558048285,1.0230984256252094,30.916040407163266,5.279444803099096,0.783170362674888,0.09654288567219324,0.0980127128827437,0.8072916865348816,0.98046875,0.6253423988819122,1.0,0.26221683621406555,0.765625,0.057291666666666664,0.026041666666666668,0.8462073829141445,nan,nan,0.7843724775593728,nan,nan,3.270518525833541,0.003133984212309051,0.02587496750663633,0.0247110437043239,0.0029377719195125443,0.00024528587596109856,0.006318441943528185,3.2072970113846537,27.033897680618978,0.4238467031437708,0.013325642393809753,22.298212225491692
|
| 21 |
+
152640,2500853760,1.2494459269873914e-05,2.630654734048221e-07,7.013759173446488e-08,4.74590800081387e-06,0.0061094434931874275,8.414895660280308e-07,3.4336626464112974,1.0000124945588067,1.0000002630655112,1.0000000701375942,1.000004745927456,1.006132383254144,1.0000008414899568,30.62122283916587,5.232481948228477,0.813337096143335,0.10270289539170391,0.09435162096355443,0.828125,0.993489570915699,0.6386957168579102,1.0,0.265707403421402,0.8020833333333334,0.052083333333333336,0.015625,0.8568195650974909,nan,nan,0.7913187418598682,nan,nan,3.258241435029992,0.0029394699419463583,0.025673328518768565,0.024522480120456755,0.0026887864410842814,0.0002077829029380045,0.005901345823635851,3.1963082227900173,26.730302690459997,0.42523463333193484,0.012558778301146429,22.349555910823902
|
| 22 |
+
160272,2625896448,6.435205250454601e-06,1.1268508615103201e-07,2.8719114197883755e-08,1.70474809380039e-07,0.022062172181904316,9.896947545939838e-07,3.4251043742212155,1.0000064352376588,1.0000001126850935,1.0000000287191146,1.0000001704748271,1.0223376871039251,1.0000009896954065,30.357565489774327,5.197130130527193,0.8154316004859328,0.09561392832414417,0.09465851115010246,0.84375,0.973958358168602,0.6414612233638763,1.0,0.259065181016922,0.8385416666666666,0.036458333333333336,0.015625,0.8581597870603824,nan,nan,0.7885215880038837,nan,nan,3.247579244718445,0.0028502093880190572,0.025577856430892554,0.024440282265897152,0.0024975931229576902,0.0001826487794413172,0.005574025756633999,3.186456609519939,26.478360676228125,0.4261874816686977,0.011780043193471874,22.31080618739019
|
| 23 |
+
167904,2750939136,0.15100144676449645,3.7479387771099937e-07,1.0078791490286676e-07,7.844008299073835e-05,0.0052454761462286115,0.00028957266681572946,3.417536733982464,1.1762823466293175,1.0000003747939585,1.0000001007879205,1.0000784462298635,1.005261358377418,1.000289655075986,30.13867984554494,5.188656018205629,0.8082747591129469,0.10684290414140468,0.10151565150029843,0.859375,0.993489570915699,0.7200010418891907,1.0,0.2799270749092102,0.828125,0.041666666666666664,0.020833333333333332,0.8679247488228915,nan,nan,0.8286701744267096,nan,nan,3.2371608583760336,0.0027627226592366635,0.025493685386659727,0.024358112658201547,0.002326810687079041,0.00016316627914021346,0.005273023347646029,3.1767833182662413,26.2093397681943,0.42736674006442604,0.010995367597369719,22.349209576385487
|
| 24 |
+
175536,2875981824,0.00019331821772539115,1.6583007411696826e-08,1.6972786287539066e-08,3.88504524551081e-07,0.004437281284481287,7.153848287089204e-07,3.410616899258457,1.0001933551462796,1.0000000165830074,1.0000000169727865,1.0000003885046356,1.0044472452693447,1.0000007153850996,29.933002533728246,5.133949181655629,0.8536142294715505,0.10339662785162544,0.08870100973706477,0.921875,1.0,0.7422434687614441,1.0,0.2636651396751404,0.8958333333333334,0.041666666666666664,0.010416666666666666,0.9012114116339944,nan,nan,0.8394600030345222,nan,nan,3.2268067029787058,0.002709646647882919,0.02544728018482399,0.024317530257162684,0.0021835139631796877,0.0001491750481173703,0.005004304943743418,3.1669952340090846,25.98575680448324,0.42838117209276816,0.01021072727974235,22.341432252666834
|
| 25 |
+
183168,3001024512,1.8471460094815484e-07,2.1166036479058903e-08,3.386979052777406e-08,0.0601997848193534,0.0018476938712410629,7.391462446548758e-07,3.399654319548669,1.0000001847146227,1.0000000211660367,1.0000000338697914,1.0639737219818788,1.001849501552964,1.000000739146596,29.5975348950214,5.09476558535047,0.8697118371063481,0.09784383181723441,0.09604410402166431,0.9114583432674408,1.0,0.7244653403759003,1.0,0.2665698826313019,0.9166666666666666,0.03125,0.020833333333333332,0.8949221364261272,nan,nan,0.8317280330229551,nan,nan,3.2174319857529707,0.002623755854856007,0.025354729421894963,0.02422648012360843,0.002046964534564742,0.00013604742976231496,0.004738267825103983,3.1583057231029623,25.806989983005344,0.4289220711235855,0.009432097738825898,22.34089107471882
|
| 26 |
+
190800,3126067200,0.00017245068374904804,4.938112851959886e-08,4.3636315893991195e-08,4.2093195418146934e-07,0.0015873260563239455,8.732290609714255e-06,3.3947878195904195,1.000172467312164,1.0000000493811303,1.000000043636317,1.000000420932043,1.0015892158473132,1.000008732360424,29.468434042109582,5.067172138796996,0.8485238551164891,0.10155339970762661,0.09711333334380796,0.90625,1.0,0.7510666847229004,1.0,0.27176350355148315,0.90625,0.046875,0.015625,0.8998140874318779,nan,nan,0.8423065947523961,nan,nan,3.208637458871683,0.0025446454543362625,0.0252682711709594,0.02414330588804999,0.0019282715610707403,0.00012429954474326848,0.00450266885913289,3.1501259796066274,25.649209898847705,0.42967234351974865,0.008665408697250875,22.336073254386527
|
| 27 |
+
198432,3251109888,5.0565374323241485e-05,7.889088604606798e-09,2.8031802434824726e-09,6.590056926825127e-07,0.0008981626669992693,2.183121296184254e-06,3.3878261482653516,1.0000505679054799,1.0000000078890885,1.0000000028031804,1.0000006590060064,1.000898808720334,1.000002183124118,29.259256807651173,5.037172719585626,0.8509204707926192,0.10672053271808238,0.09300483493694142,0.9114583432674408,1.0,0.7867019772529602,1.0,0.27273574471473694,0.8958333333333334,0.046875,0.010416666666666666,0.8998506684632351,nan,nan,0.8564161458052695,nan,nan,3.2007712695583606,0.002469119694540967,0.025172913366320833,0.02405653907086018,0.0018203284834888607,0.00011419428393995792,0.004283244200142759,3.142854912689439,25.489338298888576,0.4303874587425396,0.007916498943711495,22.35966160771351
|
| 28 |
+
206064,3376152576,8.463807353109587e-05,9.402127187918552e-09,2.3313678987335607e-08,1.5256204477509527e-07,0.0018797942175297067,2.685832271254185e-06,3.3829556283696243,1.0000846425244658,1.0000000094021273,1.0000000233136794,1.0000001525620568,1.0018828222311673,1.0000026858380062,29.11818753925184,5.017165410731906,0.8761511397464141,0.09871106286995826,0.11283333893013414,0.9166666567325592,1.0,0.7998972833156586,1.0,0.28599876165390015,0.9427083333333334,0.046875,0.026041666666666668,0.9027730933545778,nan,nan,0.8607151846711835,nan,nan,3.1927069598857045,0.0023980218368958436,0.025088725460984156,0.023966601544813462,0.0017197453579588797,0.00010512906915837654,0.004080939759445575,3.135347779660605,25.265620226130295,0.4314688891250507,0.007191071867150021,24.19942350988203
|
| 29 |
+
213696,3501195264,4.6704893179594364e-06,2.291514700214492e-09,5.890738785652161e-09,2.198869353264854e-07,0.0017841969529399648,1.2071695266513416e-05,3.378094641763407,1.0000046705077026,1.0000000022915148,1.0000000058907388,1.0000002198869598,1.0017872400534553,1.0000120718315513,28.972176960623088,4.996283024440716,0.861700398118265,0.11359232019016073,0.11154035664666123,0.9479166567325592,1.0,0.8148572444915771,1.0,0.30157560110092163,0.9270833333333334,0.052083333333333336,0.03125,0.9147318235288063,nan,nan,0.8709795386530459,nan,nan,3.185782349373622,0.002335918920981772,0.025001855445882546,0.02388748698413911,0.0016367913956869757,9.905093357599232e-05,0.0039001130884200297,3.1289211151630707,25.152951323011386,0.4317072817911267,0.006494652022090374,22.322558973464613
|
| 30 |
+
221328,3626237952,2.7558038254937856e-05,2.5451517005059543e-09,5.8863478535897684e-09,2.2390114651216209e-07,0.0022364153410308063,2.585966086599001e-06,3.372967812116258,1.0000275586159029,1.0000000025451516,1.0000000058863479,1.0000002239011778,1.002240324937926,1.0000025859713573,28.828895391905856,4.975880870537674,0.8716101440327826,0.10894181712412283,0.09967977025776417,0.9270833432674408,1.0,0.8311209380626678,1.0,0.28408530354499817,0.9270833333333334,0.057291666666666664,0.015625,0.902157774932372,nan,nan,0.8729762282843391,nan,nan,3.179532835295178,0.0022416949502647525,0.02487293354866559,0.023757665476499476,0.0015494999535357631,9.30695865791321e-05,0.0037138858316013894,3.123304067337059,25.02225879778152,0.43228317164978014,0.005832543055900948,34.29056291479241
|
| 31 |
+
228960,3751280640,3.2691150408936664e-05,1.2725757392306747e-09,1.7930250617581578e-09,5.564956211401295e-07,0.0007775570338708349,3.972254205564241e-06,3.3687029501112797,1.0000326920830254,1.0000000012725758,1.000000001793025,1.0000005564957846,1.0007780383014762,1.0000039722684921,28.703868139352647,4.957811914509575,0.8777121004456594,0.11465507063482289,0.1089348451979831,0.9479166567325592,1.0,0.8221265077590942,1.0,0.3002714514732361,0.953125,0.057291666666666664,0.020833333333333332,0.9149630313816791,nan,nan,0.871281542815268,nan,nan,3.174063793336029,0.0021906337994019027,0.024790732660474008,0.023678033252439973,0.0014786931241761478,8.838519182140508e-05,0.0035678075530059367,3.118269488946967,24.96424980333516,0.4325226621440684,0.005209787318395601,53.37698064832394
|
| 32 |
+
236592,3876323328,3.947917264213174e-06,1.5306044764962223e-09,1.2857516162867455e-09,2.9276868218630625e-07,0.0025748176412889734,2.3446699799478665e-06,3.3664444530926025,1.000003947931002,1.0000000015306045,1.0000000012857515,1.000000292768727,1.0025812449991887,1.0000023446742494,28.63793840338847,4.948646605225428,0.8795156036996552,0.1141163893716145,0.10328045386817702,0.9583333432674408,0.993489570915699,0.8547855913639069,1.0,0.294089674949646,0.9583333333333334,0.052083333333333336,0.015625,0.9233163401174048,nan,nan,0.8842996122936407,nan,nan,3.1689412118769416,0.002145125103656521,0.024712299598510432,0.02359946700646708,0.0014163617430684194,8.510464388235407e-05,0.0034403636310502412,3.113542470128219,24.868284439640725,0.4328529019986496,0.004631127461369502,29.268299630886787
|
| 33 |
+
244224,4001366016,4.6268494600099075e-06,2.0334859629489443e-09,3.0743843026925788e-09,6.025090044659009e-07,0.0008407029963564128,2.5615056529204594e-06,3.3638790048426017,1.0000046268676344,1.0000000020334858,1.0000000030743843,1.0000006025091885,1.0008410684345836,1.0000025615110653,28.568609221911434,4.938494012334539,0.8769072915741676,0.1216281243967265,0.11229535619818196,0.96875,1.0,0.8593157529830933,1.0,0.3116205930709839,0.953125,0.06770833333333333,0.03125,0.9262118098946909,nan,nan,0.8854607736381391,nan,nan,3.164553047192318,0.002104703569383083,0.024649552350758382,0.023535560323042597,0.0013661297047639484,8.249732795144875e-05,0.0033277421657616816,3.1094868425527165,24.81525699066416,0.43324738108324556,0.004100970320511953,22.361317553508155
|
| 34 |
+
251856,4126408704,4.7985156470531365e-06,1.014547001965127e-09,1.02333103102481e-09,7.837792850295955e-07,0.0006678711579297669,2.9243639119158615e-06,3.362671254745995,1.0000047985289233,1.0000000010145471,1.0000000010233312,1.0000007837796394,1.0006682725039282,1.0000029243708666,28.52909951983733,4.93282518586551,0.8866219760115448,0.11539136078810014,0.10301073490272811,0.953125,1.0,0.8518752753734589,1.0,0.2950495779514313,0.9635416666666666,0.052083333333333336,0.020833333333333332,0.9208514858037233,nan,nan,0.879909515225639,nan,nan,3.161239551834549,0.0020716458880052926,0.024584677151162217,0.023476230430792284,0.0013226106471862997,8.034236919468047e-05,0.003235867036719921,3.1064681590155314,24.75433340356285,0.43357180779413235,0.003623353354756449,22.317172732290594
|
| 35 |
+
259488,4251451392,5.488988904289727e-06,1.276967698249365e-09,7.653022937592624e-10,4.3730044296808046e-07,0.00043173308222321793,2.9072622140802196e-06,3.3599689442198724,1.0000054890081436,1.0000000012769679,1.0000000007653023,1.0000004373005436,1.0004318603835824,1.0000029072696228,28.455783625239533,4.922317760177671,0.8731778872131126,0.12110879418025419,0.11285843488476618,0.953125,1.0,0.8443557918071747,1.0,0.311160147190094,0.9427083333333334,0.06770833333333333,0.026041666666666668,0.918317312549334,nan,nan,0.8783338889479637,nan,nan,3.1581197201315336,0.0020416142916257476,0.024526534671439632,0.023413161964043142,0.0012845422870115756,7.86086515265168e-05,0.003158513180050873,3.103616725091102,24.69352214978676,0.43382871570748976,0.003201913898652705,22.367000131699058
|
| 36 |
+
267120,4376494080,8.88067006599158e-06,2.55393539649873e-09,0.0,6.235127898435167e-07,0.0003945393364119809,3.690400518507886e-06,3.3598740883947658,1.000008880714761,1.0000000025539353,1.0,1.0000006235129848,1.0003946665826224,1.000003690412272,28.451344674763025,4.921678934077086,0.8846740853420942,0.11677271906891233,0.1055899439031569,0.9635416567325592,1.0,0.834057629108429,1.0,0.2991274297237396,0.9583333333333334,0.06770833333333333,0.020833333333333332,0.9242384273869296,nan,nan,0.875353540914754,nan,nan,3.155097626292835,0.0020188480111698577,0.024474214077868598,0.023361801008734875,0.0012545160194193242,7.732296492645649e-05,0.0030972488490945745,3.100813656019673,24.6133470708969,0.434212078482881,0.0028398614619212603,22.445669612039808
|
| 37 |
+
274752,4501536768,2.326941944374994e-05,1.2725757392306747e-09,0.0,6.392607474481338e-07,0.0004938236379530281,3.6220714889623196e-06,3.3591884400617955,1.000023269852552,1.0000000012725758,1.0,1.0000006392609722,1.0004939952127678,1.0000036220830382,28.43424367894811,4.9192521723757165,0.8779230234551729,0.12219707225745285,0.10943134074006347,0.9583333432674408,1.0,0.8593025803565979,1.0,0.30880650877952576,0.953125,0.06770833333333333,0.026041666666666668,0.9242404433898628,nan,nan,0.8844859789436063,nan,nan,3.1527549099570216,0.0019987078368687624,0.024435171942326812,0.02331356975000357,0.0012325732095073406,7.625201385729624e-05,0.0030463196193615686,3.0986522964034604,24.62000664981156,0.43419404538344414,0.0025399532871463355,22.612788054644373
|
| 38 |
+
282384,4626579456,3.18529187097738e-05,1.2725757392306747e-09,0.0,4.928822932015464e-07,0.0006022857196512632,4.088658016598856e-06,3.3581147539274148,1.0000318536469912,1.0000000012725758,1.0,1.0000004928824184,1.000602552717841,1.0000040886735342,28.40533739359961,4.915139483256139,0.8806485988411962,0.1256455572558983,0.11699782186665605,0.953125,1.0,0.862305074930191,1.0,0.320359468460083,0.953125,0.06770833333333333,0.036458333333333336,0.9192015286535025,nan,nan,0.8856364097446203,nan,nan,3.1509455295681827,0.0019837058889575075,0.024399246999250033,0.02327939209360356,0.0012136313505200255,7.541497246098337e-05,0.0030063671942387096,3.0969877529871126,24.619779416600114,0.4341672666573924,0.002304473351749283,22.635423638863394
|
| 39 |
+
290016,4751622144,1.3125715213391231e-05,1.2725756282083722e-09,0.0,5.222224643830486e-07,0.0005902854372834554,3.3768478431284166e-06,3.3575075567156696,1.0000131258038403,1.0000000012725756,1.0,1.0000005222226036,1.0005905998596414,1.0000033768580483,28.385844086131986,4.9123502445926714,0.8773501792325491,0.1210271995944324,0.10575870428189035,0.96875,1.0,0.8637010157108307,1.0,0.3037146031856537,0.9479166666666666,0.06770833333333333,0.026041666666666668,0.926881126438578,nan,nan,0.8871848577012619,nan,nan,3.149781499081438,0.001971559824057665,0.024365635739592544,0.023245585637585574,0.0011979868669688006,7.489847599830246e-05,0.0029757595972288507,3.0959500541727,24.656425340682954,0.4341152670599942,0.0021352149741550937,22.63972551829562
|
| 40 |
+
297648,4876664832,1.5578012835248956e-05,2.033485851926642e-09,0.0,5.444575634783178e-07,0.0003544911669450812,2.789994681506869e-06,3.356835586950183,1.0000155781797382,1.0000000020334858,1.0,1.0000005444577145,1.0003545622637922,1.0000027900014676,28.367730390237494,4.909729123881957,0.8781666190356361,0.12303606338306888,0.10776075947626712,0.9583333432674408,1.0,0.8652417659759521,1.0,0.3077535331249237,0.9583333333333334,0.06770833333333333,0.015625,0.9239366735952595,nan,nan,0.887539424157391,nan,nan,3.148833188750917,0.001962698873865534,0.024339563773074824,0.02322076163209121,0.001188384991605811,7.457676942925916e-05,0.0029515993178637514,3.095095584040168,24.61022261659804,0.4343059164329395,0.0020334671566161833,22.680954894185675
|
training_logs/train_stdout/14_bpe1_large_train.log.gz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:695b601be157ea06cf6c9ccc760643e50f90449ce79ce2dee67e9dfc57a54153
|
| 3 |
+
size 7289427
|
training_logs/train_stdout_text67/10_bittoken_small_train.log.gz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:98e8e71382b5d92869d60edbb68c10b581c9bedd2ebf943e217e8db55b6fb4d6
|
| 3 |
+
size 5271718
|
training_logs/train_stdout_text67/11_bpe_small_train.log.gz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f1ab0216cd340bb91834a7f35dadeaff12fe372a259e89e8c05241a6e495e00b
|
| 3 |
+
size 4934250
|
training_logs/train_stdout_text67/12_fone_small_train.log.gz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3004f061b1eb105e31f74282047783be041ca5492ed2c826ae02ff4b2e6dc1c8
|
| 3 |
+
size 5442231
|