EPFL ALiBi and training provenance 501-550 of 1184
29db7a6 verified about 1 month ago
175 kB
wandb: [wandb.login()] Loaded credentials for https://api.wandb.ai from WANDB_API_KEY.
wandb: Currently logged in as: ruben-navasardyan4869 (ruben-navasardyan4869-epfl) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.26.1
wandb: Run data is saved locally in /mloscratch/homes/navasard/sparse/wandb/run-20260509_201725-v6634orv
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run floral-surf-18
wandb: ⭐️ View project at https://wandb.ai/ruben-navasardyan4869-epfl/stochastic_sparsity
wandb: 🚀 View run at https://wandb.ai/ruben-navasardyan4869-epfl/stochastic_sparsity/runs/v6634orv
New run: Checkpoints will be saved to ./checkpoints/run_20260509_201713
Running config:
{
  "model_path": "/mloscratch/homes/navasard/qwen3_06b_base",
  "model_dtype": "bfloat16",
  "init_from_scratch": true,
  "train_path": "/mloscratch/homes/navasard/data/fineweb/tokenized",
  "val_path": null,
  "max_seq_length": 8192,
  "num_workers": 8,
  "output_dir": "./checkpoints/run_20260509_201713",
  "resume_from_checkpoint": null,
  "num_epochs": 1,
  "max_steps": 20005,
  "per_device_batch_size": 3,
  "gradient_accumulation_steps": 8,
  "learning_rate": 0.0005,
  "weight_decay": 0.01,
  "warmup_steps": 250,
  "lr_scheduler": "cosine",
  "max_grad_norm": 1.0,
  "save_steps": 1000,
  "eval_steps": 250,
  "eval_batches": 200,
  "logging_steps": 10,
  "seed": 42,
  "verbose": true,
  "checkpoint_activations": false,
  "strategy": "ddp",
  "mixed_precision": "bf16",
  "wandb_enabled": true,
  "wandb_project": "stochastic_sparsity",
  "wandb_name": null,
  "wandb_tags": [
    "qwen3",
    "0.6b"
  ],
  "attention_type": "dense",
  "sparse_block_size": 128,
  "sparse_num_random_blocks": 31,
  "sparse_mask_pool_size": 1000,
  "sparse_val_dense_compare": false,
  "sparse_layer_indices": null
}
Loading tokenizer from /mloscratch/homes/navasard/qwen3_06b_base...
Initializing model from scratch (random weights)...
Checkpoint activations: False


================================================================================
MODEL ARCHITECTURE
================================================================================
Model config:
  vocab_size: 151936
  hidden_size: 1024
  num_hidden_layers: 28
  num_attention_heads: 16
  num_key_value_heads: 8
  head_dim: 128
  intermediate_size: 3072
  max_position_embeddings: 32768
  rope_theta: 1000000
  attention_type: dense

Parameter count:
  Total parameters: 596,049,920
  Trainable parameters: 596,049,920
  Non-trainable parameters: 0
  Model size (BF16): 1.19 GB

--------------------------------------------------------------------------------
LAYER-BY-LAYER SUMMARY
--------------------------------------------------------------------------------
===========================================================================================================================================================
Layer (type:depth-idx)                                  Input Shape               Output Shape              Param #                   Trainable
===========================================================================================================================================================
Qwen3ForCausalLM                                        [3, 8192]                 --                        --                        True
├─Qwen3Model: 1-1                                       --                        [3, 8192, 1024]           --                        True
│    └─Embedding: 2-1                                   [3, 8192]                 [3, 8192, 1024]           155,582,464               True
│    └─ModuleList: 2-2                                  --                        --                        --                        True
│    │    └─Qwen3DecoderLayer: 3-1                      [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-2                      [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-3                      [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-4                      [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-5                      [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-6                      [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-7                      [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-8                      [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-9                      [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-10                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-11                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-12                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-13                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-14                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-15                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-16                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-17                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-18                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-19                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-20                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-21                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-22                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-23                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-24                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-25                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-26                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-27                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    │    └─Qwen3DecoderLayer: 3-28                     [3, 8192, 1024]           [3, 8192, 1024]           15,730,944                True
│    └─Qwen3RMSNorm: 2-3                                [3, 8192, 1024]           [3, 8192, 1024]           1,024                     True
├─Linear: 1-2                                           [3, 8192, 1024]           [3, 8192, 151936]         155,582,464               True
===========================================================================================================================================================
Total params: 751,632,384
Trainable params: 751,632,384
Non-trainable params: 0
Total mult-adds (G): 2.25
===========================================================================================================================================================
Input size (MB): 0.20
Forward/backward pass size (MB): 63052.97
Params size (MB): 1503.26
Estimated Total Size (MB): 64556.43
===========================================================================================================================================================
================================================================================

Loading dataset from /mloscratch/homes/navasard/data/fineweb/tokenized...
Using pre-tokenized dataset (no runtime tokenization)
Loading pre-tokenized HF dataset from /mloscratch/homes/navasard/data/fineweb/tokenized...
Loaded 5088012 pre-tokenized examples (stored_len=2048, pack_factor=4, packed_examples=1272003)
Dataset split: 1017602 train, 254401 validation
Training examples (packed, train split):  1017602
Validation examples (packed, val split):  254401
Per-device batch size:                    3
World size (GPUs):                        1
Gradient accumulation steps:              8
Effective batch size (seqs/step):         24
Sequence length:                          8192
Tokens per optimizer step:                196,608
Micro-batches per epoch (per rank):       339200
Optimizer steps per epoch:                42400
Total optimizer steps:                    20005  (capped by max_steps; 0.47 epochs)
Starting training...
Step 10/20005 | Loss: 12.0632 | LR: 2.00e-05 | 11163.0ms/step | 2.1 samples/s
Step 20/20005 | Loss: 11.1836 | LR: 4.00e-05 | 5754.2ms/step | 4.2 samples/s
Step 30/20005 | Loss: 10.4191 | LR: 6.00e-05 | 5744.4ms/step | 4.2 samples/s
Step 40/20005 | Loss: 9.8689 | LR: 8.00e-05 | 5820.6ms/step | 4.1 samples/s
Step 50/20005 | Loss: 9.0287 | LR: 1.00e-04 | 5732.2ms/step | 4.2 samples/s
Step 60/20005 | Loss: 8.2503 | LR: 1.20e-04 | 5730.7ms/step | 4.2 samples/s
Step 70/20005 | Loss: 7.6931 | LR: 1.40e-04 | 5751.5ms/step | 4.2 samples/s
Step 80/20005 | Loss: 7.4528 | LR: 1.60e-04 | 5760.7ms/step | 4.2 samples/s
Step 90/20005 | Loss: 7.3087 | LR: 1.80e-04 | 5762.0ms/step | 4.2 samples/s
Step 100/20005 | Loss: 7.1341 | LR: 2.00e-04 | 5766.0ms/step | 4.2 samples/s
Step 110/20005 | Loss: 6.9958 | LR: 2.20e-04 | 5776.2ms/step | 4.2 samples/s
Step 120/20005 | Loss: 6.8348 | LR: 2.40e-04 | 5771.9ms/step | 4.2 samples/s
Step 130/20005 | Loss: 6.7293 | LR: 2.60e-04 | 5777.9ms/step | 4.2 samples/s
Step 140/20005 | Loss: 6.6111 | LR: 2.80e-04 | 5776.8ms/step | 4.2 samples/s
Step 150/20005 | Loss: 6.5219 | LR: 3.00e-04 | 5779.2ms/step | 4.2 samples/s
Step 160/20005 | Loss: 6.4155 | LR: 3.20e-04 | 5778.4ms/step | 4.2 samples/s
Step 170/20005 | Loss: 6.3576 | LR: 3.40e-04 | 5781.0ms/step | 4.2 samples/s
Step 180/20005 | Loss: 6.2855 | LR: 3.60e-04 | 5778.8ms/step | 4.2 samples/s
Step 190/20005 | Loss: 6.1958 | LR: 3.80e-04 | 5781.1ms/step | 4.2 samples/s
Step 200/20005 | Loss: 6.1686 | LR: 4.00e-04 | 5784.4ms/step | 4.1 samples/s
Step 210/20005 | Loss: 6.0964 | LR: 4.20e-04 | 5785.6ms/step | 4.1 samples/s
Step 220/20005 | Loss: 5.9872 | LR: 4.40e-04 | 5781.5ms/step | 4.2 samples/s
Step 230/20005 | Loss: 6.0203 | LR: 4.60e-04 | 5779.9ms/step | 4.2 samples/s
Step 240/20005 | Loss: 5.9308 | LR: 4.80e-04 | 5782.4ms/step | 4.2 samples/s
Step 250/20005 | Loss: 5.8559 | LR: 5.00e-04 | 5785.2ms/step | 4.1 samples/s
wandb: updating run metadata
wandb: uploading output.log; uploading wandb-summary.json; uploading config.yaml
wandb: uploading output.log
wandb: uploading history steps 1999-1999, summary, console lines 2193-2193
wandb: 
wandb: Run history:
wandb:             eval/loss █▅▃▂▂▂▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
wandb:       eval/perplexity █▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
wandb:     speed/ms_per_step ▁▇▆▇▆▇▇▇▇██▇▇▇▇▆▆▇▆▇▆▇▇▇▇▆▇▇▆▇▇▆▆▆▅▅▅▆▆▅
wandb: speed/samples_per_sec ████████████████████████████████████▁███
wandb:   speed/steps_per_sec ▁███████████████████████████████████████
wandb:           train/epoch ▁▁▁▁▁▁▁▂▂▂▂▂▂▂▂▃▃▃▃▃▃▄▄▄▄▅▅▅▅▅▅▆▇▇▇█████
wandb:     train/global_step ▁▁▁▁▂▂▂▂▂▂▃▃▃▄▄▄▄▄▄▄▅▅▅▅▅▅▆▆▆▇▇▇▇▇██████
wandb:   train/learning_rate █████▇▇▇▆▆▆▆▅▄▄▄▄▄▄▄▄▃▃▃▃▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁
wandb:            train/loss █▅▅▄▄▃▃▃▃▃▃▃▃▃▃▂▂▂▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
wandb: 
wandb: Run summary:
wandb:             eval/loss 2.89823
wandb:       eval/perplexity 18.14208
wandb:     speed/ms_per_step 5779.72794
wandb: speed/samples_per_sec 4.15244
wandb:   speed/steps_per_sec 0.17302
wandb:           train/epoch 0.4717
wandb:     train/global_step 20000
wandb:   train/learning_rate 0.0
wandb:            train/loss 2.88394
wandb: 
wandb: 🚀 View run floral-surf-18 at: https://wandb.ai/ruben-navasardyan4869-epfl/stochastic_sparsity/runs/v6634orv
wandb: ⭐️ View project at: https://wandb.ai/ruben-navasardyan4869-epfl/stochastic_sparsity
wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20260509_201725-v6634orv/logs
Validation at step 250: Loss = 5.8286, Perplexity = 339.87
Step 260/20005 | Loss: 5.8050 | LR: 5.00e-04 | 42638.5ms/step | 0.6 samples/s
Step 270/20005 | Loss: 5.7424 | LR: 5.00e-04 | 5781.7ms/step | 4.2 samples/s
Step 280/20005 | Loss: 5.7162 | LR: 5.00e-04 | 5784.0ms/step | 4.1 samples/s
Step 290/20005 | Loss: 5.6187 | LR: 5.00e-04 | 5784.3ms/step | 4.1 samples/s
Step 300/20005 | Loss: 5.5468 | LR: 5.00e-04 | 5788.1ms/step | 4.1 samples/s
Step 310/20005 | Loss: 5.5122 | LR: 5.00e-04 | 5785.2ms/step | 4.1 samples/s
Step 320/20005 | Loss: 5.4400 | LR: 5.00e-04 | 5789.6ms/step | 4.1 samples/s
Step 330/20005 | Loss: 5.4348 | LR: 5.00e-04 | 5788.9ms/step | 4.1 samples/s
Step 340/20005 | Loss: 5.3711 | LR: 5.00e-04 | 5782.7ms/step | 4.2 samples/s
Step 350/20005 | Loss: 5.3337 | LR: 5.00e-04 | 5788.8ms/step | 4.1 samples/s
Step 360/20005 | Loss: 5.2797 | LR: 5.00e-04 | 5786.4ms/step | 4.1 samples/s
Step 370/20005 | Loss: 5.2381 | LR: 5.00e-04 | 5788.1ms/step | 4.1 samples/s
Step 380/20005 | Loss: 5.1747 | LR: 5.00e-04 | 5785.6ms/step | 4.1 samples/s
Step 390/20005 | Loss: 5.1563 | LR: 5.00e-04 | 5782.6ms/step | 4.2 samples/s
Step 400/20005 | Loss: 5.1232 | LR: 5.00e-04 | 5788.0ms/step | 4.1 samples/s
Step 410/20005 | Loss: 5.0761 | LR: 5.00e-04 | 5785.0ms/step | 4.1 samples/s
Step 420/20005 | Loss: 5.0708 | LR: 5.00e-04 | 5787.5ms/step | 4.1 samples/s
Step 430/20005 | Loss: 4.9995 | LR: 5.00e-04 | 5794.8ms/step | 4.1 samples/s
Step 440/20005 | Loss: 4.9578 | LR: 5.00e-04 | 5787.5ms/step | 4.1 samples/s
Step 450/20005 | Loss: 4.9266 | LR: 5.00e-04 | 5786.1ms/step | 4.1 samples/s
Step 460/20005 | Loss: 4.9103 | LR: 5.00e-04 | 5831.8ms/step | 4.1 samples/s
Step 470/20005 | Loss: 4.8793 | LR: 5.00e-04 | 5781.6ms/step | 4.2 samples/s
Step 480/20005 | Loss: 4.8488 | LR: 5.00e-04 | 5782.1ms/step | 4.2 samples/s
Step 490/20005 | Loss: 4.8245 | LR: 5.00e-04 | 5782.1ms/step | 4.2 samples/s
Step 500/20005 | Loss: 4.7789 | LR: 5.00e-04 | 5787.0ms/step | 4.1 samples/s
Validation at step 500: Loss = 4.7722, Perplexity = 118.17
Step 510/20005 | Loss: 4.7805 | LR: 5.00e-04 | 40717.3ms/step | 0.6 samples/s
Step 520/20005 | Loss: 4.7539 | LR: 5.00e-04 | 5784.5ms/step | 4.1 samples/s
Step 530/20005 | Loss: 4.6844 | LR: 5.00e-04 | 5786.3ms/step | 4.1 samples/s
Step 540/20005 | Loss: 4.6855 | LR: 5.00e-04 | 5779.5ms/step | 4.2 samples/s
Step 550/20005 | Loss: 4.6596 | LR: 5.00e-04 | 5786.0ms/step | 4.1 samples/s
Step 560/20005 | Loss: 4.5955 | LR: 5.00e-04 | 5785.7ms/step | 4.1 samples/s
Step 570/20005 | Loss: 4.6083 | LR: 5.00e-04 | 5784.2ms/step | 4.1 samples/s
Step 580/20005 | Loss: 4.5764 | LR: 5.00e-04 | 5786.4ms/step | 4.1 samples/s
Step 590/20005 | Loss: 4.5262 | LR: 5.00e-04 | 5786.6ms/step | 4.1 samples/s
Step 600/20005 | Loss: 4.5142 | LR: 5.00e-04 | 5782.1ms/step | 4.2 samples/s
Step 610/20005 | Loss: 4.4919 | LR: 5.00e-04 | 5786.4ms/step | 4.1 samples/s
Step 620/20005 | Loss: 4.4191 | LR: 5.00e-04 | 5782.6ms/step | 4.2 samples/s
Step 630/20005 | Loss: 4.4253 | LR: 5.00e-04 | 5788.1ms/step | 4.1 samples/s
Step 640/20005 | Loss: 4.3731 | LR: 5.00e-04 | 5783.4ms/step | 4.1 samples/s
Step 650/20005 | Loss: 4.3806 | LR: 4.99e-04 | 5782.5ms/step | 4.2 samples/s
Step 660/20005 | Loss: 4.3500 | LR: 4.99e-04 | 5783.6ms/step | 4.1 samples/s
Step 670/20005 | Loss: 4.3508 | LR: 4.99e-04 | 5783.1ms/step | 4.2 samples/s
Step 680/20005 | Loss: 4.3210 | LR: 4.99e-04 | 5782.6ms/step | 4.2 samples/s
Step 690/20005 | Loss: 4.2915 | LR: 4.99e-04 | 5783.5ms/step | 4.1 samples/s
Step 700/20005 | Loss: 4.2922 | LR: 4.99e-04 | 5783.1ms/step | 4.2 samples/s
Step 710/20005 | Loss: 4.2758 | LR: 4.99e-04 | 5781.8ms/step | 4.2 samples/s
Step 720/20005 | Loss: 4.2319 | LR: 4.99e-04 | 5783.1ms/step | 4.2 samples/s
Step 730/20005 | Loss: 4.2223 | LR: 4.99e-04 | 5781.1ms/step | 4.2 samples/s
Step 740/20005 | Loss: 4.2064 | LR: 4.99e-04 | 5781.3ms/step | 4.2 samples/s
Step 750/20005 | Loss: 4.1876 | LR: 4.99e-04 | 5779.5ms/step | 4.2 samples/s
Validation at step 750: Loss = 4.1875, Perplexity = 65.86
Step 760/20005 | Loss: 4.1786 | LR: 4.99e-04 | 40702.6ms/step | 0.6 samples/s
Step 770/20005 | Loss: 4.1912 | LR: 4.99e-04 | 5781.4ms/step | 4.2 samples/s
Step 780/20005 | Loss: 4.1686 | LR: 4.99e-04 | 5786.6ms/step | 4.1 samples/s
Step 790/20005 | Loss: 4.1423 | LR: 4.99e-04 | 5784.1ms/step | 4.1 samples/s
Step 800/20005 | Loss: 4.1443 | LR: 4.99e-04 | 5782.0ms/step | 4.2 samples/s
Step 810/20005 | Loss: 4.1128 | LR: 4.99e-04 | 5783.2ms/step | 4.1 samples/s
Step 820/20005 | Loss: 4.1002 | LR: 4.99e-04 | 5782.4ms/step | 4.2 samples/s
Step 830/20005 | Loss: 4.1025 | LR: 4.99e-04 | 5780.5ms/step | 4.2 samples/s
Step 840/20005 | Loss: 4.0407 | LR: 4.99e-04 | 5781.5ms/step | 4.2 samples/s
Step 850/20005 | Loss: 4.0566 | LR: 4.99e-04 | 5782.4ms/step | 4.2 samples/s
Step 860/20005 | Loss: 4.0384 | LR: 4.99e-04 | 5783.5ms/step | 4.1 samples/s
Step 870/20005 | Loss: 4.0470 | LR: 4.99e-04 | 5784.4ms/step | 4.1 samples/s
Step 880/20005 | Loss: 4.0458 | LR: 4.99e-04 | 5783.0ms/step | 4.2 samples/s
Step 890/20005 | Loss: 4.0228 | LR: 4.99e-04 | 5785.1ms/step | 4.1 samples/s
Step 900/20005 | Loss: 4.0273 | LR: 4.99e-04 | 5781.6ms/step | 4.2 samples/s
Step 910/20005 | Loss: 3.9987 | LR: 4.99e-04 | 5784.4ms/step | 4.1 samples/s
Step 920/20005 | Loss: 3.9998 | LR: 4.99e-04 | 5779.8ms/step | 4.2 samples/s
Step 930/20005 | Loss: 3.9725 | LR: 4.99e-04 | 5816.4ms/step | 4.1 samples/s
Step 940/20005 | Loss: 3.9699 | LR: 4.98e-04 | 5782.0ms/step | 4.2 samples/s
Step 950/20005 | Loss: 3.9666 | LR: 4.98e-04 | 5778.6ms/step | 4.2 samples/s
Step 960/20005 | Loss: 3.9306 | LR: 4.98e-04 | 5778.1ms/step | 4.2 samples/s
Step 970/20005 | Loss: 3.9618 | LR: 4.98e-04 | 5783.5ms/step | 4.1 samples/s
Step 980/20005 | Loss: 3.9529 | LR: 4.98e-04 | 5784.2ms/step | 4.1 samples/s
Step 990/20005 | Loss: 3.9356 | LR: 4.98e-04 | 5782.7ms/step | 4.2 samples/s
Step 1000/20005 | Loss: 3.9464 | LR: 4.98e-04 | 5784.8ms/step | 4.1 samples/s
Validation at step 1000: Loss = 3.9224, Perplexity = 50.52
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-1000
Step 1010/20005 | Loss: 3.9379 | LR: 4.98e-04 | 41189.6ms/step | 0.6 samples/s
Step 1020/20005 | Loss: 3.9283 | LR: 4.98e-04 | 5783.2ms/step | 4.1 samples/s
Step 1030/20005 | Loss: 3.8804 | LR: 4.98e-04 | 5781.1ms/step | 4.2 samples/s
Step 1040/20005 | Loss: 3.8940 | LR: 4.98e-04 | 5784.0ms/step | 4.1 samples/s
Step 1050/20005 | Loss: 3.8713 | LR: 4.98e-04 | 5783.1ms/step | 4.2 samples/s
Step 1060/20005 | Loss: 3.8799 | LR: 4.98e-04 | 5783.7ms/step | 4.1 samples/s
Step 1070/20005 | Loss: 3.8659 | LR: 4.98e-04 | 5780.2ms/step | 4.2 samples/s
Step 1080/20005 | Loss: 3.8717 | LR: 4.98e-04 | 5786.1ms/step | 4.1 samples/s
Step 1090/20005 | Loss: 3.8552 | LR: 4.98e-04 | 5781.6ms/step | 4.2 samples/s
Step 1100/20005 | Loss: 3.8767 | LR: 4.98e-04 | 5782.8ms/step | 4.2 samples/s
Step 1110/20005 | Loss: 3.8482 | LR: 4.98e-04 | 5781.7ms/step | 4.2 samples/s
Step 1120/20005 | Loss: 3.7999 | LR: 4.98e-04 | 5779.8ms/step | 4.2 samples/s
Step 1130/20005 | Loss: 3.8320 | LR: 4.98e-04 | 5778.3ms/step | 4.2 samples/s
Step 1140/20005 | Loss: 3.8323 | LR: 4.98e-04 | 5783.6ms/step | 4.1 samples/s
Step 1150/20005 | Loss: 3.8271 | LR: 4.97e-04 | 5781.9ms/step | 4.2 samples/s
Step 1160/20005 | Loss: 3.8239 | LR: 4.97e-04 | 5780.7ms/step | 4.2 samples/s
Step 1170/20005 | Loss: 3.8005 | LR: 4.97e-04 | 5779.8ms/step | 4.2 samples/s
Step 1180/20005 | Loss: 3.8194 | LR: 4.97e-04 | 5778.6ms/step | 4.2 samples/s
Step 1190/20005 | Loss: 3.7899 | LR: 4.97e-04 | 5780.5ms/step | 4.2 samples/s
Step 1200/20005 | Loss: 3.7880 | LR: 4.97e-04 | 5781.2ms/step | 4.2 samples/s
Step 1210/20005 | Loss: 3.7989 | LR: 4.97e-04 | 5782.1ms/step | 4.2 samples/s
Step 1220/20005 | Loss: 3.8084 | LR: 4.97e-04 | 5780.4ms/step | 4.2 samples/s
Step 1230/20005 | Loss: 3.7902 | LR: 4.97e-04 | 5779.7ms/step | 4.2 samples/s
Step 1240/20005 | Loss: 3.7925 | LR: 4.97e-04 | 5780.0ms/step | 4.2 samples/s
Step 1250/20005 | Loss: 3.7417 | LR: 4.97e-04 | 5780.1ms/step | 4.2 samples/s
Validation at step 1250: Loss = 3.7637, Perplexity = 43.11
Step 1260/20005 | Loss: 3.7645 | LR: 4.97e-04 | 40679.0ms/step | 0.6 samples/s
Step 1270/20005 | Loss: 3.7524 | LR: 4.97e-04 | 5784.9ms/step | 4.1 samples/s
Step 1280/20005 | Loss: 3.7250 | LR: 4.97e-04 | 5780.1ms/step | 4.2 samples/s
Step 1290/20005 | Loss: 3.7278 | LR: 4.97e-04 | 5778.3ms/step | 4.2 samples/s
Step 1300/20005 | Loss: 3.7402 | LR: 4.97e-04 | 5779.7ms/step | 4.2 samples/s
Step 1310/20005 | Loss: 3.7426 | LR: 4.96e-04 | 5780.7ms/step | 4.2 samples/s
Step 1320/20005 | Loss: 3.7539 | LR: 4.96e-04 | 5781.3ms/step | 4.2 samples/s
Step 1330/20005 | Loss: 3.7373 | LR: 4.96e-04 | 5777.1ms/step | 4.2 samples/s
Step 1340/20005 | Loss: 3.7387 | LR: 4.96e-04 | 5776.9ms/step | 4.2 samples/s
Step 1350/20005 | Loss: 3.7411 | LR: 4.96e-04 | 5778.3ms/step | 4.2 samples/s
Step 1360/20005 | Loss: 3.7518 | LR: 4.96e-04 | 5780.0ms/step | 4.2 samples/s
Step 1370/20005 | Loss: 3.7122 | LR: 4.96e-04 | 5780.6ms/step | 4.2 samples/s
Step 1380/20005 | Loss: 3.7095 | LR: 4.96e-04 | 5783.8ms/step | 4.1 samples/s
Step 1390/20005 | Loss: 3.6936 | LR: 4.96e-04 | 5782.0ms/step | 4.2 samples/s
Step 1400/20005 | Loss: 3.6938 | LR: 4.96e-04 | 5818.2ms/step | 4.1 samples/s
Step 1410/20005 | Loss: 3.7071 | LR: 4.96e-04 | 5785.6ms/step | 4.1 samples/s
Step 1420/20005 | Loss: 3.6940 | LR: 4.96e-04 | 5780.8ms/step | 4.2 samples/s
Step 1430/20005 | Loss: 3.6942 | LR: 4.96e-04 | 5781.6ms/step | 4.2 samples/s
Step 1440/20005 | Loss: 3.6913 | LR: 4.96e-04 | 5781.9ms/step | 4.2 samples/s
Step 1450/20005 | Loss: 3.6305 | LR: 4.95e-04 | 5782.1ms/step | 4.2 samples/s
Step 1460/20005 | Loss: 3.6922 | LR: 4.95e-04 | 5781.5ms/step | 4.2 samples/s
Step 1470/20005 | Loss: 3.6690 | LR: 4.95e-04 | 5780.6ms/step | 4.2 samples/s
Step 1480/20005 | Loss: 3.6491 | LR: 4.95e-04 | 5780.1ms/step | 4.2 samples/s
Step 1490/20005 | Loss: 3.6551 | LR: 4.95e-04 | 5779.6ms/step | 4.2 samples/s
Step 1500/20005 | Loss: 3.6634 | LR: 4.95e-04 | 5780.2ms/step | 4.2 samples/s
Validation at step 1500: Loss = 3.6573, Perplexity = 38.76
Step 1510/20005 | Loss: 3.6474 | LR: 4.95e-04 | 40690.4ms/step | 0.6 samples/s
Step 1520/20005 | Loss: 3.6222 | LR: 4.95e-04 | 5783.7ms/step | 4.1 samples/s
Step 1530/20005 | Loss: 3.6487 | LR: 4.95e-04 | 5783.7ms/step | 4.1 samples/s
Step 1540/20005 | Loss: 3.6358 | LR: 4.95e-04 | 5783.8ms/step | 4.1 samples/s
Step 1550/20005 | Loss: 3.6373 | LR: 4.95e-04 | 5782.5ms/step | 4.2 samples/s
Step 1560/20005 | Loss: 3.6638 | LR: 4.95e-04 | 5783.0ms/step | 4.2 samples/s
Step 1570/20005 | Loss: 3.6270 | LR: 4.95e-04 | 5781.7ms/step | 4.2 samples/s
Step 1580/20005 | Loss: 3.6423 | LR: 4.94e-04 | 5780.0ms/step | 4.2 samples/s
Step 1590/20005 | Loss: 3.6107 | LR: 4.94e-04 | 5783.9ms/step | 4.1 samples/s
Step 1600/20005 | Loss: 3.6389 | LR: 4.94e-04 | 5781.6ms/step | 4.2 samples/s
Step 1610/20005 | Loss: 3.6317 | LR: 4.94e-04 | 5781.8ms/step | 4.2 samples/s
Step 1620/20005 | Loss: 3.6117 | LR: 4.94e-04 | 5786.5ms/step | 4.1 samples/s
Step 1630/20005 | Loss: 3.6149 | LR: 4.94e-04 | 5779.5ms/step | 4.2 samples/s
Step 1640/20005 | Loss: 3.6086 | LR: 4.94e-04 | 5784.0ms/step | 4.1 samples/s
Step 1650/20005 | Loss: 3.5996 | LR: 4.94e-04 | 5783.0ms/step | 4.2 samples/s
Step 1660/20005 | Loss: 3.5914 | LR: 4.94e-04 | 5781.1ms/step | 4.2 samples/s
Step 1670/20005 | Loss: 3.6129 | LR: 4.94e-04 | 5780.6ms/step | 4.2 samples/s
Step 1680/20005 | Loss: 3.5788 | LR: 4.94e-04 | 5781.9ms/step | 4.2 samples/s
Step 1690/20005 | Loss: 3.6212 | LR: 4.93e-04 | 5781.4ms/step | 4.2 samples/s
Step 1700/20005 | Loss: 3.5592 | LR: 4.93e-04 | 5780.6ms/step | 4.2 samples/s
Step 1710/20005 | Loss: 3.5829 | LR: 4.93e-04 | 5779.8ms/step | 4.2 samples/s
Step 1720/20005 | Loss: 3.5877 | LR: 4.93e-04 | 5780.1ms/step | 4.2 samples/s
Step 1730/20005 | Loss: 3.5610 | LR: 4.93e-04 | 5783.6ms/step | 4.1 samples/s
Step 1740/20005 | Loss: 3.5795 | LR: 4.93e-04 | 5780.5ms/step | 4.2 samples/s
Step 1750/20005 | Loss: 3.5866 | LR: 4.93e-04 | 5782.7ms/step | 4.2 samples/s
Validation at step 1750: Loss = 3.5758, Perplexity = 35.72
Step 1760/20005 | Loss: 3.5752 | LR: 4.93e-04 | 40683.3ms/step | 0.6 samples/s
Step 1770/20005 | Loss: 3.5958 | LR: 4.93e-04 | 5785.6ms/step | 4.1 samples/s
Step 1780/20005 | Loss: 3.5697 | LR: 4.93e-04 | 5778.9ms/step | 4.2 samples/s
Step 1790/20005 | Loss: 3.5626 | LR: 4.93e-04 | 5781.2ms/step | 4.2 samples/s
Step 1800/20005 | Loss: 3.5596 | LR: 4.92e-04 | 5786.0ms/step | 4.1 samples/s
Step 1810/20005 | Loss: 3.5618 | LR: 4.92e-04 | 5784.5ms/step | 4.1 samples/s
Step 1820/20005 | Loss: 3.5583 | LR: 4.92e-04 | 5777.9ms/step | 4.2 samples/s
Step 1830/20005 | Loss: 3.5897 | LR: 4.92e-04 | 5785.5ms/step | 4.1 samples/s
Step 1840/20005 | Loss: 3.5410 | LR: 4.92e-04 | 5781.2ms/step | 4.2 samples/s
Step 1850/20005 | Loss: 3.5378 | LR: 4.92e-04 | 5782.7ms/step | 4.2 samples/s
Step 1860/20005 | Loss: 3.5860 | LR: 4.92e-04 | 5783.3ms/step | 4.1 samples/s
Step 1870/20005 | Loss: 3.5251 | LR: 4.92e-04 | 5823.5ms/step | 4.1 samples/s
Step 1880/20005 | Loss: 3.5514 | LR: 4.92e-04 | 5785.7ms/step | 4.1 samples/s
Step 1890/20005 | Loss: 3.5411 | LR: 4.92e-04 | 5782.6ms/step | 4.2 samples/s
Step 1900/20005 | Loss: 3.5334 | LR: 4.91e-04 | 5780.1ms/step | 4.2 samples/s
Step 1910/20005 | Loss: 3.5440 | LR: 4.91e-04 | 5781.3ms/step | 4.2 samples/s
Step 1920/20005 | Loss: 3.5162 | LR: 4.91e-04 | 5779.7ms/step | 4.2 samples/s
Step 1930/20005 | Loss: 3.5323 | LR: 4.91e-04 | 5782.7ms/step | 4.2 samples/s
Step 1940/20005 | Loss: 3.5083 | LR: 4.91e-04 | 5783.3ms/step | 4.1 samples/s
Step 1950/20005 | Loss: 3.5361 | LR: 4.91e-04 | 5783.6ms/step | 4.1 samples/s
Step 1960/20005 | Loss: 3.5149 | LR: 4.91e-04 | 5784.8ms/step | 4.1 samples/s
Step 1970/20005 | Loss: 3.5085 | LR: 4.91e-04 | 5785.1ms/step | 4.1 samples/s
Step 1980/20005 | Loss: 3.5505 | LR: 4.91e-04 | 5780.3ms/step | 4.2 samples/s
Step 1990/20005 | Loss: 3.5255 | LR: 4.90e-04 | 5777.8ms/step | 4.2 samples/s
Step 2000/20005 | Loss: 3.5122 | LR: 4.90e-04 | 5779.7ms/step | 4.2 samples/s
Validation at step 2000: Loss = 3.5099, Perplexity = 33.45
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-2000
Step 2010/20005 | Loss: 3.5317 | LR: 4.90e-04 | 41170.1ms/step | 0.6 samples/s
Step 2020/20005 | Loss: 3.5175 | LR: 4.90e-04 | 5778.6ms/step | 4.2 samples/s
Step 2030/20005 | Loss: 3.5105 | LR: 4.90e-04 | 5781.6ms/step | 4.2 samples/s
Step 2040/20005 | Loss: 3.4892 | LR: 4.90e-04 | 5779.1ms/step | 4.2 samples/s
Step 2050/20005 | Loss: 3.5104 | LR: 4.90e-04 | 5781.8ms/step | 4.2 samples/s
Step 2060/20005 | Loss: 3.5115 | LR: 4.90e-04 | 5782.0ms/step | 4.2 samples/s
Step 2070/20005 | Loss: 3.4715 | LR: 4.90e-04 | 5777.3ms/step | 4.2 samples/s
Step 2080/20005 | Loss: 3.4974 | LR: 4.89e-04 | 5780.4ms/step | 4.2 samples/s
Step 2090/20005 | Loss: 3.4960 | LR: 4.89e-04 | 5780.4ms/step | 4.2 samples/s
Step 2100/20005 | Loss: 3.4753 | LR: 4.89e-04 | 5781.4ms/step | 4.2 samples/s
Step 2110/20005 | Loss: 3.4859 | LR: 4.89e-04 | 5780.6ms/step | 4.2 samples/s
Step 2120/20005 | Loss: 3.4847 | LR: 4.89e-04 | 5779.5ms/step | 4.2 samples/s
Step 2130/20005 | Loss: 3.4797 | LR: 4.89e-04 | 5779.8ms/step | 4.2 samples/s
Step 2140/20005 | Loss: 3.4988 | LR: 4.89e-04 | 5779.8ms/step | 4.2 samples/s
Step 2150/20005 | Loss: 3.5022 | LR: 4.89e-04 | 5780.3ms/step | 4.2 samples/s
Step 2160/20005 | Loss: 3.4588 | LR: 4.89e-04 | 5778.3ms/step | 4.2 samples/s
Step 2170/20005 | Loss: 3.4649 | LR: 4.88e-04 | 5779.3ms/step | 4.2 samples/s
Step 2180/20005 | Loss: 3.4828 | LR: 4.88e-04 | 5780.3ms/step | 4.2 samples/s
Step 2190/20005 | Loss: 3.4671 | LR: 4.88e-04 | 5778.6ms/step | 4.2 samples/s
Step 2200/20005 | Loss: 3.4213 | LR: 4.88e-04 | 5781.0ms/step | 4.2 samples/s
Step 2210/20005 | Loss: 3.4618 | LR: 4.88e-04 | 5783.0ms/step | 4.2 samples/s
Step 2220/20005 | Loss: 3.4741 | LR: 4.88e-04 | 5780.2ms/step | 4.2 samples/s
Step 2230/20005 | Loss: 3.4620 | LR: 4.88e-04 | 5779.6ms/step | 4.2 samples/s
Step 2240/20005 | Loss: 3.4530 | LR: 4.88e-04 | 5780.5ms/step | 4.2 samples/s
Step 2250/20005 | Loss: 3.4504 | LR: 4.87e-04 | 5783.5ms/step | 4.1 samples/s
Validation at step 2250: Loss = 3.4584, Perplexity = 31.77
Step 2260/20005 | Loss: 3.4973 | LR: 4.87e-04 | 40642.2ms/step | 0.6 samples/s
Step 2270/20005 | Loss: 3.4511 | LR: 4.87e-04 | 5780.3ms/step | 4.2 samples/s
Step 2280/20005 | Loss: 3.4746 | LR: 4.87e-04 | 5779.3ms/step | 4.2 samples/s
Step 2290/20005 | Loss: 3.4008 | LR: 4.87e-04 | 5779.5ms/step | 4.2 samples/s
Step 2300/20005 | Loss: 3.4338 | LR: 4.87e-04 | 5778.5ms/step | 4.2 samples/s
Step 2310/20005 | Loss: 3.4714 | LR: 4.87e-04 | 5781.4ms/step | 4.2 samples/s
Step 2320/20005 | Loss: 3.4518 | LR: 4.87e-04 | 5778.7ms/step | 4.2 samples/s
Step 2330/20005 | Loss: 3.4093 | LR: 4.86e-04 | 5778.8ms/step | 4.2 samples/s
Step 2340/20005 | Loss: 3.4390 | LR: 4.86e-04 | 5818.1ms/step | 4.1 samples/s
Step 2350/20005 | Loss: 3.4233 | LR: 4.86e-04 | 5781.6ms/step | 4.2 samples/s
Step 2360/20005 | Loss: 3.4607 | LR: 4.86e-04 | 5781.4ms/step | 4.2 samples/s
Step 2370/20005 | Loss: 3.4440 | LR: 4.86e-04 | 5780.9ms/step | 4.2 samples/s
Step 2380/20005 | Loss: 3.4392 | LR: 4.86e-04 | 5781.3ms/step | 4.2 samples/s
Step 2390/20005 | Loss: 3.4055 | LR: 4.86e-04 | 5780.9ms/step | 4.2 samples/s
Step 2400/20005 | Loss: 3.4374 | LR: 4.86e-04 | 5782.2ms/step | 4.2 samples/s
Step 2410/20005 | Loss: 3.4270 | LR: 4.85e-04 | 5781.3ms/step | 4.2 samples/s
Step 2420/20005 | Loss: 3.4444 | LR: 4.85e-04 | 5781.1ms/step | 4.2 samples/s
Step 2430/20005 | Loss: 3.4257 | LR: 4.85e-04 | 5779.1ms/step | 4.2 samples/s
Step 2440/20005 | Loss: 3.4118 | LR: 4.85e-04 | 5780.1ms/step | 4.2 samples/s
Step 2450/20005 | Loss: 3.4131 | LR: 4.85e-04 | 5779.8ms/step | 4.2 samples/s
Step 2460/20005 | Loss: 3.4129 | LR: 4.85e-04 | 5780.6ms/step | 4.2 samples/s
Step 2470/20005 | Loss: 3.4214 | LR: 4.85e-04 | 5780.0ms/step | 4.2 samples/s
Step 2480/20005 | Loss: 3.4394 | LR: 4.84e-04 | 5782.1ms/step | 4.2 samples/s
Step 2490/20005 | Loss: 3.4300 | LR: 4.84e-04 | 5780.1ms/step | 4.2 samples/s
Step 2500/20005 | Loss: 3.4086 | LR: 4.84e-04 | 5781.4ms/step | 4.2 samples/s
Validation at step 2500: Loss = 3.4136, Perplexity = 30.37
Step 2510/20005 | Loss: 3.4290 | LR: 4.84e-04 | 40661.9ms/step | 0.6 samples/s
Step 2520/20005 | Loss: 3.3950 | LR: 4.84e-04 | 5779.6ms/step | 4.2 samples/s
Step 2530/20005 | Loss: 3.3939 | LR: 4.84e-04 | 5780.8ms/step | 4.2 samples/s
Step 2540/20005 | Loss: 3.4156 | LR: 4.84e-04 | 5778.7ms/step | 4.2 samples/s
Step 2550/20005 | Loss: 3.4088 | LR: 4.83e-04 | 5780.7ms/step | 4.2 samples/s
Step 2560/20005 | Loss: 3.4507 | LR: 4.83e-04 | 5780.1ms/step | 4.2 samples/s
Step 2570/20005 | Loss: 3.4034 | LR: 4.83e-04 | 5784.4ms/step | 4.1 samples/s
Step 2580/20005 | Loss: 3.3856 | LR: 4.83e-04 | 5778.6ms/step | 4.2 samples/s
Step 2590/20005 | Loss: 3.4028 | LR: 4.83e-04 | 5784.1ms/step | 4.1 samples/s
Step 2600/20005 | Loss: 3.4024 | LR: 4.83e-04 | 5782.8ms/step | 4.2 samples/s
Step 2610/20005 | Loss: 3.3858 | LR: 4.83e-04 | 5787.5ms/step | 4.1 samples/s
Step 2620/20005 | Loss: 3.4151 | LR: 4.82e-04 | 5783.3ms/step | 4.1 samples/s
Step 2630/20005 | Loss: 3.3988 | LR: 4.82e-04 | 5784.1ms/step | 4.1 samples/s
Step 2640/20005 | Loss: 3.4240 | LR: 4.82e-04 | 5782.7ms/step | 4.2 samples/s
Step 2650/20005 | Loss: 3.3830 | LR: 4.82e-04 | 5779.7ms/step | 4.2 samples/s
Step 2660/20005 | Loss: 3.3883 | LR: 4.82e-04 | 5784.1ms/step | 4.1 samples/s
Step 2670/20005 | Loss: 3.3883 | LR: 4.82e-04 | 5783.0ms/step | 4.2 samples/s
Step 2680/20005 | Loss: 3.3591 | LR: 4.82e-04 | 5781.0ms/step | 4.2 samples/s
Step 2690/20005 | Loss: 3.3965 | LR: 4.81e-04 | 5784.9ms/step | 4.1 samples/s
Step 2700/20005 | Loss: 3.3864 | LR: 4.81e-04 | 5785.2ms/step | 4.1 samples/s
Step 2710/20005 | Loss: 3.3813 | LR: 4.81e-04 | 5785.4ms/step | 4.1 samples/s
Step 2720/20005 | Loss: 3.3764 | LR: 4.81e-04 | 5787.6ms/step | 4.1 samples/s
Step 2730/20005 | Loss: 3.4121 | LR: 4.81e-04 | 5786.2ms/step | 4.1 samples/s
Step 2740/20005 | Loss: 3.3632 | LR: 4.81e-04 | 5786.8ms/step | 4.1 samples/s
Step 2750/20005 | Loss: 3.4235 | LR: 4.81e-04 | 5787.2ms/step | 4.1 samples/s
Validation at step 2750: Loss = 3.3779, Perplexity = 29.31
Step 2760/20005 | Loss: 3.3601 | LR: 4.80e-04 | 40667.5ms/step | 0.6 samples/s
Step 2770/20005 | Loss: 3.3928 | LR: 4.80e-04 | 5786.0ms/step | 4.1 samples/s
Step 2780/20005 | Loss: 3.3657 | LR: 4.80e-04 | 5782.1ms/step | 4.2 samples/s
Step 2790/20005 | Loss: 3.3482 | LR: 4.80e-04 | 5780.0ms/step | 4.2 samples/s
Step 2800/20005 | Loss: 3.3757 | LR: 4.80e-04 | 5783.4ms/step | 4.1 samples/s
Step 2810/20005 | Loss: 3.3955 | LR: 4.80e-04 | 5817.7ms/step | 4.1 samples/s
Step 2820/20005 | Loss: 3.3465 | LR: 4.79e-04 | 5781.1ms/step | 4.2 samples/s
Step 2830/20005 | Loss: 3.3777 | LR: 4.79e-04 | 5781.7ms/step | 4.2 samples/s
Step 2840/20005 | Loss: 3.3694 | LR: 4.79e-04 | 5781.4ms/step | 4.2 samples/s
Step 2850/20005 | Loss: 3.3689 | LR: 4.79e-04 | 5781.0ms/step | 4.2 samples/s
Step 2860/20005 | Loss: 3.3595 | LR: 4.79e-04 | 5783.1ms/step | 4.1 samples/s
Step 2870/20005 | Loss: 3.3610 | LR: 4.79e-04 | 5783.6ms/step | 4.1 samples/s
Step 2880/20005 | Loss: 3.3817 | LR: 4.78e-04 | 5784.6ms/step | 4.1 samples/s
Step 2890/20005 | Loss: 3.3666 | LR: 4.78e-04 | 5782.9ms/step | 4.2 samples/s
Step 2900/20005 | Loss: 3.3404 | LR: 4.78e-04 | 5786.4ms/step | 4.1 samples/s
Step 2910/20005 | Loss: 3.3343 | LR: 4.78e-04 | 5781.5ms/step | 4.2 samples/s
Step 2920/20005 | Loss: 3.3831 | LR: 4.78e-04 | 5783.0ms/step | 4.2 samples/s
Step 2930/20005 | Loss: 3.3275 | LR: 4.78e-04 | 5778.9ms/step | 4.2 samples/s
Step 2940/20005 | Loss: 3.3785 | LR: 4.77e-04 | 5782.7ms/step | 4.2 samples/s
Step 2950/20005 | Loss: 3.3393 | LR: 4.77e-04 | 5787.6ms/step | 4.1 samples/s
Step 2960/20005 | Loss: 3.3531 | LR: 4.77e-04 | 5786.8ms/step | 4.1 samples/s
Step 2970/20005 | Loss: 3.3645 | LR: 4.77e-04 | 5780.6ms/step | 4.2 samples/s
Step 2980/20005 | Loss: 3.3647 | LR: 4.77e-04 | 5781.2ms/step | 4.2 samples/s
Step 2990/20005 | Loss: 3.3242 | LR: 4.77e-04 | 5784.2ms/step | 4.1 samples/s
Step 3000/20005 | Loss: 3.3618 | LR: 4.76e-04 | 5783.5ms/step | 4.1 samples/s
Validation at step 3000: Loss = 3.3461, Perplexity = 28.39
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-3000
Step 3010/20005 | Loss: 3.3488 | LR: 4.76e-04 | 41235.6ms/step | 0.6 samples/s
Step 3020/20005 | Loss: 3.3191 | LR: 4.76e-04 | 5783.7ms/step | 4.1 samples/s
Step 3030/20005 | Loss: 3.3670 | LR: 4.76e-04 | 5784.4ms/step | 4.1 samples/s
Step 3040/20005 | Loss: 3.3724 | LR: 4.76e-04 | 5786.2ms/step | 4.1 samples/s
Step 3050/20005 | Loss: 3.3735 | LR: 4.76e-04 | 5784.5ms/step | 4.1 samples/s
Step 3060/20005 | Loss: 3.3373 | LR: 4.75e-04 | 5784.7ms/step | 4.1 samples/s
Step 3070/20005 | Loss: 3.3192 | LR: 4.75e-04 | 5783.4ms/step | 4.1 samples/s
Step 3080/20005 | Loss: 3.3312 | LR: 4.75e-04 | 5781.1ms/step | 4.2 samples/s
Step 3090/20005 | Loss: 3.3537 | LR: 4.75e-04 | 5782.5ms/step | 4.2 samples/s
Step 3100/20005 | Loss: 3.3229 | LR: 4.75e-04 | 5784.5ms/step | 4.1 samples/s
Step 3110/20005 | Loss: 3.3054 | LR: 4.75e-04 | 5787.0ms/step | 4.1 samples/s
Step 3120/20005 | Loss: 3.3378 | LR: 4.74e-04 | 5782.8ms/step | 4.2 samples/s
Step 3130/20005 | Loss: 3.3495 | LR: 4.74e-04 | 5785.4ms/step | 4.1 samples/s
Step 3140/20005 | Loss: 3.3412 | LR: 4.74e-04 | 5784.4ms/step | 4.1 samples/s
Step 3150/20005 | Loss: 3.3075 | LR: 4.74e-04 | 5786.6ms/step | 4.1 samples/s
Step 3160/20005 | Loss: 3.3261 | LR: 4.74e-04 | 5790.1ms/step | 4.1 samples/s
Step 3170/20005 | Loss: 3.3206 | LR: 4.74e-04 | 5786.0ms/step | 4.1 samples/s
Step 3180/20005 | Loss: 3.3385 | LR: 4.73e-04 | 5782.9ms/step | 4.2 samples/s
Step 3190/20005 | Loss: 3.3487 | LR: 4.73e-04 | 5785.4ms/step | 4.1 samples/s
Step 3200/20005 | Loss: 3.3467 | LR: 4.73e-04 | 5786.8ms/step | 4.1 samples/s
Step 3210/20005 | Loss: 3.3065 | LR: 4.73e-04 | 5782.3ms/step | 4.2 samples/s
Step 3220/20005 | Loss: 3.3220 | LR: 4.73e-04 | 5787.9ms/step | 4.1 samples/s
Step 3230/20005 | Loss: 3.3163 | LR: 4.72e-04 | 5786.0ms/step | 4.1 samples/s
Step 3240/20005 | Loss: 3.3043 | LR: 4.72e-04 | 5783.6ms/step | 4.1 samples/s
Step 3250/20005 | Loss: 3.3109 | LR: 4.72e-04 | 5783.1ms/step | 4.2 samples/s
Validation at step 3250: Loss = 3.3182, Perplexity = 27.61
Step 3260/20005 | Loss: 3.3135 | LR: 4.72e-04 | 40742.0ms/step | 0.6 samples/s
Step 3270/20005 | Loss: 3.3168 | LR: 4.72e-04 | 5786.2ms/step | 4.1 samples/s
Step 3280/20005 | Loss: 3.3076 | LR: 4.72e-04 | 5788.1ms/step | 4.1 samples/s
Step 3290/20005 | Loss: 3.3241 | LR: 4.71e-04 | 5861.9ms/step | 4.1 samples/s
Step 3300/20005 | Loss: 3.3013 | LR: 4.71e-04 | 5787.9ms/step | 4.1 samples/s
Step 3310/20005 | Loss: 3.3161 | LR: 4.71e-04 | 5782.4ms/step | 4.2 samples/s
Step 3320/20005 | Loss: 3.3284 | LR: 4.71e-04 | 5783.4ms/step | 4.1 samples/s
Step 3330/20005 | Loss: 3.3142 | LR: 4.71e-04 | 5787.8ms/step | 4.1 samples/s
Step 3340/20005 | Loss: 3.3072 | LR: 4.70e-04 | 5782.8ms/step | 4.2 samples/s
Step 3350/20005 | Loss: 3.2966 | LR: 4.70e-04 | 5783.6ms/step | 4.1 samples/s
Step 3360/20005 | Loss: 3.3033 | LR: 4.70e-04 | 5782.3ms/step | 4.2 samples/s
Step 3370/20005 | Loss: 3.3188 | LR: 4.70e-04 | 5782.4ms/step | 4.2 samples/s
Step 3380/20005 | Loss: 3.3094 | LR: 4.70e-04 | 5787.3ms/step | 4.1 samples/s
Step 3390/20005 | Loss: 3.3077 | LR: 4.69e-04 | 5786.1ms/step | 4.1 samples/s
Step 3400/20005 | Loss: 3.2989 | LR: 4.69e-04 | 5783.7ms/step | 4.1 samples/s
Step 3410/20005 | Loss: 3.2833 | LR: 4.69e-04 | 5781.2ms/step | 4.2 samples/s
Step 3420/20005 | Loss: 3.2820 | LR: 4.69e-04 | 5787.0ms/step | 4.1 samples/s
Step 3430/20005 | Loss: 3.2819 | LR: 4.69e-04 | 5783.9ms/step | 4.1 samples/s
Step 3440/20005 | Loss: 3.3050 | LR: 4.69e-04 | 5787.7ms/step | 4.1 samples/s
Step 3450/20005 | Loss: 3.2841 | LR: 4.68e-04 | 5783.0ms/step | 4.2 samples/s
Step 3460/20005 | Loss: 3.2691 | LR: 4.68e-04 | 5783.3ms/step | 4.1 samples/s
Step 3470/20005 | Loss: 3.2713 | LR: 4.68e-04 | 5784.5ms/step | 4.1 samples/s
Step 3480/20005 | Loss: 3.3070 | LR: 4.68e-04 | 5780.6ms/step | 4.2 samples/s
Step 3490/20005 | Loss: 3.2965 | LR: 4.68e-04 | 5785.1ms/step | 4.1 samples/s
Step 3500/20005 | Loss: 3.2941 | LR: 4.67e-04 | 5783.0ms/step | 4.2 samples/s
Validation at step 3500: Loss = 3.2905, Perplexity = 26.86
Step 3510/20005 | Loss: 3.2981 | LR: 4.67e-04 | 40741.7ms/step | 0.6 samples/s
Step 3520/20005 | Loss: 3.2834 | LR: 4.67e-04 | 5783.3ms/step | 4.1 samples/s
Step 3530/20005 | Loss: 3.2921 | LR: 4.67e-04 | 5787.2ms/step | 4.1 samples/s
Step 3540/20005 | Loss: 3.2984 | LR: 4.67e-04 | 5782.7ms/step | 4.2 samples/s
Step 3550/20005 | Loss: 3.2996 | LR: 4.66e-04 | 5783.3ms/step | 4.1 samples/s
Step 3560/20005 | Loss: 3.2792 | LR: 4.66e-04 | 5783.7ms/step | 4.1 samples/s
Step 3570/20005 | Loss: 3.2649 | LR: 4.66e-04 | 5787.3ms/step | 4.1 samples/s
Step 3580/20005 | Loss: 3.2855 | LR: 4.66e-04 | 5787.8ms/step | 4.1 samples/s
Step 3590/20005 | Loss: 3.2762 | LR: 4.66e-04 | 5783.1ms/step | 4.1 samples/s
Step 3600/20005 | Loss: 3.2993 | LR: 4.65e-04 | 5783.1ms/step | 4.1 samples/s
Step 3610/20005 | Loss: 3.2678 | LR: 4.65e-04 | 5781.1ms/step | 4.2 samples/s
Step 3620/20005 | Loss: 3.2680 | LR: 4.65e-04 | 5787.3ms/step | 4.1 samples/s
Step 3630/20005 | Loss: 3.2615 | LR: 4.65e-04 | 5784.7ms/step | 4.1 samples/s
Step 3640/20005 | Loss: 3.2790 | LR: 4.65e-04 | 5783.6ms/step | 4.1 samples/s
Step 3650/20005 | Loss: 3.2826 | LR: 4.64e-04 | 5784.6ms/step | 4.1 samples/s
Step 3660/20005 | Loss: 3.2567 | LR: 4.64e-04 | 5783.3ms/step | 4.1 samples/s
Step 3670/20005 | Loss: 3.2827 | LR: 4.64e-04 | 5784.5ms/step | 4.1 samples/s
Step 3680/20005 | Loss: 3.2776 | LR: 4.64e-04 | 5781.4ms/step | 4.2 samples/s
Step 3690/20005 | Loss: 3.2138 | LR: 4.64e-04 | 5784.7ms/step | 4.1 samples/s
Step 3700/20005 | Loss: 3.2168 | LR: 4.63e-04 | 5782.2ms/step | 4.2 samples/s
Step 3710/20005 | Loss: 3.2648 | LR: 4.63e-04 | 5785.5ms/step | 4.1 samples/s
Step 3720/20005 | Loss: 3.2535 | LR: 4.63e-04 | 5782.9ms/step | 4.2 samples/s
Step 3730/20005 | Loss: 3.2880 | LR: 4.63e-04 | 5783.4ms/step | 4.1 samples/s
Step 3740/20005 | Loss: 3.2749 | LR: 4.62e-04 | 5782.2ms/step | 4.2 samples/s
Step 3750/20005 | Loss: 3.2329 | LR: 4.62e-04 | 5784.7ms/step | 4.1 samples/s
Validation at step 3750: Loss = 3.2675, Perplexity = 26.24
Step 3760/20005 | Loss: 3.2754 | LR: 4.62e-04 | 40793.6ms/step | 0.6 samples/s
Step 3770/20005 | Loss: 3.2702 | LR: 4.62e-04 | 5787.5ms/step | 4.1 samples/s
Step 3780/20005 | Loss: 3.2487 | LR: 4.62e-04 | 5788.7ms/step | 4.1 samples/s
Step 3790/20005 | Loss: 3.2667 | LR: 4.61e-04 | 5787.5ms/step | 4.1 samples/s
Step 3800/20005 | Loss: 3.2465 | LR: 4.61e-04 | 5787.2ms/step | 4.1 samples/s
Step 3810/20005 | Loss: 3.2816 | LR: 4.61e-04 | 5784.9ms/step | 4.1 samples/s
Step 3820/20005 | Loss: 3.2757 | LR: 4.61e-04 | 5784.8ms/step | 4.1 samples/s
Step 3830/20005 | Loss: 3.2309 | LR: 4.61e-04 | 5782.1ms/step | 4.2 samples/s
Step 3840/20005 | Loss: 3.2804 | LR: 4.60e-04 | 5785.7ms/step | 4.1 samples/s
Step 3850/20005 | Loss: 3.2578 | LR: 4.60e-04 | 5785.0ms/step | 4.1 samples/s
Step 3860/20005 | Loss: 3.2330 | LR: 4.60e-04 | 5785.2ms/step | 4.1 samples/s
Step 3870/20005 | Loss: 3.2744 | LR: 4.60e-04 | 5787.8ms/step | 4.1 samples/s
Step 3880/20005 | Loss: 3.2258 | LR: 4.59e-04 | 5784.0ms/step | 4.1 samples/s
Step 3890/20005 | Loss: 3.2299 | LR: 4.59e-04 | 5785.6ms/step | 4.1 samples/s
Step 3900/20005 | Loss: 3.2596 | LR: 4.59e-04 | 5780.9ms/step | 4.2 samples/s
Step 3910/20005 | Loss: 3.2224 | LR: 4.59e-04 | 5785.4ms/step | 4.1 samples/s
Step 3920/20005 | Loss: 3.2420 | LR: 4.59e-04 | 5785.1ms/step | 4.1 samples/s
Step 3930/20005 | Loss: 3.2461 | LR: 4.58e-04 | 5783.3ms/step | 4.1 samples/s
Step 3940/20005 | Loss: 3.2508 | LR: 4.58e-04 | 5783.6ms/step | 4.1 samples/s
Step 3950/20005 | Loss: 3.2988 | LR: 4.58e-04 | 5783.8ms/step | 4.1 samples/s
Step 3960/20005 | Loss: 3.2730 | LR: 4.58e-04 | 5784.8ms/step | 4.1 samples/s
Step 3970/20005 | Loss: 3.2278 | LR: 4.58e-04 | 5787.2ms/step | 4.1 samples/s
Step 3980/20005 | Loss: 3.2163 | LR: 4.57e-04 | 5782.6ms/step | 4.2 samples/s
Step 3990/20005 | Loss: 3.2754 | LR: 4.57e-04 | 5782.1ms/step | 4.2 samples/s
Step 4000/20005 | Loss: 3.2230 | LR: 4.57e-04 | 5782.1ms/step | 4.2 samples/s
Validation at step 4000: Loss = 3.2485, Perplexity = 25.75
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-4000
Step 4010/20005 | Loss: 3.2798 | LR: 4.57e-04 | 41272.5ms/step | 0.6 samples/s
Step 4020/20005 | Loss: 3.2439 | LR: 4.56e-04 | 5784.7ms/step | 4.1 samples/s
Step 4030/20005 | Loss: 3.2447 | LR: 4.56e-04 | 5789.5ms/step | 4.1 samples/s
Step 4040/20005 | Loss: 3.2597 | LR: 4.56e-04 | 5786.8ms/step | 4.1 samples/s
Step 4050/20005 | Loss: 3.2521 | LR: 4.56e-04 | 5784.2ms/step | 4.1 samples/s
Step 4060/20005 | Loss: 3.2573 | LR: 4.55e-04 | 5787.8ms/step | 4.1 samples/s
Step 4070/20005 | Loss: 3.2595 | LR: 4.55e-04 | 5786.8ms/step | 4.1 samples/s
Step 4080/20005 | Loss: 3.2160 | LR: 4.55e-04 | 5783.8ms/step | 4.1 samples/s
Step 4090/20005 | Loss: 3.2460 | LR: 4.55e-04 | 5785.5ms/step | 4.1 samples/s
Step 4100/20005 | Loss: 3.2269 | LR: 4.55e-04 | 5785.3ms/step | 4.1 samples/s
Step 4110/20005 | Loss: 3.2394 | LR: 4.54e-04 | 5785.8ms/step | 4.1 samples/s
Step 4120/20005 | Loss: 3.2201 | LR: 4.54e-04 | 5785.7ms/step | 4.1 samples/s
Step 4130/20005 | Loss: 3.2235 | LR: 4.54e-04 | 5784.2ms/step | 4.1 samples/s
Step 4140/20005 | Loss: 3.2319 | LR: 4.54e-04 | 5787.3ms/step | 4.1 samples/s
Step 4150/20005 | Loss: 3.2214 | LR: 4.53e-04 | 5784.6ms/step | 4.1 samples/s
Step 4160/20005 | Loss: 3.2442 | LR: 4.53e-04 | 5787.6ms/step | 4.1 samples/s
Step 4170/20005 | Loss: 3.2461 | LR: 4.53e-04 | 5786.2ms/step | 4.1 samples/s
Step 4180/20005 | Loss: 3.2233 | LR: 4.53e-04 | 5784.9ms/step | 4.1 samples/s
Step 4190/20005 | Loss: 3.2057 | LR: 4.53e-04 | 5779.9ms/step | 4.2 samples/s
Step 4200/20005 | Loss: 3.2199 | LR: 4.52e-04 | 5781.3ms/step | 4.2 samples/s
Step 4210/20005 | Loss: 3.2294 | LR: 4.52e-04 | 5782.5ms/step | 4.2 samples/s
Step 4220/20005 | Loss: 3.1969 | LR: 4.52e-04 | 5787.6ms/step | 4.1 samples/s
Step 4230/20005 | Loss: 3.2317 | LR: 4.52e-04 | 5822.6ms/step | 4.1 samples/s
Step 4240/20005 | Loss: 3.2216 | LR: 4.51e-04 | 5783.8ms/step | 4.1 samples/s
Step 4250/20005 | Loss: 3.2561 | LR: 4.51e-04 | 5784.5ms/step | 4.1 samples/s
Validation at step 4250: Loss = 3.2273, Perplexity = 25.21
Step 4260/20005 | Loss: 3.2200 | LR: 4.51e-04 | 40749.4ms/step | 0.6 samples/s
Step 4270/20005 | Loss: 3.2249 | LR: 4.51e-04 | 5785.6ms/step | 4.1 samples/s
Step 4280/20005 | Loss: 3.2448 | LR: 4.50e-04 | 5785.0ms/step | 4.1 samples/s
Step 4290/20005 | Loss: 3.2298 | LR: 4.50e-04 | 5785.9ms/step | 4.1 samples/s
Step 4300/20005 | Loss: 3.2169 | LR: 4.50e-04 | 5783.0ms/step | 4.2 samples/s
Step 4310/20005 | Loss: 3.2067 | LR: 4.50e-04 | 5783.4ms/step | 4.1 samples/s
Step 4320/20005 | Loss: 3.2152 | LR: 4.49e-04 | 5783.3ms/step | 4.1 samples/s
Step 4330/20005 | Loss: 3.2195 | LR: 4.49e-04 | 5785.2ms/step | 4.1 samples/s
Step 4340/20005 | Loss: 3.2121 | LR: 4.49e-04 | 5783.5ms/step | 4.1 samples/s
Step 4350/20005 | Loss: 3.1714 | LR: 4.49e-04 | 5785.7ms/step | 4.1 samples/s
Step 4360/20005 | Loss: 3.2288 | LR: 4.48e-04 | 5785.3ms/step | 4.1 samples/s
Step 4370/20005 | Loss: 3.2225 | LR: 4.48e-04 | 5787.4ms/step | 4.1 samples/s
Step 4380/20005 | Loss: 3.2097 | LR: 4.48e-04 | 5784.8ms/step | 4.1 samples/s
Step 4390/20005 | Loss: 3.2342 | LR: 4.48e-04 | 5785.1ms/step | 4.1 samples/s
Step 4400/20005 | Loss: 3.1850 | LR: 4.48e-04 | 5786.6ms/step | 4.1 samples/s
Step 4410/20005 | Loss: 3.1960 | LR: 4.47e-04 | 5790.0ms/step | 4.1 samples/s
Step 4420/20005 | Loss: 3.1865 | LR: 4.47e-04 | 5787.4ms/step | 4.1 samples/s
Step 4430/20005 | Loss: 3.2088 | LR: 4.47e-04 | 5786.0ms/step | 4.1 samples/s
Step 4440/20005 | Loss: 3.2309 | LR: 4.47e-04 | 5788.7ms/step | 4.1 samples/s
Step 4450/20005 | Loss: 3.1863 | LR: 4.46e-04 | 5791.2ms/step | 4.1 samples/s
Step 4460/20005 | Loss: 3.2319 | LR: 4.46e-04 | 5787.1ms/step | 4.1 samples/s
Step 4470/20005 | Loss: 3.1823 | LR: 4.46e-04 | 5788.8ms/step | 4.1 samples/s
Step 4480/20005 | Loss: 3.2321 | LR: 4.46e-04 | 5787.7ms/step | 4.1 samples/s
Step 4490/20005 | Loss: 3.2027 | LR: 4.45e-04 | 5783.5ms/step | 4.1 samples/s
Step 4500/20005 | Loss: 3.1744 | LR: 4.45e-04 | 5791.2ms/step | 4.1 samples/s
Validation at step 4500: Loss = 3.2093, Perplexity = 24.76
Step 4510/20005 | Loss: 3.1880 | LR: 4.45e-04 | 40702.1ms/step | 0.6 samples/s
Step 4520/20005 | Loss: 3.1917 | LR: 4.45e-04 | 5785.0ms/step | 4.1 samples/s
Step 4530/20005 | Loss: 3.2258 | LR: 4.44e-04 | 5786.4ms/step | 4.1 samples/s
Step 4540/20005 | Loss: 3.2124 | LR: 4.44e-04 | 5781.1ms/step | 4.2 samples/s
Step 4550/20005 | Loss: 3.2003 | LR: 4.44e-04 | 5786.9ms/step | 4.1 samples/s
Step 4560/20005 | Loss: 3.2043 | LR: 4.44e-04 | 5781.1ms/step | 4.2 samples/s
Step 4570/20005 | Loss: 3.2126 | LR: 4.43e-04 | 5783.7ms/step | 4.1 samples/s
Step 4580/20005 | Loss: 3.2145 | LR: 4.43e-04 | 5784.2ms/step | 4.1 samples/s
Step 4590/20005 | Loss: 3.1959 | LR: 4.43e-04 | 5785.8ms/step | 4.1 samples/s
Step 4600/20005 | Loss: 3.1762 | LR: 4.43e-04 | 5785.6ms/step | 4.1 samples/s
Step 4610/20005 | Loss: 3.2017 | LR: 4.42e-04 | 5786.9ms/step | 4.1 samples/s
Step 4620/20005 | Loss: 3.2234 | LR: 4.42e-04 | 5783.2ms/step | 4.1 samples/s
Step 4630/20005 | Loss: 3.1999 | LR: 4.42e-04 | 5785.3ms/step | 4.1 samples/s
Step 4640/20005 | Loss: 3.2124 | LR: 4.42e-04 | 5787.7ms/step | 4.1 samples/s
Step 4650/20005 | Loss: 3.2139 | LR: 4.41e-04 | 5784.7ms/step | 4.1 samples/s
Step 4660/20005 | Loss: 3.2094 | LR: 4.41e-04 | 5783.8ms/step | 4.1 samples/s
Step 4670/20005 | Loss: 3.1884 | LR: 4.41e-04 | 5781.4ms/step | 4.2 samples/s
Step 4680/20005 | Loss: 3.1737 | LR: 4.40e-04 | 5779.9ms/step | 4.2 samples/s
Step 4690/20005 | Loss: 3.1836 | LR: 4.40e-04 | 5784.1ms/step | 4.1 samples/s
Step 4700/20005 | Loss: 3.1781 | LR: 4.40e-04 | 5784.4ms/step | 4.1 samples/s
Step 4710/20005 | Loss: 3.2189 | LR: 4.40e-04 | 5845.7ms/step | 4.1 samples/s
Step 4720/20005 | Loss: 3.1837 | LR: 4.39e-04 | 5781.7ms/step | 4.2 samples/s
Step 4730/20005 | Loss: 3.2009 | LR: 4.39e-04 | 5782.0ms/step | 4.2 samples/s
Step 4740/20005 | Loss: 3.1537 | LR: 4.39e-04 | 5779.8ms/step | 4.2 samples/s
Step 4750/20005 | Loss: 3.1827 | LR: 4.39e-04 | 5783.9ms/step | 4.1 samples/s
Validation at step 4750: Loss = 3.1931, Perplexity = 24.36
Step 4760/20005 | Loss: 3.2032 | LR: 4.38e-04 | 40669.9ms/step | 0.6 samples/s
Step 4770/20005 | Loss: 3.2182 | LR: 4.38e-04 | 5780.8ms/step | 4.2 samples/s
Step 4780/20005 | Loss: 3.1923 | LR: 4.38e-04 | 5782.1ms/step | 4.2 samples/s
Step 4790/20005 | Loss: 3.2062 | LR: 4.38e-04 | 5783.1ms/step | 4.2 samples/s
Step 4800/20005 | Loss: 3.1561 | LR: 4.37e-04 | 5781.4ms/step | 4.2 samples/s
Step 4810/20005 | Loss: 3.1705 | LR: 4.37e-04 | 5777.6ms/step | 4.2 samples/s
Step 4820/20005 | Loss: 3.1642 | LR: 4.37e-04 | 5780.1ms/step | 4.2 samples/s
Step 4830/20005 | Loss: 3.1712 | LR: 4.37e-04 | 5781.9ms/step | 4.2 samples/s
Step 4840/20005 | Loss: 3.2025 | LR: 4.36e-04 | 5782.3ms/step | 4.2 samples/s
Step 4850/20005 | Loss: 3.1605 | LR: 4.36e-04 | 5783.6ms/step | 4.1 samples/s
Step 4860/20005 | Loss: 3.1784 | LR: 4.36e-04 | 5779.8ms/step | 4.2 samples/s
Step 4870/20005 | Loss: 3.1904 | LR: 4.36e-04 | 5780.1ms/step | 4.2 samples/s
Step 4880/20005 | Loss: 3.1701 | LR: 4.35e-04 | 5781.5ms/step | 4.2 samples/s
Step 4890/20005 | Loss: 3.1663 | LR: 4.35e-04 | 5781.2ms/step | 4.2 samples/s
Step 4900/20005 | Loss: 3.1454 | LR: 4.35e-04 | 5779.9ms/step | 4.2 samples/s
Step 4910/20005 | Loss: 3.2031 | LR: 4.34e-04 | 5781.4ms/step | 4.2 samples/s
Step 4920/20005 | Loss: 3.1672 | LR: 4.34e-04 | 5779.4ms/step | 4.2 samples/s
Step 4930/20005 | Loss: 3.1722 | LR: 4.34e-04 | 5781.7ms/step | 4.2 samples/s
Step 4940/20005 | Loss: 3.1946 | LR: 4.34e-04 | 5779.2ms/step | 4.2 samples/s
Step 4950/20005 | Loss: 3.1961 | LR: 4.33e-04 | 5782.6ms/step | 4.2 samples/s
Step 4960/20005 | Loss: 3.1564 | LR: 4.33e-04 | 5783.9ms/step | 4.1 samples/s
Step 4970/20005 | Loss: 3.1929 | LR: 4.33e-04 | 5781.0ms/step | 4.2 samples/s
Step 4980/20005 | Loss: 3.1829 | LR: 4.33e-04 | 5779.1ms/step | 4.2 samples/s
Step 4990/20005 | Loss: 3.1806 | LR: 4.32e-04 | 5780.6ms/step | 4.2 samples/s
Step 5000/20005 | Loss: 3.1953 | LR: 4.32e-04 | 5781.7ms/step | 4.2 samples/s
Validation at step 5000: Loss = 3.1788, Perplexity = 24.02
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-5000
Step 5010/20005 | Loss: 3.1568 | LR: 4.32e-04 | 41150.6ms/step | 0.6 samples/s
Step 5020/20005 | Loss: 3.1581 | LR: 4.31e-04 | 5780.0ms/step | 4.2 samples/s
Step 5030/20005 | Loss: 3.1555 | LR: 4.31e-04 | 5779.8ms/step | 4.2 samples/s
Step 5040/20005 | Loss: 3.1853 | LR: 4.31e-04 | 5779.9ms/step | 4.2 samples/s
Step 5050/20005 | Loss: 3.1785 | LR: 4.31e-04 | 5779.6ms/step | 4.2 samples/s
Step 5060/20005 | Loss: 3.1881 | LR: 4.30e-04 | 5781.3ms/step | 4.2 samples/s
Step 5070/20005 | Loss: 3.1764 | LR: 4.30e-04 | 5782.4ms/step | 4.2 samples/s
Step 5080/20005 | Loss: 3.1656 | LR: 4.30e-04 | 5780.9ms/step | 4.2 samples/s
Step 5090/20005 | Loss: 3.1908 | LR: 4.30e-04 | 5780.5ms/step | 4.2 samples/s
Step 5100/20005 | Loss: 3.1665 | LR: 4.29e-04 | 5781.3ms/step | 4.2 samples/s
Step 5110/20005 | Loss: 3.1911 | LR: 4.29e-04 | 5782.4ms/step | 4.2 samples/s
Step 5120/20005 | Loss: 3.1637 | LR: 4.29e-04 | 5779.9ms/step | 4.2 samples/s
Step 5130/20005 | Loss: 3.1741 | LR: 4.28e-04 | 5779.8ms/step | 4.2 samples/s
Step 5140/20005 | Loss: 3.1717 | LR: 4.28e-04 | 5780.2ms/step | 4.2 samples/s
Step 5150/20005 | Loss: 3.1658 | LR: 4.28e-04 | 5781.7ms/step | 4.2 samples/s
Step 5160/20005 | Loss: 3.1530 | LR: 4.28e-04 | 5778.4ms/step | 4.2 samples/s
Step 5170/20005 | Loss: 3.1905 | LR: 4.27e-04 | 5779.2ms/step | 4.2 samples/s
Step 5180/20005 | Loss: 3.1539 | LR: 4.27e-04 | 5831.5ms/step | 4.1 samples/s
Step 5190/20005 | Loss: 3.1911 | LR: 4.27e-04 | 5779.8ms/step | 4.2 samples/s
Step 5200/20005 | Loss: 3.1485 | LR: 4.26e-04 | 5781.5ms/step | 4.2 samples/s
Step 5210/20005 | Loss: 3.1615 | LR: 4.26e-04 | 5782.1ms/step | 4.2 samples/s
Step 5220/20005 | Loss: 3.1679 | LR: 4.26e-04 | 5780.0ms/step | 4.2 samples/s
Step 5230/20005 | Loss: 3.1398 | LR: 4.26e-04 | 5781.9ms/step | 4.2 samples/s
Step 5240/20005 | Loss: 3.1801 | LR: 4.25e-04 | 5783.7ms/step | 4.1 samples/s
Step 5250/20005 | Loss: 3.1857 | LR: 4.25e-04 | 5782.1ms/step | 4.2 samples/s
Validation at step 5250: Loss = 3.1635, Perplexity = 23.65
Step 5260/20005 | Loss: 3.1569 | LR: 4.25e-04 | 40651.1ms/step | 0.6 samples/s
Step 5270/20005 | Loss: 3.1375 | LR: 4.24e-04 | 5783.6ms/step | 4.1 samples/s
Step 5280/20005 | Loss: 3.1602 | LR: 4.24e-04 | 5781.7ms/step | 4.2 samples/s
Step 5290/20005 | Loss: 3.1612 | LR: 4.24e-04 | 5781.4ms/step | 4.2 samples/s
Step 5300/20005 | Loss: 3.1474 | LR: 4.24e-04 | 5781.7ms/step | 4.2 samples/s
Step 5310/20005 | Loss: 3.1654 | LR: 4.23e-04 | 5781.7ms/step | 4.2 samples/s
Step 5320/20005 | Loss: 3.1592 | LR: 4.23e-04 | 5782.1ms/step | 4.2 samples/s
Step 5330/20005 | Loss: 3.1643 | LR: 4.23e-04 | 5781.2ms/step | 4.2 samples/s
Step 5340/20005 | Loss: 3.1771 | LR: 4.22e-04 | 5781.9ms/step | 4.2 samples/s
Step 5350/20005 | Loss: 3.1420 | LR: 4.22e-04 | 5780.8ms/step | 4.2 samples/s
Step 5360/20005 | Loss: 3.1722 | LR: 4.22e-04 | 5780.2ms/step | 4.2 samples/s
Step 5370/20005 | Loss: 3.1735 | LR: 4.22e-04 | 5779.6ms/step | 4.2 samples/s
Step 5380/20005 | Loss: 3.1516 | LR: 4.21e-04 | 5781.6ms/step | 4.2 samples/s
Step 5390/20005 | Loss: 3.1115 | LR: 4.21e-04 | 5778.7ms/step | 4.2 samples/s
Step 5400/20005 | Loss: 3.1619 | LR: 4.21e-04 | 5782.4ms/step | 4.2 samples/s
Step 5410/20005 | Loss: 3.1566 | LR: 4.20e-04 | 5779.7ms/step | 4.2 samples/s
Step 5420/20005 | Loss: 3.1282 | LR: 4.20e-04 | 5782.2ms/step | 4.2 samples/s
Step 5430/20005 | Loss: 3.1586 | LR: 4.20e-04 | 5780.2ms/step | 4.2 samples/s
Step 5440/20005 | Loss: 3.1515 | LR: 4.20e-04 | 5782.4ms/step | 4.2 samples/s
Step 5450/20005 | Loss: 3.1698 | LR: 4.19e-04 | 5783.9ms/step | 4.1 samples/s
Step 5460/20005 | Loss: 3.1405 | LR: 4.19e-04 | 5781.4ms/step | 4.2 samples/s
Step 5470/20005 | Loss: 3.1586 | LR: 4.19e-04 | 5778.5ms/step | 4.2 samples/s
Step 5480/20005 | Loss: 3.1677 | LR: 4.18e-04 | 5779.8ms/step | 4.2 samples/s
Step 5490/20005 | Loss: 3.1325 | LR: 4.18e-04 | 5779.8ms/step | 4.2 samples/s
Step 5500/20005 | Loss: 3.1444 | LR: 4.18e-04 | 5782.1ms/step | 4.2 samples/s
Validation at step 5500: Loss = 3.1489, Perplexity = 23.31
Step 5510/20005 | Loss: 3.1545 | LR: 4.18e-04 | 40644.2ms/step | 0.6 samples/s
Step 5520/20005 | Loss: 3.1556 | LR: 4.17e-04 | 5778.4ms/step | 4.2 samples/s
Step 5530/20005 | Loss: 3.1451 | LR: 4.17e-04 | 5777.7ms/step | 4.2 samples/s
Step 5540/20005 | Loss: 3.1181 | LR: 4.17e-04 | 5777.7ms/step | 4.2 samples/s
Step 5550/20005 | Loss: 3.1337 | LR: 4.16e-04 | 5781.2ms/step | 4.2 samples/s
Step 5560/20005 | Loss: 3.1672 | LR: 4.16e-04 | 5781.9ms/step | 4.2 samples/s
Step 5570/20005 | Loss: 3.1542 | LR: 4.16e-04 | 5785.0ms/step | 4.1 samples/s
Step 5580/20005 | Loss: 3.1159 | LR: 4.15e-04 | 5780.8ms/step | 4.2 samples/s
Step 5590/20005 | Loss: 3.1339 | LR: 4.15e-04 | 5781.0ms/step | 4.2 samples/s
Step 5600/20005 | Loss: 3.1454 | LR: 4.15e-04 | 5782.6ms/step | 4.2 samples/s
Step 5610/20005 | Loss: 3.1414 | LR: 4.15e-04 | 5784.4ms/step | 4.1 samples/s
Step 5620/20005 | Loss: 3.1183 | LR: 4.14e-04 | 5785.7ms/step | 4.1 samples/s
Step 5630/20005 | Loss: 3.1272 | LR: 4.14e-04 | 5779.8ms/step | 4.2 samples/s
Step 5640/20005 | Loss: 3.1340 | LR: 4.14e-04 | 5781.7ms/step | 4.2 samples/s
Step 5650/20005 | Loss: 3.1683 | LR: 4.13e-04 | 5824.5ms/step | 4.1 samples/s
Step 5660/20005 | Loss: 3.1284 | LR: 4.13e-04 | 5784.1ms/step | 4.1 samples/s
Step 5670/20005 | Loss: 3.1451 | LR: 4.13e-04 | 5781.9ms/step | 4.2 samples/s
Step 5680/20005 | Loss: 3.1274 | LR: 4.12e-04 | 5786.4ms/step | 4.1 samples/s
Step 5690/20005 | Loss: 3.1643 | LR: 4.12e-04 | 5782.2ms/step | 4.2 samples/s
Step 5700/20005 | Loss: 3.1318 | LR: 4.12e-04 | 5782.6ms/step | 4.2 samples/s
Step 5710/20005 | Loss: 3.1239 | LR: 4.12e-04 | 5782.4ms/step | 4.2 samples/s
Step 5720/20005 | Loss: 3.1062 | LR: 4.11e-04 | 5780.0ms/step | 4.2 samples/s
Step 5730/20005 | Loss: 3.1399 | LR: 4.11e-04 | 5784.2ms/step | 4.1 samples/s
Step 5740/20005 | Loss: 3.1474 | LR: 4.11e-04 | 5784.7ms/step | 4.1 samples/s
Step 5750/20005 | Loss: 3.1159 | LR: 4.10e-04 | 5785.6ms/step | 4.1 samples/s
Validation at step 5750: Loss = 3.1354, Perplexity = 23.00
Step 5760/20005 | Loss: 3.1036 | LR: 4.10e-04 | 40713.9ms/step | 0.6 samples/s
Step 5770/20005 | Loss: 3.1409 | LR: 4.10e-04 | 5783.6ms/step | 4.1 samples/s
Step 5780/20005 | Loss: 3.1665 | LR: 4.09e-04 | 5787.1ms/step | 4.1 samples/s
Step 5790/20005 | Loss: 3.1079 | LR: 4.09e-04 | 5789.9ms/step | 4.1 samples/s
Step 5800/20005 | Loss: 3.1283 | LR: 4.09e-04 | 5786.4ms/step | 4.1 samples/s
Step 5810/20005 | Loss: 3.1651 | LR: 4.08e-04 | 5788.6ms/step | 4.1 samples/s
Step 5820/20005 | Loss: 3.1369 | LR: 4.08e-04 | 5782.5ms/step | 4.2 samples/s
Step 5830/20005 | Loss: 3.1500 | LR: 4.08e-04 | 5785.5ms/step | 4.1 samples/s
Step 5840/20005 | Loss: 3.1435 | LR: 4.08e-04 | 5782.8ms/step | 4.2 samples/s
Step 5850/20005 | Loss: 3.1336 | LR: 4.07e-04 | 5785.4ms/step | 4.1 samples/s
Step 5860/20005 | Loss: 3.1414 | LR: 4.07e-04 | 5784.6ms/step | 4.1 samples/s
Step 5870/20005 | Loss: 3.1488 | LR: 4.07e-04 | 5785.1ms/step | 4.1 samples/s
Step 5880/20005 | Loss: 3.1186 | LR: 4.06e-04 | 5781.8ms/step | 4.2 samples/s
Step 5890/20005 | Loss: 3.1419 | LR: 4.06e-04 | 5783.1ms/step | 4.1 samples/s
Step 5900/20005 | Loss: 3.1040 | LR: 4.06e-04 | 5782.6ms/step | 4.2 samples/s
Step 5910/20005 | Loss: 3.1318 | LR: 4.05e-04 | 5783.9ms/step | 4.1 samples/s
Step 5920/20005 | Loss: 3.1176 | LR: 4.05e-04 | 5781.6ms/step | 4.2 samples/s
Step 5930/20005 | Loss: 3.1256 | LR: 4.05e-04 | 5783.7ms/step | 4.1 samples/s
Step 5940/20005 | Loss: 3.1153 | LR: 4.04e-04 | 5783.7ms/step | 4.1 samples/s
Step 5950/20005 | Loss: 3.1342 | LR: 4.04e-04 | 5781.6ms/step | 4.2 samples/s
Step 5960/20005 | Loss: 3.1254 | LR: 4.04e-04 | 5784.8ms/step | 4.1 samples/s
Step 5970/20005 | Loss: 3.1141 | LR: 4.04e-04 | 5782.5ms/step | 4.2 samples/s
Step 5980/20005 | Loss: 3.1351 | LR: 4.03e-04 | 5781.6ms/step | 4.2 samples/s
Step 5990/20005 | Loss: 3.1056 | LR: 4.03e-04 | 5781.0ms/step | 4.2 samples/s
Step 6000/20005 | Loss: 3.0929 | LR: 4.03e-04 | 5778.4ms/step | 4.2 samples/s
Validation at step 6000: Loss = 3.1232, Perplexity = 22.72
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-6000
Step 6010/20005 | Loss: 3.1181 | LR: 4.02e-04 | 41179.8ms/step | 0.6 samples/s
Step 6020/20005 | Loss: 3.1279 | LR: 4.02e-04 | 5783.6ms/step | 4.1 samples/s
Step 6030/20005 | Loss: 3.1124 | LR: 4.02e-04 | 5780.8ms/step | 4.2 samples/s
Step 6040/20005 | Loss: 3.1277 | LR: 4.01e-04 | 5783.6ms/step | 4.1 samples/s
Step 6050/20005 | Loss: 3.1408 | LR: 4.01e-04 | 5786.1ms/step | 4.1 samples/s
Step 6060/20005 | Loss: 3.0992 | LR: 4.01e-04 | 5784.1ms/step | 4.1 samples/s
Step 6070/20005 | Loss: 3.1230 | LR: 4.00e-04 | 5783.3ms/step | 4.1 samples/s
Step 6080/20005 | Loss: 3.1151 | LR: 4.00e-04 | 5784.1ms/step | 4.1 samples/s
Step 6090/20005 | Loss: 3.1343 | LR: 4.00e-04 | 5784.1ms/step | 4.1 samples/s
Step 6100/20005 | Loss: 3.1033 | LR: 3.99e-04 | 5786.3ms/step | 4.1 samples/s
Step 6110/20005 | Loss: 3.1322 | LR: 3.99e-04 | 5780.8ms/step | 4.2 samples/s
Step 6120/20005 | Loss: 3.1345 | LR: 3.99e-04 | 5779.6ms/step | 4.2 samples/s
Step 6130/20005 | Loss: 3.1033 | LR: 3.98e-04 | 5822.1ms/step | 4.1 samples/s
Step 6140/20005 | Loss: 3.1078 | LR: 3.98e-04 | 5781.2ms/step | 4.2 samples/s
Step 6150/20005 | Loss: 3.1303 | LR: 3.98e-04 | 5785.0ms/step | 4.1 samples/s
Step 6160/20005 | Loss: 3.1275 | LR: 3.97e-04 | 5787.5ms/step | 4.1 samples/s
Step 6170/20005 | Loss: 3.1197 | LR: 3.97e-04 | 5784.7ms/step | 4.1 samples/s
Step 6180/20005 | Loss: 3.1188 | LR: 3.97e-04 | 5783.5ms/step | 4.1 samples/s
Step 6190/20005 | Loss: 3.1225 | LR: 3.97e-04 | 5783.6ms/step | 4.1 samples/s
Step 6200/20005 | Loss: 3.1328 | LR: 3.96e-04 | 5786.8ms/step | 4.1 samples/s
Step 6210/20005 | Loss: 3.1088 | LR: 3.96e-04 | 5783.5ms/step | 4.1 samples/s
Step 6220/20005 | Loss: 3.1267 | LR: 3.96e-04 | 5783.9ms/step | 4.1 samples/s
Step 6230/20005 | Loss: 3.0924 | LR: 3.95e-04 | 5784.7ms/step | 4.1 samples/s
Step 6240/20005 | Loss: 3.0699 | LR: 3.95e-04 | 5782.6ms/step | 4.2 samples/s
Step 6250/20005 | Loss: 3.1005 | LR: 3.95e-04 | 5781.3ms/step | 4.2 samples/s
Validation at step 6250: Loss = 3.1125, Perplexity = 22.48
Step 6260/20005 | Loss: 3.1000 | LR: 3.94e-04 | 40695.6ms/step | 0.6 samples/s
Step 6270/20005 | Loss: 3.0979 | LR: 3.94e-04 | 5782.9ms/step | 4.2 samples/s
Step 6280/20005 | Loss: 3.1277 | LR: 3.94e-04 | 5779.7ms/step | 4.2 samples/s
Step 6290/20005 | Loss: 3.1204 | LR: 3.93e-04 | 5782.3ms/step | 4.2 samples/s
Step 6300/20005 | Loss: 3.1070 | LR: 3.93e-04 | 5783.7ms/step | 4.1 samples/s
Step 6310/20005 | Loss: 3.1050 | LR: 3.93e-04 | 5784.4ms/step | 4.1 samples/s
Step 6320/20005 | Loss: 3.1131 | LR: 3.92e-04 | 5784.3ms/step | 4.1 samples/s
Step 6330/20005 | Loss: 3.1239 | LR: 3.92e-04 | 5785.7ms/step | 4.1 samples/s
Step 6340/20005 | Loss: 3.1050 | LR: 3.92e-04 | 5782.0ms/step | 4.2 samples/s
Step 6350/20005 | Loss: 3.0929 | LR: 3.91e-04 | 5781.4ms/step | 4.2 samples/s
Step 6360/20005 | Loss: 3.1280 | LR: 3.91e-04 | 5782.4ms/step | 4.2 samples/s
Step 6370/20005 | Loss: 3.0857 | LR: 3.91e-04 | 5783.1ms/step | 4.2 samples/s
Step 6380/20005 | Loss: 3.0962 | LR: 3.90e-04 | 5778.5ms/step | 4.2 samples/s
Step 6390/20005 | Loss: 3.0795 | LR: 3.90e-04 | 5784.1ms/step | 4.1 samples/s
Step 6400/20005 | Loss: 3.0981 | LR: 3.90e-04 | 5784.6ms/step | 4.1 samples/s
Step 6410/20005 | Loss: 3.1048 | LR: 3.89e-04 | 5786.2ms/step | 4.1 samples/s
Step 6420/20005 | Loss: 3.1174 | LR: 3.89e-04 | 5783.9ms/step | 4.1 samples/s
Step 6430/20005 | Loss: 3.1022 | LR: 3.89e-04 | 5781.9ms/step | 4.2 samples/s
Step 6440/20005 | Loss: 3.1144 | LR: 3.88e-04 | 5782.1ms/step | 4.2 samples/s
Step 6450/20005 | Loss: 3.0879 | LR: 3.88e-04 | 5788.0ms/step | 4.1 samples/s
Step 6460/20005 | Loss: 3.0863 | LR: 3.88e-04 | 5788.1ms/step | 4.1 samples/s
Step 6470/20005 | Loss: 3.1090 | LR: 3.87e-04 | 5789.4ms/step | 4.1 samples/s
Step 6480/20005 | Loss: 3.1184 | LR: 3.87e-04 | 5786.5ms/step | 4.1 samples/s
Step 6490/20005 | Loss: 3.1028 | LR: 3.87e-04 | 5788.1ms/step | 4.1 samples/s
Step 6500/20005 | Loss: 3.1011 | LR: 3.86e-04 | 5784.5ms/step | 4.1 samples/s
Validation at step 6500: Loss = 3.1011, Perplexity = 22.22
Step 6510/20005 | Loss: 3.0853 | LR: 3.86e-04 | 40778.5ms/step | 0.6 samples/s
Step 6520/20005 | Loss: 3.0961 | LR: 3.86e-04 | 5791.6ms/step | 4.1 samples/s
Step 6530/20005 | Loss: 3.1140 | LR: 3.85e-04 | 5783.7ms/step | 4.1 samples/s
Step 6540/20005 | Loss: 3.0918 | LR: 3.85e-04 | 5785.6ms/step | 4.1 samples/s
Step 6550/20005 | Loss: 3.1171 | LR: 3.85e-04 | 5788.2ms/step | 4.1 samples/s
Step 6560/20005 | Loss: 3.1087 | LR: 3.84e-04 | 5789.4ms/step | 4.1 samples/s
Step 6570/20005 | Loss: 3.0573 | LR: 3.84e-04 | 5786.3ms/step | 4.1 samples/s
Step 6580/20005 | Loss: 3.0896 | LR: 3.84e-04 | 5800.7ms/step | 4.1 samples/s
Step 6590/20005 | Loss: 3.1200 | LR: 3.83e-04 | 5786.3ms/step | 4.1 samples/s
Step 6600/20005 | Loss: 3.0988 | LR: 3.83e-04 | 5838.8ms/step | 4.1 samples/s
Step 6610/20005 | Loss: 3.0903 | LR: 3.83e-04 | 5786.2ms/step | 4.1 samples/s
Step 6620/20005 | Loss: 3.0966 | LR: 3.82e-04 | 5788.3ms/step | 4.1 samples/s
Step 6630/20005 | Loss: 3.1068 | LR: 3.82e-04 | 5788.4ms/step | 4.1 samples/s
Step 6640/20005 | Loss: 3.1084 | LR: 3.82e-04 | 5784.2ms/step | 4.1 samples/s
Step 6650/20005 | Loss: 3.1056 | LR: 3.81e-04 | 5787.5ms/step | 4.1 samples/s
Step 6660/20005 | Loss: 3.1083 | LR: 3.81e-04 | 5785.1ms/step | 4.1 samples/s
Step 6670/20005 | Loss: 3.1001 | LR: 3.81e-04 | 5792.1ms/step | 4.1 samples/s
Step 6680/20005 | Loss: 3.1122 | LR: 3.80e-04 | 5789.9ms/step | 4.1 samples/s
Step 6690/20005 | Loss: 3.0982 | LR: 3.80e-04 | 5786.6ms/step | 4.1 samples/s
Step 6700/20005 | Loss: 3.1053 | LR: 3.80e-04 | 5783.3ms/step | 4.1 samples/s
Step 6710/20005 | Loss: 3.1106 | LR: 3.79e-04 | 5787.9ms/step | 4.1 samples/s
Step 6720/20005 | Loss: 3.0591 | LR: 3.79e-04 | 5793.2ms/step | 4.1 samples/s
Step 6730/20005 | Loss: 3.0701 | LR: 3.79e-04 | 5784.6ms/step | 4.1 samples/s
Step 6740/20005 | Loss: 3.1003 | LR: 3.78e-04 | 5781.6ms/step | 4.2 samples/s
Step 6750/20005 | Loss: 3.0871 | LR: 3.78e-04 | 5781.9ms/step | 4.2 samples/s
Validation at step 6750: Loss = 3.0902, Perplexity = 21.98
Step 6760/20005 | Loss: 3.1024 | LR: 3.78e-04 | 40689.6ms/step | 0.6 samples/s
Step 6770/20005 | Loss: 3.0894 | LR: 3.77e-04 | 5784.3ms/step | 4.1 samples/s
Step 6780/20005 | Loss: 3.0844 | LR: 3.77e-04 | 5784.4ms/step | 4.1 samples/s
Step 6790/20005 | Loss: 3.0960 | LR: 3.77e-04 | 5782.9ms/step | 4.2 samples/s
Step 6800/20005 | Loss: 3.0614 | LR: 3.76e-04 | 5785.0ms/step | 4.1 samples/s
Step 6810/20005 | Loss: 3.0948 | LR: 3.76e-04 | 5784.0ms/step | 4.1 samples/s
Step 6820/20005 | Loss: 3.0779 | LR: 3.76e-04 | 5784.4ms/step | 4.1 samples/s
Step 6830/20005 | Loss: 3.0725 | LR: 3.75e-04 | 5784.5ms/step | 4.1 samples/s
Step 6840/20005 | Loss: 3.1000 | LR: 3.75e-04 | 5781.4ms/step | 4.2 samples/s
Step 6850/20005 | Loss: 3.0885 | LR: 3.74e-04 | 5784.8ms/step | 4.1 samples/s
Step 6860/20005 | Loss: 3.0897 | LR: 3.74e-04 | 5784.0ms/step | 4.1 samples/s
Step 6870/20005 | Loss: 3.0558 | LR: 3.74e-04 | 5782.1ms/step | 4.2 samples/s
Step 6880/20005 | Loss: 3.0785 | LR: 3.73e-04 | 5785.4ms/step | 4.1 samples/s
Step 6890/20005 | Loss: 3.0972 | LR: 3.73e-04 | 5781.7ms/step | 4.2 samples/s
Step 6900/20005 | Loss: 3.0509 | LR: 3.73e-04 | 5784.3ms/step | 4.1 samples/s
Step 6910/20005 | Loss: 3.1001 | LR: 3.72e-04 | 5785.1ms/step | 4.1 samples/s
Step 6920/20005 | Loss: 3.1017 | LR: 3.72e-04 | 5783.0ms/step | 4.2 samples/s
Step 6930/20005 | Loss: 3.0854 | LR: 3.72e-04 | 5782.0ms/step | 4.2 samples/s
Step 6940/20005 | Loss: 3.0767 | LR: 3.71e-04 | 5786.0ms/step | 4.1 samples/s
Step 6950/20005 | Loss: 3.0664 | LR: 3.71e-04 | 5785.9ms/step | 4.1 samples/s
Step 6960/20005 | Loss: 3.0827 | LR: 3.71e-04 | 5785.1ms/step | 4.1 samples/s
Step 6970/20005 | Loss: 3.0864 | LR: 3.70e-04 | 5783.1ms/step | 4.2 samples/s
Step 6980/20005 | Loss: 3.0941 | LR: 3.70e-04 | 5782.8ms/step | 4.2 samples/s
Step 6990/20005 | Loss: 3.0755 | LR: 3.70e-04 | 5783.8ms/step | 4.1 samples/s
Step 7000/20005 | Loss: 3.1338 | LR: 3.69e-04 | 5784.1ms/step | 4.1 samples/s
Validation at step 7000: Loss = 3.0807, Perplexity = 21.77
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-7000
Step 7010/20005 | Loss: 3.0802 | LR: 3.69e-04 | 41193.0ms/step | 0.6 samples/s
Step 7020/20005 | Loss: 3.0717 | LR: 3.69e-04 | 5790.7ms/step | 4.1 samples/s
Step 7030/20005 | Loss: 3.0910 | LR: 3.68e-04 | 5786.5ms/step | 4.1 samples/s
Step 7040/20005 | Loss: 3.0748 | LR: 3.68e-04 | 5782.5ms/step | 4.2 samples/s
Step 7050/20005 | Loss: 3.0886 | LR: 3.68e-04 | 5783.1ms/step | 4.2 samples/s
Step 7060/20005 | Loss: 3.0908 | LR: 3.67e-04 | 5783.9ms/step | 4.1 samples/s
Step 7070/20005 | Loss: 3.0958 | LR: 3.67e-04 | 5784.3ms/step | 4.1 samples/s
Step 7080/20005 | Loss: 3.0695 | LR: 3.66e-04 | 5817.2ms/step | 4.1 samples/s
Step 7090/20005 | Loss: 3.0643 | LR: 3.66e-04 | 5779.6ms/step | 4.2 samples/s
Step 7100/20005 | Loss: 3.0907 | LR: 3.66e-04 | 5779.9ms/step | 4.2 samples/s
Step 7110/20005 | Loss: 3.0695 | LR: 3.65e-04 | 5780.9ms/step | 4.2 samples/s
Step 7120/20005 | Loss: 3.0768 | LR: 3.65e-04 | 5783.5ms/step | 4.1 samples/s
Step 7130/20005 | Loss: 3.0718 | LR: 3.65e-04 | 5781.7ms/step | 4.2 samples/s
Step 7140/20005 | Loss: 3.0777 | LR: 3.64e-04 | 5780.2ms/step | 4.2 samples/s
Step 7150/20005 | Loss: 3.0555 | LR: 3.64e-04 | 5783.6ms/step | 4.1 samples/s
Step 7160/20005 | Loss: 3.0763 | LR: 3.64e-04 | 5785.7ms/step | 4.1 samples/s
Step 7170/20005 | Loss: 3.0901 | LR: 3.63e-04 | 5779.4ms/step | 4.2 samples/s
Step 7180/20005 | Loss: 3.0724 | LR: 3.63e-04 | 5784.2ms/step | 4.1 samples/s
Step 7190/20005 | Loss: 3.0811 | LR: 3.63e-04 | 5785.0ms/step | 4.1 samples/s
Step 7200/20005 | Loss: 3.0541 | LR: 3.62e-04 | 5781.8ms/step | 4.2 samples/s
Step 7210/20005 | Loss: 3.0861 | LR: 3.62e-04 | 5779.9ms/step | 4.2 samples/s
Step 7220/20005 | Loss: 3.0666 | LR: 3.62e-04 | 5780.5ms/step | 4.2 samples/s
Step 7230/20005 | Loss: 3.0697 | LR: 3.61e-04 | 5782.8ms/step | 4.2 samples/s
Step 7240/20005 | Loss: 3.0566 | LR: 3.61e-04 | 5783.0ms/step | 4.2 samples/s
Step 7250/20005 | Loss: 3.0766 | LR: 3.60e-04 | 5783.0ms/step | 4.2 samples/s
Validation at step 7250: Loss = 3.0696, Perplexity = 21.53
Step 7260/20005 | Loss: 3.0650 | LR: 3.60e-04 | 40715.3ms/step | 0.6 samples/s
Step 7270/20005 | Loss: 3.0930 | LR: 3.60e-04 | 5783.8ms/step | 4.1 samples/s
Step 7280/20005 | Loss: 3.0524 | LR: 3.59e-04 | 5783.5ms/step | 4.1 samples/s
Step 7290/20005 | Loss: 3.0827 | LR: 3.59e-04 | 5783.4ms/step | 4.1 samples/s
Step 7300/20005 | Loss: 3.0754 | LR: 3.59e-04 | 5781.9ms/step | 4.2 samples/s
Step 7310/20005 | Loss: 3.0597 | LR: 3.58e-04 | 5781.2ms/step | 4.2 samples/s
Step 7320/20005 | Loss: 3.0540 | LR: 3.58e-04 | 5781.8ms/step | 4.2 samples/s
Step 7330/20005 | Loss: 3.0702 | LR: 3.58e-04 | 5784.5ms/step | 4.1 samples/s
Step 7340/20005 | Loss: 3.0710 | LR: 3.57e-04 | 5782.3ms/step | 4.2 samples/s
Step 7350/20005 | Loss: 3.0557 | LR: 3.57e-04 | 5782.8ms/step | 4.2 samples/s
Step 7360/20005 | Loss: 3.0344 | LR: 3.57e-04 | 5783.7ms/step | 4.1 samples/s
Step 7370/20005 | Loss: 3.0570 | LR: 3.56e-04 | 5786.9ms/step | 4.1 samples/s
Step 7380/20005 | Loss: 3.0635 | LR: 3.56e-04 | 5789.3ms/step | 4.1 samples/s
Step 7390/20005 | Loss: 3.0905 | LR: 3.55e-04 | 5782.3ms/step | 4.2 samples/s
Step 7400/20005 | Loss: 3.0686 | LR: 3.55e-04 | 5780.2ms/step | 4.2 samples/s
Step 7410/20005 | Loss: 3.0676 | LR: 3.55e-04 | 5779.3ms/step | 4.2 samples/s
Step 7420/20005 | Loss: 3.0511 | LR: 3.54e-04 | 5777.7ms/step | 4.2 samples/s
Step 7430/20005 | Loss: 3.0880 | LR: 3.54e-04 | 5784.8ms/step | 4.1 samples/s
Step 7440/20005 | Loss: 3.0536 | LR: 3.54e-04 | 5785.0ms/step | 4.1 samples/s
Step 7450/20005 | Loss: 3.0688 | LR: 3.53e-04 | 5784.0ms/step | 4.1 samples/s
Step 7460/20005 | Loss: 3.0638 | LR: 3.53e-04 | 5785.2ms/step | 4.1 samples/s
Step 7470/20005 | Loss: 3.0530 | LR: 3.53e-04 | 5785.3ms/step | 4.1 samples/s
Step 7480/20005 | Loss: 3.0227 | LR: 3.52e-04 | 5783.8ms/step | 4.1 samples/s
Step 7490/20005 | Loss: 3.0590 | LR: 3.52e-04 | 5784.4ms/step | 4.1 samples/s
Step 7500/20005 | Loss: 3.0622 | LR: 3.51e-04 | 5782.6ms/step | 4.2 samples/s
Validation at step 7500: Loss = 3.0600, Perplexity = 21.33
Step 7510/20005 | Loss: 3.0494 | LR: 3.51e-04 | 40685.3ms/step | 0.6 samples/s
Step 7520/20005 | Loss: 3.0726 | LR: 3.51e-04 | 5781.1ms/step | 4.2 samples/s
Step 7530/20005 | Loss: 3.0537 | LR: 3.50e-04 | 5784.3ms/step | 4.1 samples/s
Step 7540/20005 | Loss: 3.0655 | LR: 3.50e-04 | 5783.5ms/step | 4.1 samples/s
Step 7550/20005 | Loss: 3.0840 | LR: 3.50e-04 | 5821.2ms/step | 4.1 samples/s
Step 7560/20005 | Loss: 3.0684 | LR: 3.49e-04 | 5784.2ms/step | 4.1 samples/s
Step 7570/20005 | Loss: 3.0706 | LR: 3.49e-04 | 5782.0ms/step | 4.2 samples/s
Step 7580/20005 | Loss: 3.0802 | LR: 3.49e-04 | 5784.1ms/step | 4.1 samples/s
Step 7590/20005 | Loss: 3.0782 | LR: 3.48e-04 | 5780.5ms/step | 4.2 samples/s
Step 7600/20005 | Loss: 3.0726 | LR: 3.48e-04 | 5784.3ms/step | 4.1 samples/s
Step 7610/20005 | Loss: 3.0661 | LR: 3.47e-04 | 5782.0ms/step | 4.2 samples/s
Step 7620/20005 | Loss: 3.0368 | LR: 3.47e-04 | 5781.7ms/step | 4.2 samples/s
Step 7630/20005 | Loss: 3.0701 | LR: 3.47e-04 | 5781.9ms/step | 4.2 samples/s
Step 7640/20005 | Loss: 3.0564 | LR: 3.46e-04 | 5780.5ms/step | 4.2 samples/s
Step 7650/20005 | Loss: 3.0631 | LR: 3.46e-04 | 5781.7ms/step | 4.2 samples/s
Step 7660/20005 | Loss: 3.0579 | LR: 3.46e-04 | 5782.1ms/step | 4.2 samples/s
Step 7670/20005 | Loss: 3.0673 | LR: 3.45e-04 | 5782.0ms/step | 4.2 samples/s
Step 7680/20005 | Loss: 3.0497 | LR: 3.45e-04 | 5783.1ms/step | 4.2 samples/s
Step 7690/20005 | Loss: 3.0295 | LR: 3.44e-04 | 5784.5ms/step | 4.1 samples/s
Step 7700/20005 | Loss: 3.0427 | LR: 3.44e-04 | 5784.7ms/step | 4.1 samples/s
Step 7710/20005 | Loss: 3.0721 | LR: 3.44e-04 | 5786.2ms/step | 4.1 samples/s
Step 7720/20005 | Loss: 3.0271 | LR: 3.43e-04 | 5781.3ms/step | 4.2 samples/s
Step 7730/20005 | Loss: 3.0506 | LR: 3.43e-04 | 5790.7ms/step | 4.1 samples/s
Step 7740/20005 | Loss: 3.0506 | LR: 3.43e-04 | 5790.0ms/step | 4.1 samples/s
Step 7750/20005 | Loss: 3.0185 | LR: 3.42e-04 | 5789.8ms/step | 4.1 samples/s
Validation at step 7750: Loss = 3.0522, Perplexity = 21.16
Step 7760/20005 | Loss: 3.0467 | LR: 3.42e-04 | 40757.9ms/step | 0.6 samples/s
Step 7770/20005 | Loss: 3.0577 | LR: 3.42e-04 | 5789.4ms/step | 4.1 samples/s
Step 7780/20005 | Loss: 3.0442 | LR: 3.41e-04 | 5785.1ms/step | 4.1 samples/s
Step 7790/20005 | Loss: 3.0652 | LR: 3.41e-04 | 5783.4ms/step | 4.1 samples/s
Step 7800/20005 | Loss: 3.0498 | LR: 3.40e-04 | 5786.6ms/step | 4.1 samples/s
Step 7810/20005 | Loss: 3.0261 | LR: 3.40e-04 | 5787.8ms/step | 4.1 samples/s
Step 7820/20005 | Loss: 3.0488 | LR: 3.40e-04 | 5788.1ms/step | 4.1 samples/s
Step 7830/20005 | Loss: 3.0581 | LR: 3.39e-04 | 5790.1ms/step | 4.1 samples/s
Step 7840/20005 | Loss: 3.0548 | LR: 3.39e-04 | 5793.8ms/step | 4.1 samples/s
Step 7850/20005 | Loss: 3.0504 | LR: 3.39e-04 | 5785.3ms/step | 4.1 samples/s
Step 7860/20005 | Loss: 3.0421 | LR: 3.38e-04 | 5782.8ms/step | 4.2 samples/s
Step 7870/20005 | Loss: 3.0369 | LR: 3.38e-04 | 5784.6ms/step | 4.1 samples/s
Step 7880/20005 | Loss: 3.0636 | LR: 3.37e-04 | 5791.1ms/step | 4.1 samples/s
Step 7890/20005 | Loss: 3.0499 | LR: 3.37e-04 | 5788.8ms/step | 4.1 samples/s
Step 7900/20005 | Loss: 3.0501 | LR: 3.37e-04 | 5786.9ms/step | 4.1 samples/s
Step 7910/20005 | Loss: 3.0398 | LR: 3.36e-04 | 5789.1ms/step | 4.1 samples/s
Step 7920/20005 | Loss: 3.0624 | LR: 3.36e-04 | 5790.9ms/step | 4.1 samples/s
Step 7930/20005 | Loss: 3.0413 | LR: 3.36e-04 | 5785.4ms/step | 4.1 samples/s
Step 7940/20005 | Loss: 3.0630 | LR: 3.35e-04 | 5782.1ms/step | 4.2 samples/s
Step 7950/20005 | Loss: 3.0645 | LR: 3.35e-04 | 5779.7ms/step | 4.2 samples/s
Step 7960/20005 | Loss: 3.0472 | LR: 3.34e-04 | 5781.0ms/step | 4.2 samples/s
Step 7970/20005 | Loss: 3.0431 | LR: 3.34e-04 | 5780.6ms/step | 4.2 samples/s
Step 7980/20005 | Loss: 3.0382 | LR: 3.34e-04 | 5781.3ms/step | 4.2 samples/s
Step 7990/20005 | Loss: 3.0193 | LR: 3.33e-04 | 5781.9ms/step | 4.2 samples/s
Step 8000/20005 | Loss: 3.0531 | LR: 3.33e-04 | 5781.4ms/step | 4.2 samples/s
Validation at step 8000: Loss = 3.0427, Perplexity = 20.96
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-8000
Step 8010/20005 | Loss: 3.0476 | LR: 3.33e-04 | 41139.5ms/step | 0.6 samples/s
Step 8020/20005 | Loss: 3.0353 | LR: 3.32e-04 | 5778.4ms/step | 4.2 samples/s
Step 8030/20005 | Loss: 3.0032 | LR: 3.32e-04 | 5822.5ms/step | 4.1 samples/s
Step 8040/20005 | Loss: 3.0443 | LR: 3.31e-04 | 5781.3ms/step | 4.2 samples/s
Step 8050/20005 | Loss: 3.0002 | LR: 3.31e-04 | 5782.5ms/step | 4.2 samples/s
Step 8060/20005 | Loss: 3.0689 | LR: 3.31e-04 | 5781.7ms/step | 4.2 samples/s
Step 8070/20005 | Loss: 3.0390 | LR: 3.30e-04 | 5782.4ms/step | 4.2 samples/s
Step 8080/20005 | Loss: 3.0518 | LR: 3.30e-04 | 5779.9ms/step | 4.2 samples/s
Step 8090/20005 | Loss: 3.0070 | LR: 3.30e-04 | 5784.4ms/step | 4.1 samples/s
Step 8100/20005 | Loss: 3.0304 | LR: 3.29e-04 | 5781.8ms/step | 4.2 samples/s
Step 8110/20005 | Loss: 3.0329 | LR: 3.29e-04 | 5779.1ms/step | 4.2 samples/s
Step 8120/20005 | Loss: 3.0371 | LR: 3.28e-04 | 5779.1ms/step | 4.2 samples/s
Step 8130/20005 | Loss: 3.0342 | LR: 3.28e-04 | 5780.5ms/step | 4.2 samples/s
Step 8140/20005 | Loss: 3.0145 | LR: 3.28e-04 | 5781.8ms/step | 4.2 samples/s
Step 8150/20005 | Loss: 3.0473 | LR: 3.27e-04 | 5781.9ms/step | 4.2 samples/s
Step 8160/20005 | Loss: 3.0199 | LR: 3.27e-04 | 5783.0ms/step | 4.2 samples/s
Step 8170/20005 | Loss: 3.0127 | LR: 3.27e-04 | 5784.2ms/step | 4.1 samples/s
Step 8180/20005 | Loss: 3.0553 | LR: 3.26e-04 | 5785.6ms/step | 4.1 samples/s
Step 8190/20005 | Loss: 3.0345 | LR: 3.26e-04 | 5781.4ms/step | 4.2 samples/s
Step 8200/20005 | Loss: 3.0286 | LR: 3.25e-04 | 5780.9ms/step | 4.2 samples/s
Step 8210/20005 | Loss: 3.0153 | LR: 3.25e-04 | 5781.0ms/step | 4.2 samples/s
Step 8220/20005 | Loss: 3.0146 | LR: 3.25e-04 | 5781.8ms/step | 4.2 samples/s
Step 8230/20005 | Loss: 3.0349 | LR: 3.24e-04 | 5784.3ms/step | 4.1 samples/s
Step 8240/20005 | Loss: 3.0301 | LR: 3.24e-04 | 5784.3ms/step | 4.1 samples/s
Step 8250/20005 | Loss: 3.0383 | LR: 3.24e-04 | 5782.0ms/step | 4.2 samples/s
Validation at step 8250: Loss = 3.0360, Perplexity = 20.82
Step 8260/20005 | Loss: 3.0508 | LR: 3.23e-04 | 40674.1ms/step | 0.6 samples/s
Step 8270/20005 | Loss: 3.0533 | LR: 3.23e-04 | 5781.3ms/step | 4.2 samples/s
Step 8280/20005 | Loss: 3.0562 | LR: 3.22e-04 | 5783.8ms/step | 4.1 samples/s
Step 8290/20005 | Loss: 3.0131 | LR: 3.22e-04 | 5777.9ms/step | 4.2 samples/s
Step 8300/20005 | Loss: 3.0438 | LR: 3.22e-04 | 5780.2ms/step | 4.2 samples/s
Step 8310/20005 | Loss: 3.0576 | LR: 3.21e-04 | 5784.4ms/step | 4.1 samples/s
Step 8320/20005 | Loss: 3.0313 | LR: 3.21e-04 | 5782.6ms/step | 4.2 samples/s
Step 8330/20005 | Loss: 3.0252 | LR: 3.20e-04 | 5783.7ms/step | 4.1 samples/s
Step 8340/20005 | Loss: 3.0545 | LR: 3.20e-04 | 5780.4ms/step | 4.2 samples/s
Step 8350/20005 | Loss: 3.0269 | LR: 3.20e-04 | 5782.6ms/step | 4.2 samples/s
Step 8360/20005 | Loss: 3.0411 | LR: 3.19e-04 | 5781.1ms/step | 4.2 samples/s
Step 8370/20005 | Loss: 3.0189 | LR: 3.19e-04 | 5784.0ms/step | 4.1 samples/s
Step 8380/20005 | Loss: 3.0423 | LR: 3.19e-04 | 5783.0ms/step | 4.2 samples/s
Step 8390/20005 | Loss: 3.0246 | LR: 3.18e-04 | 5779.2ms/step | 4.2 samples/s
Step 8400/20005 | Loss: 3.0116 | LR: 3.18e-04 | 5784.9ms/step | 4.1 samples/s
Step 8410/20005 | Loss: 3.0371 | LR: 3.17e-04 | 5784.2ms/step | 4.1 samples/s
Step 8420/20005 | Loss: 3.0325 | LR: 3.17e-04 | 5780.6ms/step | 4.2 samples/s
Step 8430/20005 | Loss: 3.0510 | LR: 3.17e-04 | 5780.2ms/step | 4.2 samples/s
Step 8440/20005 | Loss: 3.0163 | LR: 3.16e-04 | 5777.5ms/step | 4.2 samples/s
Step 8450/20005 | Loss: 3.0462 | LR: 3.16e-04 | 5778.8ms/step | 4.2 samples/s
Step 8460/20005 | Loss: 3.0045 | LR: 3.16e-04 | 5781.8ms/step | 4.2 samples/s
Step 8470/20005 | Loss: 3.0111 | LR: 3.15e-04 | 5779.4ms/step | 4.2 samples/s
Step 8480/20005 | Loss: 3.0334 | LR: 3.15e-04 | 5784.4ms/step | 4.1 samples/s
Step 8490/20005 | Loss: 3.0013 | LR: 3.14e-04 | 5783.3ms/step | 4.1 samples/s
Step 8500/20005 | Loss: 3.0266 | LR: 3.14e-04 | 5820.9ms/step | 4.1 samples/s
Validation at step 8500: Loss = 3.0256, Perplexity = 20.61
Step 8510/20005 | Loss: 3.0076 | LR: 3.14e-04 | 40717.8ms/step | 0.6 samples/s
Step 8520/20005 | Loss: 3.0567 | LR: 3.13e-04 | 5778.7ms/step | 4.2 samples/s
Step 8530/20005 | Loss: 3.0220 | LR: 3.13e-04 | 5780.4ms/step | 4.2 samples/s
Step 8540/20005 | Loss: 3.0176 | LR: 3.12e-04 | 5779.7ms/step | 4.2 samples/s
Step 8550/20005 | Loss: 3.0414 | LR: 3.12e-04 | 5782.0ms/step | 4.2 samples/s
Step 8560/20005 | Loss: 3.0342 | LR: 3.12e-04 | 5779.5ms/step | 4.2 samples/s
Step 8570/20005 | Loss: 3.0136 | LR: 3.11e-04 | 5780.3ms/step | 4.2 samples/s
Step 8580/20005 | Loss: 3.0141 | LR: 3.11e-04 | 5780.1ms/step | 4.2 samples/s
Step 8590/20005 | Loss: 3.0207 | LR: 3.11e-04 | 5780.3ms/step | 4.2 samples/s
Step 8600/20005 | Loss: 3.0183 | LR: 3.10e-04 | 5782.4ms/step | 4.2 samples/s
Step 8610/20005 | Loss: 2.9963 | LR: 3.10e-04 | 5782.9ms/step | 4.2 samples/s
Step 8620/20005 | Loss: 3.0183 | LR: 3.09e-04 | 5779.4ms/step | 4.2 samples/s
Step 8630/20005 | Loss: 3.0093 | LR: 3.09e-04 | 5779.1ms/step | 4.2 samples/s
Step 8640/20005 | Loss: 3.0515 | LR: 3.09e-04 | 5777.4ms/step | 4.2 samples/s
Step 8650/20005 | Loss: 3.0105 | LR: 3.08e-04 | 5778.3ms/step | 4.2 samples/s
Step 8660/20005 | Loss: 3.0137 | LR: 3.08e-04 | 5778.8ms/step | 4.2 samples/s
Step 8670/20005 | Loss: 3.0047 | LR: 3.07e-04 | 5779.1ms/step | 4.2 samples/s
Step 8680/20005 | Loss: 3.0250 | LR: 3.07e-04 | 5778.4ms/step | 4.2 samples/s
Step 8690/20005 | Loss: 3.0202 | LR: 3.07e-04 | 5781.8ms/step | 4.2 samples/s
Step 8700/20005 | Loss: 3.0295 | LR: 3.06e-04 | 5780.6ms/step | 4.2 samples/s
Step 8710/20005 | Loss: 3.0103 | LR: 3.06e-04 | 5777.7ms/step | 4.2 samples/s
Step 8720/20005 | Loss: 3.0154 | LR: 3.05e-04 | 5780.0ms/step | 4.2 samples/s
Step 8730/20005 | Loss: 3.0255 | LR: 3.05e-04 | 5780.4ms/step | 4.2 samples/s
Step 8740/20005 | Loss: 2.9956 | LR: 3.05e-04 | 5778.9ms/step | 4.2 samples/s
Step 8750/20005 | Loss: 3.0319 | LR: 3.04e-04 | 5778.6ms/step | 4.2 samples/s
Validation at step 8750: Loss = 3.0176, Perplexity = 20.44
Step 8760/20005 | Loss: 3.0206 | LR: 3.04e-04 | 40656.1ms/step | 0.6 samples/s
Step 8770/20005 | Loss: 3.0121 | LR: 3.04e-04 | 5780.6ms/step | 4.2 samples/s
Step 8780/20005 | Loss: 3.0122 | LR: 3.03e-04 | 5779.4ms/step | 4.2 samples/s
Step 8790/20005 | Loss: 3.0157 | LR: 3.03e-04 | 5777.4ms/step | 4.2 samples/s
Step 8800/20005 | Loss: 3.0371 | LR: 3.02e-04 | 5775.3ms/step | 4.2 samples/s
Step 8810/20005 | Loss: 3.0055 | LR: 3.02e-04 | 5780.1ms/step | 4.2 samples/s
Step 8820/20005 | Loss: 3.0058 | LR: 3.02e-04 | 5780.1ms/step | 4.2 samples/s
Step 8830/20005 | Loss: 3.0113 | LR: 3.01e-04 | 5779.6ms/step | 4.2 samples/s
Step 8840/20005 | Loss: 3.0249 | LR: 3.01e-04 | 5777.4ms/step | 4.2 samples/s
Step 8850/20005 | Loss: 3.0156 | LR: 3.00e-04 | 5779.1ms/step | 4.2 samples/s
Step 8860/20005 | Loss: 2.9762 | LR: 3.00e-04 | 5781.8ms/step | 4.2 samples/s
Step 8870/20005 | Loss: 2.9895 | LR: 3.00e-04 | 5778.0ms/step | 4.2 samples/s
Step 8880/20005 | Loss: 3.0095 | LR: 2.99e-04 | 5779.5ms/step | 4.2 samples/s
Step 8890/20005 | Loss: 3.0154 | LR: 2.99e-04 | 5776.7ms/step | 4.2 samples/s
Step 8900/20005 | Loss: 3.0280 | LR: 2.98e-04 | 5781.6ms/step | 4.2 samples/s
Step 8910/20005 | Loss: 3.0105 | LR: 2.98e-04 | 5780.6ms/step | 4.2 samples/s
Step 8920/20005 | Loss: 3.0101 | LR: 2.98e-04 | 5780.8ms/step | 4.2 samples/s
Step 8930/20005 | Loss: 3.0535 | LR: 2.97e-04 | 5779.1ms/step | 4.2 samples/s
Step 8940/20005 | Loss: 3.0295 | LR: 2.97e-04 | 5779.9ms/step | 4.2 samples/s
Step 8950/20005 | Loss: 3.0607 | LR: 2.97e-04 | 5782.8ms/step | 4.2 samples/s
Step 8960/20005 | Loss: 3.0157 | LR: 2.96e-04 | 5782.3ms/step | 4.2 samples/s
Step 8970/20005 | Loss: 2.9856 | LR: 2.96e-04 | 5779.2ms/step | 4.2 samples/s
Step 8980/20005 | Loss: 2.9938 | LR: 2.95e-04 | 5819.6ms/step | 4.1 samples/s
Step 8990/20005 | Loss: 3.0387 | LR: 2.95e-04 | 5782.9ms/step | 4.2 samples/s
Step 9000/20005 | Loss: 3.0025 | LR: 2.95e-04 | 5781.3ms/step | 4.2 samples/s
Validation at step 9000: Loss = 3.0085, Perplexity = 20.26
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-9000
Step 9010/20005 | Loss: 2.9964 | LR: 2.94e-04 | 41138.1ms/step | 0.6 samples/s
Step 9020/20005 | Loss: 3.0153 | LR: 2.94e-04 | 5780.3ms/step | 4.2 samples/s
Step 9030/20005 | Loss: 3.0079 | LR: 2.93e-04 | 5780.4ms/step | 4.2 samples/s
Step 9040/20005 | Loss: 3.0002 | LR: 2.93e-04 | 5777.6ms/step | 4.2 samples/s
Step 9050/20005 | Loss: 3.0206 | LR: 2.93e-04 | 5776.4ms/step | 4.2 samples/s
Step 9060/20005 | Loss: 2.9844 | LR: 2.92e-04 | 5780.2ms/step | 4.2 samples/s
Step 9070/20005 | Loss: 3.0184 | LR: 2.92e-04 | 5778.4ms/step | 4.2 samples/s
Step 9080/20005 | Loss: 3.0096 | LR: 2.91e-04 | 5783.5ms/step | 4.1 samples/s
Step 9090/20005 | Loss: 2.9953 | LR: 2.91e-04 | 5780.1ms/step | 4.2 samples/s
Step 9100/20005 | Loss: 3.0023 | LR: 2.91e-04 | 5780.1ms/step | 4.2 samples/s
Step 9110/20005 | Loss: 3.0126 | LR: 2.90e-04 | 5783.4ms/step | 4.1 samples/s
Step 9120/20005 | Loss: 3.0084 | LR: 2.90e-04 | 5780.7ms/step | 4.2 samples/s
Step 9130/20005 | Loss: 2.9953 | LR: 2.89e-04 | 5779.9ms/step | 4.2 samples/s
Step 9140/20005 | Loss: 3.0319 | LR: 2.89e-04 | 5780.1ms/step | 4.2 samples/s
Step 9150/20005 | Loss: 3.0388 | LR: 2.89e-04 | 5782.0ms/step | 4.2 samples/s
Step 9160/20005 | Loss: 3.0041 | LR: 2.88e-04 | 5780.0ms/step | 4.2 samples/s
Step 9170/20005 | Loss: 3.0003 | LR: 2.88e-04 | 5780.1ms/step | 4.2 samples/s
Step 9180/20005 | Loss: 3.0203 | LR: 2.88e-04 | 5778.1ms/step | 4.2 samples/s
Step 9190/20005 | Loss: 2.9883 | LR: 2.87e-04 | 5777.9ms/step | 4.2 samples/s
Step 9200/20005 | Loss: 3.0248 | LR: 2.87e-04 | 5780.3ms/step | 4.2 samples/s
Step 9210/20005 | Loss: 2.9946 | LR: 2.86e-04 | 5779.8ms/step | 4.2 samples/s
Step 9220/20005 | Loss: 3.0316 | LR: 2.86e-04 | 5782.5ms/step | 4.2 samples/s
Step 9230/20005 | Loss: 3.0349 | LR: 2.86e-04 | 5779.5ms/step | 4.2 samples/s
Step 9240/20005 | Loss: 2.9931 | LR: 2.85e-04 | 5780.2ms/step | 4.2 samples/s
Step 9250/20005 | Loss: 2.9847 | LR: 2.85e-04 | 5777.7ms/step | 4.2 samples/s
Validation at step 9250: Loss = 3.0029, Perplexity = 20.14
Step 9260/20005 | Loss: 2.9832 | LR: 2.84e-04 | 40654.3ms/step | 0.6 samples/s
Step 9270/20005 | Loss: 2.9931 | LR: 2.84e-04 | 5778.6ms/step | 4.2 samples/s
Step 9280/20005 | Loss: 2.9971 | LR: 2.84e-04 | 5779.9ms/step | 4.2 samples/s
Step 9290/20005 | Loss: 2.9903 | LR: 2.83e-04 | 5779.9ms/step | 4.2 samples/s
Step 9300/20005 | Loss: 3.0105 | LR: 2.83e-04 | 5780.6ms/step | 4.2 samples/s
Step 9310/20005 | Loss: 2.9776 | LR: 2.82e-04 | 5782.0ms/step | 4.2 samples/s
Step 9320/20005 | Loss: 3.0034 | LR: 2.82e-04 | 5780.8ms/step | 4.2 samples/s
Step 9330/20005 | Loss: 3.0055 | LR: 2.82e-04 | 5779.7ms/step | 4.2 samples/s
Step 9340/20005 | Loss: 3.0234 | LR: 2.81e-04 | 5779.2ms/step | 4.2 samples/s
Step 9350/20005 | Loss: 2.9861 | LR: 2.81e-04 | 5782.2ms/step | 4.2 samples/s
Step 9360/20005 | Loss: 3.0330 | LR: 2.80e-04 | 5781.3ms/step | 4.2 samples/s
Step 9370/20005 | Loss: 3.0139 | LR: 2.80e-04 | 5778.1ms/step | 4.2 samples/s
Step 9380/20005 | Loss: 3.0093 | LR: 2.80e-04 | 5775.9ms/step | 4.2 samples/s
Step 9390/20005 | Loss: 3.0080 | LR: 2.79e-04 | 5780.6ms/step | 4.2 samples/s
Step 9400/20005 | Loss: 2.9798 | LR: 2.79e-04 | 5779.4ms/step | 4.2 samples/s
Step 9410/20005 | Loss: 2.9808 | LR: 2.78e-04 | 5782.2ms/step | 4.2 samples/s
Step 9420/20005 | Loss: 2.9936 | LR: 2.78e-04 | 5781.6ms/step | 4.2 samples/s
Step 9430/20005 | Loss: 2.9693 | LR: 2.78e-04 | 5780.1ms/step | 4.2 samples/s
Step 9440/20005 | Loss: 3.0192 | LR: 2.77e-04 | 5780.0ms/step | 4.2 samples/s
Step 9450/20005 | Loss: 3.0091 | LR: 2.77e-04 | 5778.8ms/step | 4.2 samples/s
Step 9460/20005 | Loss: 2.9738 | LR: 2.76e-04 | 5821.8ms/step | 4.1 samples/s
Step 9470/20005 | Loss: 2.9958 | LR: 2.76e-04 | 5778.0ms/step | 4.2 samples/s
Step 9480/20005 | Loss: 3.0207 | LR: 2.76e-04 | 5780.4ms/step | 4.2 samples/s
Step 9490/20005 | Loss: 2.9520 | LR: 2.75e-04 | 5780.6ms/step | 4.2 samples/s
Step 9500/20005 | Loss: 2.9954 | LR: 2.75e-04 | 5781.0ms/step | 4.2 samples/s
Validation at step 9500: Loss = 2.9950, Perplexity = 19.98
Step 9510/20005 | Loss: 2.9835 | LR: 2.75e-04 | 40664.3ms/step | 0.6 samples/s
Step 9520/20005 | Loss: 3.0070 | LR: 2.74e-04 | 5780.3ms/step | 4.2 samples/s
Step 9530/20005 | Loss: 2.9856 | LR: 2.74e-04 | 5780.3ms/step | 4.2 samples/s
Step 9540/20005 | Loss: 3.0026 | LR: 2.73e-04 | 5781.3ms/step | 4.2 samples/s
Step 9550/20005 | Loss: 2.9954 | LR: 2.73e-04 | 5778.7ms/step | 4.2 samples/s
Step 9560/20005 | Loss: 3.0050 | LR: 2.73e-04 | 5777.9ms/step | 4.2 samples/s
Step 9570/20005 | Loss: 3.0044 | LR: 2.72e-04 | 5779.9ms/step | 4.2 samples/s
Step 9580/20005 | Loss: 3.0274 | LR: 2.72e-04 | 5781.6ms/step | 4.2 samples/s
Step 9590/20005 | Loss: 2.9772 | LR: 2.71e-04 | 5780.9ms/step | 4.2 samples/s
Step 9600/20005 | Loss: 3.0001 | LR: 2.71e-04 | 5782.9ms/step | 4.2 samples/s
Step 9610/20005 | Loss: 2.9861 | LR: 2.71e-04 | 5784.0ms/step | 4.1 samples/s
Step 9620/20005 | Loss: 2.9611 | LR: 2.70e-04 | 5791.3ms/step | 4.1 samples/s
Step 9630/20005 | Loss: 3.0120 | LR: 2.70e-04 | 5793.2ms/step | 4.1 samples/s
Step 9640/20005 | Loss: 2.9898 | LR: 2.69e-04 | 5784.7ms/step | 4.1 samples/s
Step 9650/20005 | Loss: 2.9670 | LR: 2.69e-04 | 5786.4ms/step | 4.1 samples/s
Step 9660/20005 | Loss: 2.9969 | LR: 2.69e-04 | 5784.1ms/step | 4.1 samples/s
Step 9670/20005 | Loss: 2.9956 | LR: 2.68e-04 | 5782.1ms/step | 4.2 samples/s
Step 9680/20005 | Loss: 2.9772 | LR: 2.68e-04 | 5784.2ms/step | 4.1 samples/s
Step 9690/20005 | Loss: 2.9970 | LR: 2.67e-04 | 5784.9ms/step | 4.1 samples/s
Step 9700/20005 | Loss: 2.9802 | LR: 2.67e-04 | 5782.4ms/step | 4.2 samples/s
Step 9710/20005 | Loss: 3.0047 | LR: 2.67e-04 | 5778.1ms/step | 4.2 samples/s
Step 9720/20005 | Loss: 3.0157 | LR: 2.66e-04 | 5779.5ms/step | 4.2 samples/s
Step 9730/20005 | Loss: 2.9788 | LR: 2.66e-04 | 5782.6ms/step | 4.2 samples/s
Step 9740/20005 | Loss: 2.9881 | LR: 2.65e-04 | 5781.3ms/step | 4.2 samples/s
Step 9750/20005 | Loss: 2.9851 | LR: 2.65e-04 | 5784.8ms/step | 4.1 samples/s
Validation at step 9750: Loss = 2.9883, Perplexity = 19.85
Step 9760/20005 | Loss: 2.9951 | LR: 2.65e-04 | 40669.3ms/step | 0.6 samples/s
Step 9770/20005 | Loss: 2.9724 | LR: 2.64e-04 | 5784.5ms/step | 4.1 samples/s
Step 9780/20005 | Loss: 3.0146 | LR: 2.64e-04 | 5780.5ms/step | 4.2 samples/s
Step 9790/20005 | Loss: 3.0214 | LR: 2.63e-04 | 5782.2ms/step | 4.2 samples/s
Step 9800/20005 | Loss: 2.9718 | LR: 2.63e-04 | 5782.0ms/step | 4.2 samples/s
Step 9810/20005 | Loss: 2.9935 | LR: 2.63e-04 | 5781.2ms/step | 4.2 samples/s
Step 9820/20005 | Loss: 3.0146 | LR: 2.62e-04 | 5780.4ms/step | 4.2 samples/s
Step 9830/20005 | Loss: 3.0044 | LR: 2.62e-04 | 5782.5ms/step | 4.2 samples/s
Step 9840/20005 | Loss: 2.9988 | LR: 2.61e-04 | 5780.0ms/step | 4.2 samples/s
Step 9850/20005 | Loss: 3.0033 | LR: 2.61e-04 | 5782.7ms/step | 4.2 samples/s
Step 9860/20005 | Loss: 2.9726 | LR: 2.61e-04 | 5777.2ms/step | 4.2 samples/s
Step 9870/20005 | Loss: 2.9748 | LR: 2.60e-04 | 5781.3ms/step | 4.2 samples/s
Step 9880/20005 | Loss: 2.9873 | LR: 2.60e-04 | 5785.9ms/step | 4.1 samples/s
Step 9890/20005 | Loss: 2.9918 | LR: 2.59e-04 | 5779.1ms/step | 4.2 samples/s
Step 9900/20005 | Loss: 2.9857 | LR: 2.59e-04 | 5779.9ms/step | 4.2 samples/s
Step 9910/20005 | Loss: 2.9955 | LR: 2.59e-04 | 5780.0ms/step | 4.2 samples/s
Step 9920/20005 | Loss: 2.9759 | LR: 2.58e-04 | 5780.6ms/step | 4.2 samples/s
Step 9930/20005 | Loss: 2.9954 | LR: 2.58e-04 | 5779.4ms/step | 4.2 samples/s
Step 9940/20005 | Loss: 2.9914 | LR: 2.57e-04 | 5813.8ms/step | 4.1 samples/s
Step 9950/20005 | Loss: 2.9772 | LR: 2.57e-04 | 5778.0ms/step | 4.2 samples/s
Step 9960/20005 | Loss: 3.0070 | LR: 2.57e-04 | 5781.4ms/step | 4.2 samples/s
Step 9970/20005 | Loss: 2.9503 | LR: 2.56e-04 | 5778.3ms/step | 4.2 samples/s
Step 9980/20005 | Loss: 3.0046 | LR: 2.56e-04 | 5778.5ms/step | 4.2 samples/s
Step 9990/20005 | Loss: 2.9967 | LR: 2.55e-04 | 5780.1ms/step | 4.2 samples/s
Step 10000/20005 | Loss: 2.9898 | LR: 2.55e-04 | 5780.1ms/step | 4.2 samples/s
Validation at step 10000: Loss = 2.9809, Perplexity = 19.70
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-10000
Step 10010/20005 | Loss: 2.9667 | LR: 2.55e-04 | 41139.5ms/step | 0.6 samples/s
Step 10020/20005 | Loss: 3.0049 | LR: 2.54e-04 | 5781.9ms/step | 4.2 samples/s
Step 10030/20005 | Loss: 2.9882 | LR: 2.54e-04 | 5782.9ms/step | 4.2 samples/s
Step 10040/20005 | Loss: 2.9734 | LR: 2.53e-04 | 5778.1ms/step | 4.2 samples/s
Step 10050/20005 | Loss: 2.9923 | LR: 2.53e-04 | 5782.4ms/step | 4.2 samples/s
Step 10060/20005 | Loss: 2.9549 | LR: 2.53e-04 | 5781.9ms/step | 4.2 samples/s
Step 10070/20005 | Loss: 2.9529 | LR: 2.52e-04 | 5781.6ms/step | 4.2 samples/s
Step 10080/20005 | Loss: 2.9864 | LR: 2.52e-04 | 5783.2ms/step | 4.1 samples/s
Step 10090/20005 | Loss: 2.9730 | LR: 2.51e-04 | 5780.8ms/step | 4.2 samples/s
Step 10100/20005 | Loss: 2.9745 | LR: 2.51e-04 | 5779.9ms/step | 4.2 samples/s
Step 10110/20005 | Loss: 2.9618 | LR: 2.51e-04 | 5778.2ms/step | 4.2 samples/s
Step 10120/20005 | Loss: 2.9744 | LR: 2.50e-04 | 5781.8ms/step | 4.2 samples/s
Step 10130/20005 | Loss: 2.9555 | LR: 2.50e-04 | 5784.0ms/step | 4.1 samples/s
Step 10140/20005 | Loss: 2.9616 | LR: 2.50e-04 | 5780.6ms/step | 4.2 samples/s
Step 10150/20005 | Loss: 2.9964 | LR: 2.49e-04 | 5779.8ms/step | 4.2 samples/s
Step 10160/20005 | Loss: 2.9816 | LR: 2.49e-04 | 5781.4ms/step | 4.2 samples/s
Step 10170/20005 | Loss: 2.9972 | LR: 2.48e-04 | 5783.1ms/step | 4.2 samples/s
Step 10180/20005 | Loss: 2.9763 | LR: 2.48e-04 | 5781.5ms/step | 4.2 samples/s
Step 10190/20005 | Loss: 2.9759 | LR: 2.48e-04 | 5779.3ms/step | 4.2 samples/s
Step 10200/20005 | Loss: 2.9826 | LR: 2.47e-04 | 5781.6ms/step | 4.2 samples/s
Step 10210/20005 | Loss: 2.9725 | LR: 2.47e-04 | 5780.1ms/step | 4.2 samples/s
Step 10220/20005 | Loss: 2.9693 | LR: 2.46e-04 | 5781.4ms/step | 4.2 samples/s
Step 10230/20005 | Loss: 2.9439 | LR: 2.46e-04 | 5779.7ms/step | 4.2 samples/s
Step 10240/20005 | Loss: 2.9546 | LR: 2.46e-04 | 5778.1ms/step | 4.2 samples/s
Step 10250/20005 | Loss: 3.0106 | LR: 2.45e-04 | 5778.7ms/step | 4.2 samples/s
Validation at step 10250: Loss = 2.9741, Perplexity = 19.57
Step 10260/20005 | Loss: 2.9486 | LR: 2.45e-04 | 40671.1ms/step | 0.6 samples/s
Step 10270/20005 | Loss: 2.9657 | LR: 2.44e-04 | 5782.7ms/step | 4.2 samples/s
Step 10280/20005 | Loss: 2.9654 | LR: 2.44e-04 | 5781.1ms/step | 4.2 samples/s
Step 10290/20005 | Loss: 2.9794 | LR: 2.44e-04 | 5786.0ms/step | 4.1 samples/s
Step 10300/20005 | Loss: 2.9691 | LR: 2.43e-04 | 5783.0ms/step | 4.2 samples/s
Step 10310/20005 | Loss: 2.9848 | LR: 2.43e-04 | 5783.5ms/step | 4.1 samples/s
Step 10320/20005 | Loss: 2.9728 | LR: 2.42e-04 | 5780.4ms/step | 4.2 samples/s
Step 10330/20005 | Loss: 2.9789 | LR: 2.42e-04 | 5781.8ms/step | 4.2 samples/s
Step 10340/20005 | Loss: 2.9622 | LR: 2.42e-04 | 5780.7ms/step | 4.2 samples/s
Step 10350/20005 | Loss: 2.9760 | LR: 2.41e-04 | 5779.6ms/step | 4.2 samples/s
Step 10360/20005 | Loss: 2.9529 | LR: 2.41e-04 | 5786.1ms/step | 4.1 samples/s
Step 10370/20005 | Loss: 3.0025 | LR: 2.40e-04 | 5778.1ms/step | 4.2 samples/s
Step 10380/20005 | Loss: 2.9551 | LR: 2.40e-04 | 5779.7ms/step | 4.2 samples/s
Step 10390/20005 | Loss: 2.9680 | LR: 2.40e-04 | 5780.3ms/step | 4.2 samples/s
Step 10400/20005 | Loss: 2.9603 | LR: 2.39e-04 | 5782.9ms/step | 4.2 samples/s
Step 10410/20005 | Loss: 2.9726 | LR: 2.39e-04 | 5822.6ms/step | 4.1 samples/s
Step 10420/20005 | Loss: 2.9752 | LR: 2.38e-04 | 5782.4ms/step | 4.2 samples/s
Step 10430/20005 | Loss: 2.9743 | LR: 2.38e-04 | 5786.3ms/step | 4.1 samples/s
Step 10440/20005 | Loss: 2.9904 | LR: 2.38e-04 | 5781.7ms/step | 4.2 samples/s
Step 10450/20005 | Loss: 2.9661 | LR: 2.37e-04 | 5783.5ms/step | 4.1 samples/s
Step 10460/20005 | Loss: 2.9415 | LR: 2.37e-04 | 5780.5ms/step | 4.2 samples/s
Step 10470/20005 | Loss: 2.9532 | LR: 2.36e-04 | 5784.7ms/step | 4.1 samples/s
Step 10480/20005 | Loss: 2.9674 | LR: 2.36e-04 | 5783.8ms/step | 4.1 samples/s
Step 10490/20005 | Loss: 2.9723 | LR: 2.36e-04 | 5781.8ms/step | 4.2 samples/s
Step 10500/20005 | Loss: 2.9962 | LR: 2.35e-04 | 5778.7ms/step | 4.2 samples/s
Validation at step 10500: Loss = 2.9682, Perplexity = 19.46
Step 10510/20005 | Loss: 2.9641 | LR: 2.35e-04 | 40681.9ms/step | 0.6 samples/s
Step 10520/20005 | Loss: 2.9662 | LR: 2.34e-04 | 5780.0ms/step | 4.2 samples/s
Step 10530/20005 | Loss: 2.9797 | LR: 2.34e-04 | 5782.6ms/step | 4.2 samples/s
Step 10540/20005 | Loss: 2.9907 | LR: 2.34e-04 | 5783.4ms/step | 4.1 samples/s
Step 10550/20005 | Loss: 2.9460 | LR: 2.33e-04 | 5782.5ms/step | 4.2 samples/s
Step 10560/20005 | Loss: 2.9715 | LR: 2.33e-04 | 5781.4ms/step | 4.2 samples/s
Step 10570/20005 | Loss: 2.9486 | LR: 2.32e-04 | 5782.3ms/step | 4.2 samples/s
Step 10580/20005 | Loss: 2.9752 | LR: 2.32e-04 | 5779.9ms/step | 4.2 samples/s
Step 10590/20005 | Loss: 2.9732 | LR: 2.32e-04 | 5781.2ms/step | 4.2 samples/s
Step 10600/20005 | Loss: 2.9590 | LR: 2.31e-04 | 5781.2ms/step | 4.2 samples/s
Step 10610/20005 | Loss: 2.9832 | LR: 2.31e-04 | 5781.3ms/step | 4.2 samples/s
Step 10620/20005 | Loss: 2.9384 | LR: 2.30e-04 | 5782.7ms/step | 4.2 samples/s
Step 10630/20005 | Loss: 2.9510 | LR: 2.30e-04 | 5779.9ms/step | 4.2 samples/s
Step 10640/20005 | Loss: 2.9825 | LR: 2.30e-04 | 5782.2ms/step | 4.2 samples/s
Step 10650/20005 | Loss: 2.9599 | LR: 2.29e-04 | 5779.5ms/step | 4.2 samples/s
Step 10660/20005 | Loss: 2.9762 | LR: 2.29e-04 | 5780.4ms/step | 4.2 samples/s
Step 10670/20005 | Loss: 2.9486 | LR: 2.28e-04 | 5780.4ms/step | 4.2 samples/s
Step 10680/20005 | Loss: 2.9780 | LR: 2.28e-04 | 5781.5ms/step | 4.2 samples/s
Step 10690/20005 | Loss: 2.9399 | LR: 2.28e-04 | 5783.3ms/step | 4.1 samples/s
Step 10700/20005 | Loss: 2.9805 | LR: 2.27e-04 | 5779.0ms/step | 4.2 samples/s
Step 10710/20005 | Loss: 2.9810 | LR: 2.27e-04 | 5784.3ms/step | 4.1 samples/s
Step 10720/20005 | Loss: 2.9597 | LR: 2.26e-04 | 5782.8ms/step | 4.2 samples/s
Step 10730/20005 | Loss: 2.9662 | LR: 2.26e-04 | 5782.1ms/step | 4.2 samples/s
Step 10740/20005 | Loss: 2.9489 | LR: 2.26e-04 | 5779.2ms/step | 4.2 samples/s
Step 10750/20005 | Loss: 2.9605 | LR: 2.25e-04 | 5778.1ms/step | 4.2 samples/s
Validation at step 10750: Loss = 2.9617, Perplexity = 19.33
Step 10760/20005 | Loss: 2.9336 | LR: 2.25e-04 | 40697.8ms/step | 0.6 samples/s
Step 10770/20005 | Loss: 2.9846 | LR: 2.25e-04 | 5783.7ms/step | 4.1 samples/s
Step 10780/20005 | Loss: 2.9431 | LR: 2.24e-04 | 5780.6ms/step | 4.2 samples/s
Step 10790/20005 | Loss: 2.9566 | LR: 2.24e-04 | 5781.7ms/step | 4.2 samples/s
Step 10800/20005 | Loss: 2.9692 | LR: 2.23e-04 | 5783.0ms/step | 4.2 samples/s
Step 10810/20005 | Loss: 2.9405 | LR: 2.23e-04 | 5779.9ms/step | 4.2 samples/s
Step 10820/20005 | Loss: 2.9804 | LR: 2.23e-04 | 5781.7ms/step | 4.2 samples/s
Step 10830/20005 | Loss: 2.9583 | LR: 2.22e-04 | 5779.9ms/step | 4.2 samples/s
Step 10840/20005 | Loss: 2.9329 | LR: 2.22e-04 | 5781.4ms/step | 4.2 samples/s
Step 10850/20005 | Loss: 3.0068 | LR: 2.21e-04 | 5783.1ms/step | 4.1 samples/s
Step 10860/20005 | Loss: 2.9568 | LR: 2.21e-04 | 5779.5ms/step | 4.2 samples/s
Step 10870/20005 | Loss: 2.9708 | LR: 2.21e-04 | 5780.1ms/step | 4.2 samples/s
Step 10880/20005 | Loss: 2.9373 | LR: 2.20e-04 | 5781.1ms/step | 4.2 samples/s
Step 10890/20005 | Loss: 2.9561 | LR: 2.20e-04 | 5820.8ms/step | 4.1 samples/s
Step 10900/20005 | Loss: 2.9689 | LR: 2.19e-04 | 5779.6ms/step | 4.2 samples/s
Step 10910/20005 | Loss: 2.9466 | LR: 2.19e-04 | 5780.3ms/step | 4.2 samples/s
Step 10920/20005 | Loss: 2.9352 | LR: 2.19e-04 | 5780.7ms/step | 4.2 samples/s
Step 10930/20005 | Loss: 2.9371 | LR: 2.18e-04 | 5780.5ms/step | 4.2 samples/s
Step 10940/20005 | Loss: 2.9785 | LR: 2.18e-04 | 5777.3ms/step | 4.2 samples/s
Step 10950/20005 | Loss: 2.9388 | LR: 2.17e-04 | 5781.5ms/step | 4.2 samples/s
Step 10960/20005 | Loss: 2.9523 | LR: 2.17e-04 | 5778.5ms/step | 4.2 samples/s
Step 10970/20005 | Loss: 2.9389 | LR: 2.17e-04 | 5779.8ms/step | 4.2 samples/s
Step 10980/20005 | Loss: 2.9745 | LR: 2.16e-04 | 5779.9ms/step | 4.2 samples/s
Step 10990/20005 | Loss: 2.9486 | LR: 2.16e-04 | 5780.4ms/step | 4.2 samples/s
Step 11000/20005 | Loss: 2.9551 | LR: 2.15e-04 | 5780.2ms/step | 4.2 samples/s
Validation at step 11000: Loss = 2.9559, Perplexity = 19.22
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-11000
Step 11010/20005 | Loss: 2.9627 | LR: 2.15e-04 | 41134.0ms/step | 0.6 samples/s
Step 11020/20005 | Loss: 2.9584 | LR: 2.15e-04 | 5778.4ms/step | 4.2 samples/s
Step 11030/20005 | Loss: 2.9712 | LR: 2.14e-04 | 5779.1ms/step | 4.2 samples/s
Step 11040/20005 | Loss: 2.9214 | LR: 2.14e-04 | 5782.5ms/step | 4.2 samples/s
Step 11050/20005 | Loss: 2.9654 | LR: 2.13e-04 | 5780.3ms/step | 4.2 samples/s
Step 11060/20005 | Loss: 2.9597 | LR: 2.13e-04 | 5778.0ms/step | 4.2 samples/s
Step 11070/20005 | Loss: 2.9465 | LR: 2.13e-04 | 5779.9ms/step | 4.2 samples/s
Step 11080/20005 | Loss: 2.9812 | LR: 2.12e-04 | 5780.0ms/step | 4.2 samples/s
Step 11090/20005 | Loss: 2.9532 | LR: 2.12e-04 | 5779.4ms/step | 4.2 samples/s
Step 11100/20005 | Loss: 2.9271 | LR: 2.11e-04 | 5780.9ms/step | 4.2 samples/s
Step 11110/20005 | Loss: 2.9414 | LR: 2.11e-04 | 5781.9ms/step | 4.2 samples/s
Step 11120/20005 | Loss: 2.9644 | LR: 2.11e-04 | 5779.3ms/step | 4.2 samples/s
Step 11130/20005 | Loss: 2.9630 | LR: 2.10e-04 | 5775.2ms/step | 4.2 samples/s
Step 11140/20005 | Loss: 2.9331 | LR: 2.10e-04 | 5786.0ms/step | 4.1 samples/s
Step 11150/20005 | Loss: 2.9384 | LR: 2.10e-04 | 5779.4ms/step | 4.2 samples/s
Step 11160/20005 | Loss: 2.9681 | LR: 2.09e-04 | 5780.1ms/step | 4.2 samples/s
Step 11170/20005 | Loss: 2.9580 | LR: 2.09e-04 | 5781.8ms/step | 4.2 samples/s
Step 11180/20005 | Loss: 2.9820 | LR: 2.08e-04 | 5778.7ms/step | 4.2 samples/s
Step 11190/20005 | Loss: 2.9490 | LR: 2.08e-04 | 5779.4ms/step | 4.2 samples/s
Step 11200/20005 | Loss: 2.9642 | LR: 2.08e-04 | 5780.2ms/step | 4.2 samples/s
Step 11210/20005 | Loss: 2.9798 | LR: 2.07e-04 | 5779.8ms/step | 4.2 samples/s
Step 11220/20005 | Loss: 2.9525 | LR: 2.07e-04 | 5783.3ms/step | 4.1 samples/s
Step 11230/20005 | Loss: 2.9491 | LR: 2.06e-04 | 5779.6ms/step | 4.2 samples/s
Step 11240/20005 | Loss: 2.9472 | LR: 2.06e-04 | 5780.7ms/step | 4.2 samples/s
Step 11250/20005 | Loss: 2.9663 | LR: 2.06e-04 | 5778.5ms/step | 4.2 samples/s
Validation at step 11250: Loss = 2.9507, Perplexity = 19.12
Step 11260/20005 | Loss: 2.9661 | LR: 2.05e-04 | 40640.7ms/step | 0.6 samples/s
Step 11270/20005 | Loss: 2.9449 | LR: 2.05e-04 | 5780.4ms/step | 4.2 samples/s
Step 11280/20005 | Loss: 2.9740 | LR: 2.04e-04 | 5780.7ms/step | 4.2 samples/s
Step 11290/20005 | Loss: 2.9683 | LR: 2.04e-04 | 5779.3ms/step | 4.2 samples/s
Step 11300/20005 | Loss: 2.9634 | LR: 2.04e-04 | 5788.0ms/step | 4.1 samples/s
Step 11310/20005 | Loss: 2.9353 | LR: 2.03e-04 | 5788.3ms/step | 4.1 samples/s
Step 11320/20005 | Loss: 2.9416 | LR: 2.03e-04 | 5782.6ms/step | 4.2 samples/s
Step 11330/20005 | Loss: 2.9473 | LR: 2.02e-04 | 5779.8ms/step | 4.2 samples/s
Step 11340/20005 | Loss: 2.9674 | LR: 2.02e-04 | 5781.5ms/step | 4.2 samples/s
Step 11350/20005 | Loss: 2.9597 | LR: 2.02e-04 | 5783.9ms/step | 4.1 samples/s
Step 11360/20005 | Loss: 2.9430 | LR: 2.01e-04 | 5869.3ms/step | 4.1 samples/s
Step 11370/20005 | Loss: 2.9342 | LR: 2.01e-04 | 5796.0ms/step | 4.1 samples/s
Step 11380/20005 | Loss: 2.9460 | LR: 2.01e-04 | 5785.9ms/step | 4.1 samples/s
Step 11390/20005 | Loss: 2.9395 | LR: 2.00e-04 | 5786.9ms/step | 4.1 samples/s
Step 11400/20005 | Loss: 2.9319 | LR: 2.00e-04 | 5781.8ms/step | 4.2 samples/s
Step 11410/20005 | Loss: 2.9535 | LR: 1.99e-04 | 5778.0ms/step | 4.2 samples/s
Step 11420/20005 | Loss: 2.9463 | LR: 1.99e-04 | 5780.1ms/step | 4.2 samples/s
Step 11430/20005 | Loss: 2.9557 | LR: 1.99e-04 | 5785.5ms/step | 4.1 samples/s
Step 11440/20005 | Loss: 2.9526 | LR: 1.98e-04 | 5781.1ms/step | 4.2 samples/s
Step 11450/20005 | Loss: 2.9418 | LR: 1.98e-04 | 5779.9ms/step | 4.2 samples/s
Step 11460/20005 | Loss: 2.9333 | LR: 1.97e-04 | 5777.0ms/step | 4.2 samples/s
Step 11470/20005 | Loss: 2.9382 | LR: 1.97e-04 | 5778.1ms/step | 4.2 samples/s
Step 11480/20005 | Loss: 2.9217 | LR: 1.97e-04 | 5778.7ms/step | 4.2 samples/s
Step 11490/20005 | Loss: 2.9500 | LR: 1.96e-04 | 5781.7ms/step | 4.2 samples/s
Step 11500/20005 | Loss: 2.9488 | LR: 1.96e-04 | 5777.9ms/step | 4.2 samples/s
Validation at step 11500: Loss = 2.9458, Perplexity = 19.03
Step 11510/20005 | Loss: 2.9466 | LR: 1.95e-04 | 40644.8ms/step | 0.6 samples/s
Step 11520/20005 | Loss: 2.9586 | LR: 1.95e-04 | 5775.5ms/step | 4.2 samples/s
Step 11530/20005 | Loss: 2.9482 | LR: 1.95e-04 | 5776.8ms/step | 4.2 samples/s
Step 11540/20005 | Loss: 2.9676 | LR: 1.94e-04 | 5779.9ms/step | 4.2 samples/s
Step 11550/20005 | Loss: 2.9792 | LR: 1.94e-04 | 5779.8ms/step | 4.2 samples/s
Step 11560/20005 | Loss: 2.9419 | LR: 1.94e-04 | 5781.9ms/step | 4.2 samples/s
Step 11570/20005 | Loss: 2.9355 | LR: 1.93e-04 | 5783.4ms/step | 4.1 samples/s
Step 11580/20005 | Loss: 2.9358 | LR: 1.93e-04 | 5778.3ms/step | 4.2 samples/s
Step 11590/20005 | Loss: 2.9190 | LR: 1.92e-04 | 5779.1ms/step | 4.2 samples/s
Step 11600/20005 | Loss: 2.9688 | LR: 1.92e-04 | 5781.3ms/step | 4.2 samples/s
Step 11610/20005 | Loss: 2.9253 | LR: 1.92e-04 | 5779.7ms/step | 4.2 samples/s
Step 11620/20005 | Loss: 2.9580 | LR: 1.91e-04 | 5782.8ms/step | 4.2 samples/s
Step 11630/20005 | Loss: 2.9486 | LR: 1.91e-04 | 5781.1ms/step | 4.2 samples/s
Step 11640/20005 | Loss: 2.9636 | LR: 1.90e-04 | 5778.7ms/step | 4.2 samples/s
Step 11650/20005 | Loss: 2.9578 | LR: 1.90e-04 | 5780.2ms/step | 4.2 samples/s
Step 11660/20005 | Loss: 2.9657 | LR: 1.90e-04 | 5781.8ms/step | 4.2 samples/s
Step 11670/20005 | Loss: 2.9451 | LR: 1.89e-04 | 5786.1ms/step | 4.1 samples/s
Step 11680/20005 | Loss: 2.9421 | LR: 1.89e-04 | 5781.7ms/step | 4.2 samples/s
Step 11690/20005 | Loss: 2.9717 | LR: 1.89e-04 | 5785.6ms/step | 4.1 samples/s
Step 11700/20005 | Loss: 2.9297 | LR: 1.88e-04 | 5780.3ms/step | 4.2 samples/s
Step 11710/20005 | Loss: 2.9080 | LR: 1.88e-04 | 5779.8ms/step | 4.2 samples/s
Step 11720/20005 | Loss: 2.9552 | LR: 1.87e-04 | 5779.2ms/step | 4.2 samples/s
Step 11730/20005 | Loss: 2.9429 | LR: 1.87e-04 | 5779.3ms/step | 4.2 samples/s
Step 11740/20005 | Loss: 2.9545 | LR: 1.87e-04 | 5782.2ms/step | 4.2 samples/s
Step 11750/20005 | Loss: 2.9329 | LR: 1.86e-04 | 5782.4ms/step | 4.2 samples/s
Validation at step 11750: Loss = 2.9410, Perplexity = 18.93
Step 11760/20005 | Loss: 2.9338 | LR: 1.86e-04 | 40657.9ms/step | 0.6 samples/s
Step 11770/20005 | Loss: 2.9351 | LR: 1.85e-04 | 5782.3ms/step | 4.2 samples/s
Step 11780/20005 | Loss: 2.9493 | LR: 1.85e-04 | 5782.7ms/step | 4.2 samples/s
Step 11790/20005 | Loss: 2.9492 | LR: 1.85e-04 | 5788.0ms/step | 4.1 samples/s
Step 11800/20005 | Loss: 2.9360 | LR: 1.84e-04 | 5784.8ms/step | 4.1 samples/s
Step 11810/20005 | Loss: 2.9237 | LR: 1.84e-04 | 5782.0ms/step | 4.2 samples/s
Step 11820/20005 | Loss: 2.9440 | LR: 1.84e-04 | 5780.2ms/step | 4.2 samples/s
Step 11830/20005 | Loss: 2.9792 | LR: 1.83e-04 | 5794.2ms/step | 4.1 samples/s
Step 11840/20005 | Loss: 2.9408 | LR: 1.83e-04 | 5861.3ms/step | 4.1 samples/s
Step 11850/20005 | Loss: 2.9518 | LR: 1.82e-04 | 5787.0ms/step | 4.1 samples/s
Step 11860/20005 | Loss: 2.9388 | LR: 1.82e-04 | 5784.9ms/step | 4.1 samples/s
Step 11870/20005 | Loss: 2.9251 | LR: 1.82e-04 | 5785.7ms/step | 4.1 samples/s
Step 11880/20005 | Loss: 2.9448 | LR: 1.81e-04 | 5786.8ms/step | 4.1 samples/s
Step 11890/20005 | Loss: 2.9259 | LR: 1.81e-04 | 5784.3ms/step | 4.1 samples/s
Step 11900/20005 | Loss: 2.9451 | LR: 1.80e-04 | 5783.9ms/step | 4.1 samples/s
Step 11910/20005 | Loss: 2.9536 | LR: 1.80e-04 | 5787.2ms/step | 4.1 samples/s
Step 11920/20005 | Loss: 2.9759 | LR: 1.80e-04 | 5784.1ms/step | 4.1 samples/s
Step 11930/20005 | Loss: 2.9509 | LR: 1.79e-04 | 5783.5ms/step | 4.1 samples/s
Step 11940/20005 | Loss: 2.9577 | LR: 1.79e-04 | 5782.3ms/step | 4.2 samples/s
Step 11950/20005 | Loss: 2.9577 | LR: 1.79e-04 | 5783.7ms/step | 4.1 samples/s
Step 11960/20005 | Loss: 2.9426 | LR: 1.78e-04 | 5783.3ms/step | 4.1 samples/s
Step 11970/20005 | Loss: 2.9181 | LR: 1.78e-04 | 5780.9ms/step | 4.2 samples/s
Step 11980/20005 | Loss: 2.9622 | LR: 1.77e-04 | 5788.7ms/step | 4.1 samples/s
Step 11990/20005 | Loss: 2.9133 | LR: 1.77e-04 | 5784.3ms/step | 4.1 samples/s
Step 12000/20005 | Loss: 2.9010 | LR: 1.77e-04 | 5783.2ms/step | 4.1 samples/s
Validation at step 12000: Loss = 2.9367, Perplexity = 18.85
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-12000
Step 12010/20005 | Loss: 2.9518 | LR: 1.76e-04 | 41249.4ms/step | 0.6 samples/s
Step 12020/20005 | Loss: 2.9578 | LR: 1.76e-04 | 5783.7ms/step | 4.1 samples/s
Step 12030/20005 | Loss: 2.9346 | LR: 1.76e-04 | 5781.6ms/step | 4.2 samples/s
Step 12040/20005 | Loss: 2.9408 | LR: 1.75e-04 | 5779.7ms/step | 4.2 samples/s
Step 12050/20005 | Loss: 2.9487 | LR: 1.75e-04 | 5782.0ms/step | 4.2 samples/s
Step 12060/20005 | Loss: 2.9437 | LR: 1.74e-04 | 5784.1ms/step | 4.1 samples/s
Step 12070/20005 | Loss: 2.9250 | LR: 1.74e-04 | 5781.9ms/step | 4.2 samples/s
Step 12080/20005 | Loss: 2.9417 | LR: 1.74e-04 | 5779.7ms/step | 4.2 samples/s
Step 12090/20005 | Loss: 2.9268 | LR: 1.73e-04 | 5782.2ms/step | 4.2 samples/s
Step 12100/20005 | Loss: 2.9379 | LR: 1.73e-04 | 5780.2ms/step | 4.2 samples/s
Step 12110/20005 | Loss: 2.8973 | LR: 1.72e-04 | 5783.6ms/step | 4.1 samples/s
Step 12120/20005 | Loss: 2.9095 | LR: 1.72e-04 | 5783.0ms/step | 4.2 samples/s
Step 12130/20005 | Loss: 2.9135 | LR: 1.72e-04 | 5779.2ms/step | 4.2 samples/s
Step 12140/20005 | Loss: 2.9412 | LR: 1.71e-04 | 5781.3ms/step | 4.2 samples/s
Step 12150/20005 | Loss: 2.9206 | LR: 1.71e-04 | 5779.9ms/step | 4.2 samples/s
Step 12160/20005 | Loss: 2.9363 | LR: 1.71e-04 | 5783.3ms/step | 4.1 samples/s
Step 12170/20005 | Loss: 2.9302 | LR: 1.70e-04 | 5780.2ms/step | 4.2 samples/s
Step 12180/20005 | Loss: 2.9308 | LR: 1.70e-04 | 5781.8ms/step | 4.2 samples/s
Step 12190/20005 | Loss: 2.9440 | LR: 1.69e-04 | 5782.1ms/step | 4.2 samples/s
Step 12200/20005 | Loss: 2.9112 | LR: 1.69e-04 | 5783.3ms/step | 4.1 samples/s
Step 12210/20005 | Loss: 2.9194 | LR: 1.69e-04 | 5779.1ms/step | 4.2 samples/s
Step 12220/20005 | Loss: 2.9267 | LR: 1.68e-04 | 5779.8ms/step | 4.2 samples/s
Step 12230/20005 | Loss: 2.9245 | LR: 1.68e-04 | 5780.5ms/step | 4.2 samples/s
Step 12240/20005 | Loss: 2.9381 | LR: 1.68e-04 | 5782.6ms/step | 4.2 samples/s
Step 12250/20005 | Loss: 2.9310 | LR: 1.67e-04 | 5780.3ms/step | 4.2 samples/s
Validation at step 12250: Loss = 2.9325, Perplexity = 18.77
Step 12260/20005 | Loss: 2.9066 | LR: 1.67e-04 | 40698.6ms/step | 0.6 samples/s
Step 12270/20005 | Loss: 2.9272 | LR: 1.66e-04 | 5778.4ms/step | 4.2 samples/s
Step 12280/20005 | Loss: 2.9257 | LR: 1.66e-04 | 5780.4ms/step | 4.2 samples/s
Step 12290/20005 | Loss: 2.9057 | LR: 1.66e-04 | 5779.1ms/step | 4.2 samples/s
Step 12300/20005 | Loss: 2.9180 | LR: 1.65e-04 | 5780.1ms/step | 4.2 samples/s
Step 12310/20005 | Loss: 2.9173 | LR: 1.65e-04 | 5824.5ms/step | 4.1 samples/s
Step 12320/20005 | Loss: 2.9361 | LR: 1.65e-04 | 5781.7ms/step | 4.2 samples/s
Step 12330/20005 | Loss: 2.9249 | LR: 1.64e-04 | 5782.5ms/step | 4.2 samples/s
Step 12340/20005 | Loss: 2.9355 | LR: 1.64e-04 | 5782.0ms/step | 4.2 samples/s
Step 12350/20005 | Loss: 2.9728 | LR: 1.63e-04 | 5781.9ms/step | 4.2 samples/s
Step 12360/20005 | Loss: 2.8903 | LR: 1.63e-04 | 5783.8ms/step | 4.1 samples/s
Step 12370/20005 | Loss: 2.9191 | LR: 1.63e-04 | 5782.2ms/step | 4.2 samples/s
Step 12380/20005 | Loss: 2.9172 | LR: 1.62e-04 | 5785.8ms/step | 4.1 samples/s
Step 12390/20005 | Loss: 2.9390 | LR: 1.62e-04 | 5784.2ms/step | 4.1 samples/s
Step 12400/20005 | Loss: 2.9381 | LR: 1.62e-04 | 5780.1ms/step | 4.2 samples/s
Step 12410/20005 | Loss: 2.9199 | LR: 1.61e-04 | 5781.2ms/step | 4.2 samples/s
Step 12420/20005 | Loss: 2.9200 | LR: 1.61e-04 | 5782.7ms/step | 4.2 samples/s
Step 12430/20005 | Loss: 2.9230 | LR: 1.60e-04 | 5784.4ms/step | 4.1 samples/s
Step 12440/20005 | Loss: 2.9498 | LR: 1.60e-04 | 5783.5ms/step | 4.1 samples/s
Step 12450/20005 | Loss: 2.9432 | LR: 1.60e-04 | 5781.9ms/step | 4.2 samples/s
Step 12460/20005 | Loss: 2.9135 | LR: 1.59e-04 | 5783.2ms/step | 4.1 samples/s
Step 12470/20005 | Loss: 2.9335 | LR: 1.59e-04 | 5779.5ms/step | 4.2 samples/s
Step 12480/20005 | Loss: 2.9101 | LR: 1.59e-04 | 5783.4ms/step | 4.1 samples/s
Step 12490/20005 | Loss: 2.9079 | LR: 1.58e-04 | 5784.8ms/step | 4.1 samples/s
Step 12500/20005 | Loss: 2.9639 | LR: 1.58e-04 | 5782.4ms/step | 4.2 samples/s
Validation at step 12500: Loss = 2.9289, Perplexity = 18.71
Step 12510/20005 | Loss: 2.8990 | LR: 1.58e-04 | 40721.7ms/step | 0.6 samples/s
Step 12520/20005 | Loss: 2.9229 | LR: 1.57e-04 | 5786.4ms/step | 4.1 samples/s
Step 12530/20005 | Loss: 2.9132 | LR: 1.57e-04 | 5783.5ms/step | 4.1 samples/s
Step 12540/20005 | Loss: 2.8913 | LR: 1.56e-04 | 5782.2ms/step | 4.2 samples/s
Step 12550/20005 | Loss: 2.9262 | LR: 1.56e-04 | 5784.5ms/step | 4.1 samples/s
Step 12560/20005 | Loss: 2.9093 | LR: 1.56e-04 | 5781.8ms/step | 4.2 samples/s
Step 12570/20005 | Loss: 2.9524 | LR: 1.55e-04 | 5781.8ms/step | 4.2 samples/s
Step 12580/20005 | Loss: 2.8896 | LR: 1.55e-04 | 5780.1ms/step | 4.2 samples/s
Step 12590/20005 | Loss: 2.9161 | LR: 1.55e-04 | 5783.1ms/step | 4.1 samples/s
Step 12600/20005 | Loss: 2.9089 | LR: 1.54e-04 | 5781.0ms/step | 4.2 samples/s
Step 12610/20005 | Loss: 2.9437 | LR: 1.54e-04 | 5783.3ms/step | 4.1 samples/s
Step 12620/20005 | Loss: 2.9324 | LR: 1.53e-04 | 5781.9ms/step | 4.2 samples/s
Step 12630/20005 | Loss: 2.9112 | LR: 1.53e-04 | 5780.7ms/step | 4.2 samples/s
Step 12640/20005 | Loss: 2.9217 | LR: 1.53e-04 | 5782.2ms/step | 4.2 samples/s
Step 12650/20005 | Loss: 2.8932 | LR: 1.52e-04 | 5782.0ms/step | 4.2 samples/s
Step 12660/20005 | Loss: 2.9161 | LR: 1.52e-04 | 5780.4ms/step | 4.2 samples/s
Step 12670/20005 | Loss: 2.9247 | LR: 1.52e-04 | 5784.4ms/step | 4.1 samples/s
Step 12680/20005 | Loss: 2.9441 | LR: 1.51e-04 | 5784.5ms/step | 4.1 samples/s
Step 12690/20005 | Loss: 2.9077 | LR: 1.51e-04 | 5780.8ms/step | 4.2 samples/s
Step 12700/20005 | Loss: 2.9134 | LR: 1.51e-04 | 5785.1ms/step | 4.1 samples/s
Step 12710/20005 | Loss: 2.9047 | LR: 1.50e-04 | 5786.4ms/step | 4.1 samples/s
Step 12720/20005 | Loss: 2.9352 | LR: 1.50e-04 | 5785.9ms/step | 4.1 samples/s
Step 12730/20005 | Loss: 2.9379 | LR: 1.49e-04 | 5785.8ms/step | 4.1 samples/s
Step 12740/20005 | Loss: 2.9230 | LR: 1.49e-04 | 5783.1ms/step | 4.2 samples/s
Step 12750/20005 | Loss: 2.9193 | LR: 1.49e-04 | 5785.2ms/step | 4.1 samples/s
Validation at step 12750: Loss = 2.9255, Perplexity = 18.64
Step 12760/20005 | Loss: 2.9018 | LR: 1.48e-04 | 40715.7ms/step | 0.6 samples/s
Step 12770/20005 | Loss: 2.9161 | LR: 1.48e-04 | 5782.5ms/step | 4.2 samples/s
Step 12780/20005 | Loss: 2.9224 | LR: 1.48e-04 | 5784.7ms/step | 4.1 samples/s
Step 12790/20005 | Loss: 2.9173 | LR: 1.47e-04 | 5823.2ms/step | 4.1 samples/s
Step 12800/20005 | Loss: 2.9132 | LR: 1.47e-04 | 5781.3ms/step | 4.2 samples/s
Step 12810/20005 | Loss: 2.9313 | LR: 1.47e-04 | 5781.8ms/step | 4.2 samples/s
Step 12820/20005 | Loss: 2.9246 | LR: 1.46e-04 | 5781.8ms/step | 4.2 samples/s
Step 12830/20005 | Loss: 2.9108 | LR: 1.46e-04 | 5784.2ms/step | 4.1 samples/s
Step 12840/20005 | Loss: 2.9300 | LR: 1.45e-04 | 5782.7ms/step | 4.2 samples/s
Step 12850/20005 | Loss: 2.9211 | LR: 1.45e-04 | 5780.2ms/step | 4.2 samples/s
Step 12860/20005 | Loss: 2.9417 | LR: 1.45e-04 | 5783.3ms/step | 4.1 samples/s
Step 12870/20005 | Loss: 2.9279 | LR: 1.44e-04 | 5780.9ms/step | 4.2 samples/s
Step 12880/20005 | Loss: 2.9488 | LR: 1.44e-04 | 5780.3ms/step | 4.2 samples/s
Step 12890/20005 | Loss: 2.9095 | LR: 1.44e-04 | 5782.6ms/step | 4.2 samples/s
Step 12900/20005 | Loss: 2.9057 | LR: 1.43e-04 | 5782.3ms/step | 4.2 samples/s
Step 12910/20005 | Loss: 2.9177 | LR: 1.43e-04 | 5782.3ms/step | 4.2 samples/s
Step 12920/20005 | Loss: 2.9278 | LR: 1.43e-04 | 5781.7ms/step | 4.2 samples/s
Step 12930/20005 | Loss: 2.9140 | LR: 1.42e-04 | 5784.0ms/step | 4.1 samples/s
Step 12940/20005 | Loss: 2.9065 | LR: 1.42e-04 | 5784.3ms/step | 4.1 samples/s
Step 12950/20005 | Loss: 2.9216 | LR: 1.42e-04 | 5780.8ms/step | 4.2 samples/s
Step 12960/20005 | Loss: 2.9243 | LR: 1.41e-04 | 5781.1ms/step | 4.2 samples/s
Step 12970/20005 | Loss: 2.9225 | LR: 1.41e-04 | 5783.7ms/step | 4.1 samples/s
Step 12980/20005 | Loss: 2.9299 | LR: 1.40e-04 | 5782.3ms/step | 4.2 samples/s
Step 12990/20005 | Loss: 2.9322 | LR: 1.40e-04 | 5782.5ms/step | 4.2 samples/s
Step 13000/20005 | Loss: 2.9207 | LR: 1.40e-04 | 5784.1ms/step | 4.1 samples/s
Validation at step 13000: Loss = 2.9223, Perplexity = 18.58
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-13000
Step 13010/20005 | Loss: 2.9307 | LR: 1.39e-04 | 41214.0ms/step | 0.6 samples/s
Step 13020/20005 | Loss: 2.9176 | LR: 1.39e-04 | 5781.8ms/step | 4.2 samples/s
Step 13030/20005 | Loss: 2.9297 | LR: 1.39e-04 | 5780.5ms/step | 4.2 samples/s
Step 13040/20005 | Loss: 2.9020 | LR: 1.38e-04 | 5783.3ms/step | 4.1 samples/s
Step 13050/20005 | Loss: 2.9056 | LR: 1.38e-04 | 5783.7ms/step | 4.1 samples/s
Step 13060/20005 | Loss: 2.9246 | LR: 1.38e-04 | 5784.7ms/step | 4.1 samples/s
Step 13070/20005 | Loss: 2.8914 | LR: 1.37e-04 | 5786.2ms/step | 4.1 samples/s
Step 13080/20005 | Loss: 2.9469 | LR: 1.37e-04 | 5787.2ms/step | 4.1 samples/s
Step 13090/20005 | Loss: 2.9146 | LR: 1.37e-04 | 5788.7ms/step | 4.1 samples/s
Step 13100/20005 | Loss: 2.9159 | LR: 1.36e-04 | 5785.6ms/step | 4.1 samples/s
Step 13110/20005 | Loss: 2.9005 | LR: 1.36e-04 | 5785.9ms/step | 4.1 samples/s
Step 13120/20005 | Loss: 2.9136 | LR: 1.35e-04 | 5784.7ms/step | 4.1 samples/s
Step 13130/20005 | Loss: 2.9067 | LR: 1.35e-04 | 5782.6ms/step | 4.2 samples/s
Step 13140/20005 | Loss: 2.9157 | LR: 1.35e-04 | 5783.1ms/step | 4.2 samples/s
Step 13150/20005 | Loss: 2.9147 | LR: 1.34e-04 | 5782.0ms/step | 4.2 samples/s
Step 13160/20005 | Loss: 2.9254 | LR: 1.34e-04 | 5781.5ms/step | 4.2 samples/s
Step 13170/20005 | Loss: 2.9143 | LR: 1.34e-04 | 5781.7ms/step | 4.2 samples/s
Step 13180/20005 | Loss: 2.9016 | LR: 1.33e-04 | 5784.8ms/step | 4.1 samples/s
Step 13190/20005 | Loss: 2.9036 | LR: 1.33e-04 | 5784.4ms/step | 4.1 samples/s
Step 13200/20005 | Loss: 2.9287 | LR: 1.33e-04 | 5782.5ms/step | 4.2 samples/s
Step 13210/20005 | Loss: 2.9048 | LR: 1.32e-04 | 5781.9ms/step | 4.2 samples/s
Step 13220/20005 | Loss: 2.9571 | LR: 1.32e-04 | 5781.3ms/step | 4.2 samples/s
Step 13230/20005 | Loss: 2.9115 | LR: 1.32e-04 | 5782.8ms/step | 4.2 samples/s
Step 13240/20005 | Loss: 2.9353 | LR: 1.31e-04 | 5784.4ms/step | 4.1 samples/s
Step 13250/20005 | Loss: 2.9086 | LR: 1.31e-04 | 5779.7ms/step | 4.2 samples/s
Validation at step 13250: Loss = 2.9191, Perplexity = 18.52
Step 13260/20005 | Loss: 2.9125 | LR: 1.31e-04 | 40740.0ms/step | 0.6 samples/s
Step 13270/20005 | Loss: 2.8888 | LR: 1.30e-04 | 5783.5ms/step | 4.1 samples/s
Step 13280/20005 | Loss: 2.9328 | LR: 1.30e-04 | 5784.0ms/step | 4.1 samples/s
Step 13290/20005 | Loss: 2.9063 | LR: 1.30e-04 | 5782.3ms/step | 4.2 samples/s
Step 13300/20005 | Loss: 2.9296 | LR: 1.29e-04 | 5784.2ms/step | 4.1 samples/s
Step 13310/20005 | Loss: 2.9156 | LR: 1.29e-04 | 5782.7ms/step | 4.2 samples/s
Step 13320/20005 | Loss: 2.9024 | LR: 1.28e-04 | 5783.1ms/step | 4.2 samples/s
Step 13330/20005 | Loss: 2.9228 | LR: 1.28e-04 | 5781.7ms/step | 4.2 samples/s
Step 13340/20005 | Loss: 2.9188 | LR: 1.28e-04 | 5782.2ms/step | 4.2 samples/s
Step 13350/20005 | Loss: 2.9153 | LR: 1.27e-04 | 5784.5ms/step | 4.1 samples/s
Step 13360/20005 | Loss: 2.9071 | LR: 1.27e-04 | 5785.4ms/step | 4.1 samples/s
Step 13370/20005 | Loss: 2.9431 | LR: 1.27e-04 | 5785.8ms/step | 4.1 samples/s
Step 13380/20005 | Loss: 2.8990 | LR: 1.26e-04 | 5782.3ms/step | 4.2 samples/s
Step 13390/20005 | Loss: 2.9108 | LR: 1.26e-04 | 5780.8ms/step | 4.2 samples/s
Step 13400/20005 | Loss: 2.9053 | LR: 1.26e-04 | 5784.3ms/step | 4.1 samples/s
Step 13410/20005 | Loss: 2.9152 | LR: 1.25e-04 | 5782.9ms/step | 4.2 samples/s
Step 13420/20005 | Loss: 2.8916 | LR: 1.25e-04 | 5784.6ms/step | 4.1 samples/s
Step 13430/20005 | Loss: 2.9113 | LR: 1.25e-04 | 5785.8ms/step | 4.1 samples/s
Step 13440/20005 | Loss: 2.8867 | LR: 1.24e-04 | 5784.0ms/step | 4.1 samples/s
Step 13450/20005 | Loss: 2.9055 | LR: 1.24e-04 | 5787.8ms/step | 4.1 samples/s
Step 13460/20005 | Loss: 2.8966 | LR: 1.24e-04 | 5786.1ms/step | 4.1 samples/s
Step 13470/20005 | Loss: 2.8972 | LR: 1.23e-04 | 5784.0ms/step | 4.1 samples/s
Step 13480/20005 | Loss: 2.9025 | LR: 1.23e-04 | 5784.1ms/step | 4.1 samples/s
Step 13490/20005 | Loss: 2.9097 | LR: 1.23e-04 | 5780.6ms/step | 4.2 samples/s
Step 13500/20005 | Loss: 2.9001 | LR: 1.22e-04 | 5782.3ms/step | 4.2 samples/s
Validation at step 13500: Loss = 2.9160, Perplexity = 18.47
Step 13510/20005 | Loss: 2.8912 | LR: 1.22e-04 | 40728.8ms/step | 0.6 samples/s
Step 13520/20005 | Loss: 2.9021 | LR: 1.22e-04 | 5783.4ms/step | 4.1 samples/s
Step 13530/20005 | Loss: 2.9000 | LR: 1.21e-04 | 5785.6ms/step | 4.1 samples/s
Step 13540/20005 | Loss: 2.8921 | LR: 1.21e-04 | 5782.3ms/step | 4.2 samples/s
Step 13550/20005 | Loss: 2.9183 | LR: 1.21e-04 | 5783.6ms/step | 4.1 samples/s
Step 13560/20005 | Loss: 2.9182 | LR: 1.20e-04 | 5780.6ms/step | 4.2 samples/s
Step 13570/20005 | Loss: 2.8986 | LR: 1.20e-04 | 5785.0ms/step | 4.1 samples/s
Step 13580/20005 | Loss: 2.9092 | LR: 1.20e-04 | 5782.4ms/step | 4.2 samples/s
Step 13590/20005 | Loss: 2.9144 | LR: 1.19e-04 | 5783.8ms/step | 4.1 samples/s
Step 13600/20005 | Loss: 2.9052 | LR: 1.19e-04 | 5781.0ms/step | 4.2 samples/s
Step 13610/20005 | Loss: 2.9101 | LR: 1.19e-04 | 5781.8ms/step | 4.2 samples/s
Step 13620/20005 | Loss: 2.9222 | LR: 1.18e-04 | 5782.1ms/step | 4.2 samples/s
Step 13630/20005 | Loss: 2.9232 | LR: 1.18e-04 | 5785.2ms/step | 4.1 samples/s
Step 13640/20005 | Loss: 2.9243 | LR: 1.18e-04 | 5780.4ms/step | 4.2 samples/s
Step 13650/20005 | Loss: 2.8918 | LR: 1.17e-04 | 5783.8ms/step | 4.1 samples/s
Step 13660/20005 | Loss: 2.9087 | LR: 1.17e-04 | 5778.9ms/step | 4.2 samples/s
Step 13670/20005 | Loss: 2.8798 | LR: 1.16e-04 | 5781.8ms/step | 4.2 samples/s
Step 13680/20005 | Loss: 2.9266 | LR: 1.16e-04 | 5780.3ms/step | 4.2 samples/s
Step 13690/20005 | Loss: 2.8832 | LR: 1.16e-04 | 5783.8ms/step | 4.1 samples/s
Step 13700/20005 | Loss: 2.9071 | LR: 1.15e-04 | 5781.1ms/step | 4.2 samples/s
Step 13710/20005 | Loss: 2.9064 | LR: 1.15e-04 | 5779.9ms/step | 4.2 samples/s
Step 13720/20005 | Loss: 2.9012 | LR: 1.15e-04 | 5781.1ms/step | 4.2 samples/s
Step 13730/20005 | Loss: 2.9176 | LR: 1.14e-04 | 5784.1ms/step | 4.1 samples/s
Step 13740/20005 | Loss: 2.9297 | LR: 1.14e-04 | 5828.1ms/step | 4.1 samples/s
Step 13750/20005 | Loss: 2.9447 | LR: 1.14e-04 | 5781.7ms/step | 4.2 samples/s
Validation at step 13750: Loss = 2.9136, Perplexity = 18.42
Step 13760/20005 | Loss: 2.9263 | LR: 1.13e-04 | 40729.4ms/step | 0.6 samples/s
Step 13770/20005 | Loss: 2.8886 | LR: 1.13e-04 | 5782.3ms/step | 4.2 samples/s
Step 13780/20005 | Loss: 2.9361 | LR: 1.13e-04 | 5784.9ms/step | 4.1 samples/s
Step 13790/20005 | Loss: 2.9016 | LR: 1.12e-04 | 5782.1ms/step | 4.2 samples/s
Step 13800/20005 | Loss: 2.9208 | LR: 1.12e-04 | 5782.3ms/step | 4.2 samples/s
Step 13810/20005 | Loss: 2.9443 | LR: 1.12e-04 | 5781.6ms/step | 4.2 samples/s
Step 13820/20005 | Loss: 2.8978 | LR: 1.11e-04 | 5782.6ms/step | 4.2 samples/s
Step 13830/20005 | Loss: 2.9065 | LR: 1.11e-04 | 5781.6ms/step | 4.2 samples/s
Step 13840/20005 | Loss: 2.9370 | LR: 1.11e-04 | 5782.3ms/step | 4.2 samples/s
Step 13850/20005 | Loss: 2.9056 | LR: 1.10e-04 | 5784.2ms/step | 4.1 samples/s
Step 13860/20005 | Loss: 2.9083 | LR: 1.10e-04 | 5781.3ms/step | 4.2 samples/s
Step 13870/20005 | Loss: 2.8983 | LR: 1.10e-04 | 5783.6ms/step | 4.1 samples/s
Step 13880/20005 | Loss: 2.9129 | LR: 1.10e-04 | 5782.5ms/step | 4.2 samples/s
Step 13890/20005 | Loss: 2.9041 | LR: 1.09e-04 | 5783.2ms/step | 4.1 samples/s
Step 13900/20005 | Loss: 2.8784 | LR: 1.09e-04 | 5782.3ms/step | 4.2 samples/s
Step 13910/20005 | Loss: 2.9023 | LR: 1.09e-04 | 5781.9ms/step | 4.2 samples/s
Step 13920/20005 | Loss: 2.9122 | LR: 1.08e-04 | 5783.4ms/step | 4.1 samples/s
Step 13930/20005 | Loss: 2.9099 | LR: 1.08e-04 | 5779.8ms/step | 4.2 samples/s
Step 13940/20005 | Loss: 2.9244 | LR: 1.08e-04 | 5779.8ms/step | 4.2 samples/s
Step 13950/20005 | Loss: 2.9135 | LR: 1.07e-04 | 5780.5ms/step | 4.2 samples/s
Step 13960/20005 | Loss: 2.8937 | LR: 1.07e-04 | 5779.7ms/step | 4.2 samples/s
Step 13970/20005 | Loss: 2.9291 | LR: 1.07e-04 | 5780.0ms/step | 4.2 samples/s
Step 13980/20005 | Loss: 2.8964 | LR: 1.06e-04 | 5781.4ms/step | 4.2 samples/s
Step 13990/20005 | Loss: 2.9277 | LR: 1.06e-04 | 5778.9ms/step | 4.2 samples/s
Step 14000/20005 | Loss: 2.9300 | LR: 1.06e-04 | 5778.2ms/step | 4.2 samples/s
Validation at step 14000: Loss = 2.9111, Perplexity = 18.38
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-14000
Step 14010/20005 | Loss: 2.9104 | LR: 1.05e-04 | 41183.6ms/step | 0.6 samples/s
Step 14020/20005 | Loss: 2.8884 | LR: 1.05e-04 | 5779.6ms/step | 4.2 samples/s
Step 14030/20005 | Loss: 2.9428 | LR: 1.05e-04 | 5777.2ms/step | 4.2 samples/s
Step 14040/20005 | Loss: 2.9168 | LR: 1.04e-04 | 5780.2ms/step | 4.2 samples/s
Step 14050/20005 | Loss: 2.9204 | LR: 1.04e-04 | 5783.8ms/step | 4.1 samples/s
Step 14060/20005 | Loss: 2.9043 | LR: 1.04e-04 | 5783.7ms/step | 4.1 samples/s
Step 14070/20005 | Loss: 2.8811 | LR: 1.03e-04 | 5785.2ms/step | 4.1 samples/s
Step 14080/20005 | Loss: 2.9076 | LR: 1.03e-04 | 5786.9ms/step | 4.1 samples/s
Step 14090/20005 | Loss: 2.9113 | LR: 1.03e-04 | 5789.7ms/step | 4.1 samples/s
Step 14100/20005 | Loss: 2.9080 | LR: 1.02e-04 | 5782.2ms/step | 4.2 samples/s
Step 14110/20005 | Loss: 2.9114 | LR: 1.02e-04 | 5786.3ms/step | 4.1 samples/s
Step 14120/20005 | Loss: 2.9165 | LR: 1.02e-04 | 5781.1ms/step | 4.2 samples/s
Step 14130/20005 | Loss: 2.9224 | LR: 1.01e-04 | 5779.6ms/step | 4.2 samples/s
Step 14140/20005 | Loss: 2.8941 | LR: 1.01e-04 | 5779.2ms/step | 4.2 samples/s
Step 14150/20005 | Loss: 2.9039 | LR: 1.01e-04 | 5779.0ms/step | 4.2 samples/s
Step 14160/20005 | Loss: 2.9119 | LR: 1.00e-04 | 5779.0ms/step | 4.2 samples/s
Step 14170/20005 | Loss: 2.8975 | LR: 1.00e-04 | 5779.5ms/step | 4.2 samples/s
Step 14180/20005 | Loss: 2.9164 | LR: 9.98e-05 | 5782.2ms/step | 4.2 samples/s
Step 14190/20005 | Loss: 2.8919 | LR: 9.95e-05 | 5780.5ms/step | 4.2 samples/s
Step 14200/20005 | Loss: 2.9321 | LR: 9.92e-05 | 5779.9ms/step | 4.2 samples/s
Step 14210/20005 | Loss: 2.8991 | LR: 9.89e-05 | 5780.3ms/step | 4.2 samples/s
Step 14220/20005 | Loss: 2.9073 | LR: 9.85e-05 | 5846.2ms/step | 4.1 samples/s
Step 14230/20005 | Loss: 2.8840 | LR: 9.82e-05 | 5783.3ms/step | 4.1 samples/s
Step 14240/20005 | Loss: 2.9018 | LR: 9.79e-05 | 5782.7ms/step | 4.2 samples/s
Step 14250/20005 | Loss: 2.9293 | LR: 9.76e-05 | 5781.6ms/step | 4.2 samples/s
Validation at step 14250: Loss = 2.9091, Perplexity = 18.34
Step 14260/20005 | Loss: 2.8970 | LR: 9.73e-05 | 40636.2ms/step | 0.6 samples/s
Step 14270/20005 | Loss: 2.8862 | LR: 9.70e-05 | 5786.8ms/step | 4.1 samples/s
Step 14280/20005 | Loss: 2.8977 | LR: 9.66e-05 | 5787.4ms/step | 4.1 samples/s
Step 14290/20005 | Loss: 2.9141 | LR: 9.63e-05 | 5781.7ms/step | 4.2 samples/s
Step 14300/20005 | Loss: 2.8762 | LR: 9.60e-05 | 5780.9ms/step | 4.2 samples/s
Step 14310/20005 | Loss: 2.9155 | LR: 9.57e-05 | 5779.4ms/step | 4.2 samples/s
Step 14320/20005 | Loss: 2.9063 | LR: 9.54e-05 | 5778.0ms/step | 4.2 samples/s
Step 14330/20005 | Loss: 2.9035 | LR: 9.51e-05 | 5790.1ms/step | 4.1 samples/s
Step 14340/20005 | Loss: 2.9073 | LR: 9.48e-05 | 5786.3ms/step | 4.1 samples/s
Step 14350/20005 | Loss: 2.9109 | LR: 9.45e-05 | 5787.7ms/step | 4.1 samples/s
Step 14360/20005 | Loss: 2.9087 | LR: 9.41e-05 | 5786.5ms/step | 4.1 samples/s
Step 14370/20005 | Loss: 2.8920 | LR: 9.38e-05 | 5777.7ms/step | 4.2 samples/s
Step 14380/20005 | Loss: 2.9165 | LR: 9.35e-05 | 5780.3ms/step | 4.2 samples/s
Step 14390/20005 | Loss: 2.9197 | LR: 9.32e-05 | 5782.6ms/step | 4.2 samples/s
Step 14400/20005 | Loss: 2.8898 | LR: 9.29e-05 | 5777.6ms/step | 4.2 samples/s
Step 14410/20005 | Loss: 2.9168 | LR: 9.26e-05 | 5773.9ms/step | 4.2 samples/s
Step 14420/20005 | Loss: 2.9205 | LR: 9.23e-05 | 5774.0ms/step | 4.2 samples/s
Step 14430/20005 | Loss: 2.9376 | LR: 9.20e-05 | 5774.9ms/step | 4.2 samples/s
Step 14440/20005 | Loss: 2.8926 | LR: 9.17e-05 | 5776.9ms/step | 4.2 samples/s
Step 14450/20005 | Loss: 2.8944 | LR: 9.14e-05 | 5777.1ms/step | 4.2 samples/s
Step 14460/20005 | Loss: 2.8921 | LR: 9.11e-05 | 5779.9ms/step | 4.2 samples/s
Step 14470/20005 | Loss: 2.9131 | LR: 9.08e-05 | 5779.4ms/step | 4.2 samples/s
Step 14480/20005 | Loss: 2.8946 | LR: 9.04e-05 | 5777.9ms/step | 4.2 samples/s
Step 14490/20005 | Loss: 2.9104 | LR: 9.01e-05 | 5780.3ms/step | 4.2 samples/s
Step 14500/20005 | Loss: 2.8788 | LR: 8.98e-05 | 5776.0ms/step | 4.2 samples/s
Validation at step 14500: Loss = 2.9073, Perplexity = 18.31
Step 14510/20005 | Loss: 2.9275 | LR: 8.95e-05 | 40666.7ms/step | 0.6 samples/s
Step 14520/20005 | Loss: 2.9125 | LR: 8.92e-05 | 5793.1ms/step | 4.1 samples/s
Step 14530/20005 | Loss: 2.9116 | LR: 8.89e-05 | 5784.0ms/step | 4.1 samples/s
Step 14540/20005 | Loss: 2.8865 | LR: 8.86e-05 | 5786.0ms/step | 4.1 samples/s
Step 14550/20005 | Loss: 2.9305 | LR: 8.83e-05 | 5782.5ms/step | 4.2 samples/s
Step 14560/20005 | Loss: 2.9184 | LR: 8.80e-05 | 5782.2ms/step | 4.2 samples/s
Step 14570/20005 | Loss: 2.9130 | LR: 8.77e-05 | 5779.9ms/step | 4.2 samples/s
Step 14580/20005 | Loss: 2.9411 | LR: 8.74e-05 | 5782.8ms/step | 4.2 samples/s
Step 14590/20005 | Loss: 2.8984 | LR: 8.71e-05 | 5787.7ms/step | 4.1 samples/s
Step 14600/20005 | Loss: 2.8871 | LR: 8.68e-05 | 5788.5ms/step | 4.1 samples/s
Step 14610/20005 | Loss: 2.8923 | LR: 8.65e-05 | 5783.2ms/step | 4.1 samples/s
Step 14620/20005 | Loss: 2.9203 | LR: 8.62e-05 | 5783.4ms/step | 4.1 samples/s
Step 14630/20005 | Loss: 2.9139 | LR: 8.59e-05 | 5781.2ms/step | 4.2 samples/s
Step 14640/20005 | Loss: 2.9174 | LR: 8.56e-05 | 5781.6ms/step | 4.2 samples/s
Step 14650/20005 | Loss: 2.9113 | LR: 8.53e-05 | 5785.4ms/step | 4.1 samples/s
Step 14660/20005 | Loss: 2.8956 | LR: 8.50e-05 | 5781.4ms/step | 4.2 samples/s
Step 14670/20005 | Loss: 2.9250 | LR: 8.47e-05 | 5785.5ms/step | 4.1 samples/s
Step 14680/20005 | Loss: 2.9053 | LR: 8.44e-05 | 5783.1ms/step | 4.2 samples/s
Step 14690/20005 | Loss: 2.8880 | LR: 8.41e-05 | 5850.2ms/step | 4.1 samples/s
Step 14700/20005 | Loss: 2.8971 | LR: 8.38e-05 | 5782.1ms/step | 4.2 samples/s
Step 14710/20005 | Loss: 2.9022 | LR: 8.35e-05 | 5784.6ms/step | 4.1 samples/s
Step 14720/20005 | Loss: 2.9260 | LR: 8.32e-05 | 5783.7ms/step | 4.1 samples/s
Step 14730/20005 | Loss: 2.9034 | LR: 8.29e-05 | 5781.2ms/step | 4.2 samples/s
Step 14740/20005 | Loss: 2.9193 | LR: 8.26e-05 | 5779.5ms/step | 4.2 samples/s
Step 14750/20005 | Loss: 2.9184 | LR: 8.23e-05 | 5783.6ms/step | 4.1 samples/s
Validation at step 14750: Loss = 2.9057, Perplexity = 18.28
Step 14760/20005 | Loss: 2.8978 | LR: 8.20e-05 | 40736.2ms/step | 0.6 samples/s
Step 14770/20005 | Loss: 2.9148 | LR: 8.17e-05 | 5783.8ms/step | 4.1 samples/s
Step 14780/20005 | Loss: 2.8840 | LR: 8.15e-05 | 5784.7ms/step | 4.1 samples/s
Step 14790/20005 | Loss: 2.9302 | LR: 8.12e-05 | 5784.3ms/step | 4.1 samples/s
Step 14800/20005 | Loss: 2.9219 | LR: 8.09e-05 | 5783.1ms/step | 4.2 samples/s
Step 14810/20005 | Loss: 2.9204 | LR: 8.06e-05 | 5789.4ms/step | 4.1 samples/s
Step 14820/20005 | Loss: 2.8787 | LR: 8.03e-05 | 5786.4ms/step | 4.1 samples/s
Step 14830/20005 | Loss: 2.9027 | LR: 8.00e-05 | 5788.6ms/step | 4.1 samples/s
Step 14840/20005 | Loss: 2.9034 | LR: 7.97e-05 | 5782.5ms/step | 4.2 samples/s
Step 14850/20005 | Loss: 2.9233 | LR: 7.94e-05 | 5781.3ms/step | 4.2 samples/s
Step 14860/20005 | Loss: 2.9158 | LR: 7.91e-05 | 5782.7ms/step | 4.2 samples/s
Step 14870/20005 | Loss: 2.8881 | LR: 7.88e-05 | 5785.5ms/step | 4.1 samples/s
Step 14880/20005 | Loss: 2.8964 | LR: 7.85e-05 | 5782.3ms/step | 4.2 samples/s
Step 14890/20005 | Loss: 2.9237 | LR: 7.82e-05 | 5782.1ms/step | 4.2 samples/s
Step 14900/20005 | Loss: 2.8939 | LR: 7.80e-05 | 5781.3ms/step | 4.2 samples/s
Step 14910/20005 | Loss: 2.8899 | LR: 7.77e-05 | 5781.8ms/step | 4.2 samples/s
Step 14920/20005 | Loss: 2.8805 | LR: 7.74e-05 | 5780.3ms/step | 4.2 samples/s
Step 14930/20005 | Loss: 2.9317 | LR: 7.71e-05 | 5783.1ms/step | 4.2 samples/s
Step 14940/20005 | Loss: 2.9024 | LR: 7.68e-05 | 5796.8ms/step | 4.1 samples/s
Step 14950/20005 | Loss: 2.9156 | LR: 7.65e-05 | 5794.4ms/step | 4.1 samples/s
Step 14960/20005 | Loss: 2.9082 | LR: 7.62e-05 | 5785.2ms/step | 4.1 samples/s
Step 14970/20005 | Loss: 2.9255 | LR: 7.59e-05 | 5786.0ms/step | 4.1 samples/s
Step 14980/20005 | Loss: 2.9433 | LR: 7.57e-05 | 5782.4ms/step | 4.2 samples/s
Step 14990/20005 | Loss: 2.9052 | LR: 7.54e-05 | 5784.2ms/step | 4.1 samples/s
Step 15000/20005 | Loss: 2.9275 | LR: 7.51e-05 | 5780.1ms/step | 4.2 samples/s
Validation at step 15000: Loss = 2.9045, Perplexity = 18.26
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-15000
Step 15010/20005 | Loss: 2.9099 | LR: 7.48e-05 | 41188.4ms/step | 0.6 samples/s
Step 15020/20005 | Loss: 2.8782 | LR: 7.45e-05 | 5783.1ms/step | 4.2 samples/s
Step 15030/20005 | Loss: 2.8937 | LR: 7.42e-05 | 5779.4ms/step | 4.2 samples/s
Step 15040/20005 | Loss: 2.8851 | LR: 7.40e-05 | 5779.6ms/step | 4.2 samples/s
Step 15050/20005 | Loss: 2.8989 | LR: 7.37e-05 | 5779.9ms/step | 4.2 samples/s
Step 15060/20005 | Loss: 2.9130 | LR: 7.34e-05 | 5779.9ms/step | 4.2 samples/s
Step 15070/20005 | Loss: 2.9140 | LR: 7.31e-05 | 5778.6ms/step | 4.2 samples/s
Step 15080/20005 | Loss: 2.9032 | LR: 7.28e-05 | 5781.8ms/step | 4.2 samples/s
Step 15090/20005 | Loss: 2.8820 | LR: 7.26e-05 | 5779.8ms/step | 4.2 samples/s
Step 15100/20005 | Loss: 2.9138 | LR: 7.23e-05 | 5785.3ms/step | 4.1 samples/s
Step 15110/20005 | Loss: 2.8931 | LR: 7.20e-05 | 5785.0ms/step | 4.1 samples/s
Step 15120/20005 | Loss: 2.9211 | LR: 7.17e-05 | 5780.2ms/step | 4.2 samples/s
Step 15130/20005 | Loss: 2.9138 | LR: 7.14e-05 | 5786.3ms/step | 4.1 samples/s
Step 15140/20005 | Loss: 2.9161 | LR: 7.12e-05 | 5779.5ms/step | 4.2 samples/s
Step 15150/20005 | Loss: 2.8981 | LR: 7.09e-05 | 5780.0ms/step | 4.2 samples/s
Step 15160/20005 | Loss: 2.9184 | LR: 7.06e-05 | 5782.3ms/step | 4.2 samples/s
Step 15170/20005 | Loss: 2.9151 | LR: 7.03e-05 | 5845.0ms/step | 4.1 samples/s
Step 15180/20005 | Loss: 2.9113 | LR: 7.01e-05 | 5780.8ms/step | 4.2 samples/s
Step 15190/20005 | Loss: 2.8976 | LR: 6.98e-05 | 5781.4ms/step | 4.2 samples/s
Step 15200/20005 | Loss: 2.9110 | LR: 6.95e-05 | 5780.8ms/step | 4.2 samples/s
Step 15210/20005 | Loss: 2.8885 | LR: 6.92e-05 | 5780.2ms/step | 4.2 samples/s
Step 15220/20005 | Loss: 2.9094 | LR: 6.90e-05 | 5779.4ms/step | 4.2 samples/s
Step 15230/20005 | Loss: 2.9089 | LR: 6.87e-05 | 5780.5ms/step | 4.2 samples/s
Step 15240/20005 | Loss: 2.8788 | LR: 6.84e-05 | 5783.0ms/step | 4.2 samples/s
Step 15250/20005 | Loss: 2.8961 | LR: 6.81e-05 | 5779.2ms/step | 4.2 samples/s
Validation at step 15250: Loss = 2.9032, Perplexity = 18.23
Step 15260/20005 | Loss: 2.8824 | LR: 6.79e-05 | 40706.4ms/step | 0.6 samples/s
Step 15270/20005 | Loss: 2.8739 | LR: 6.76e-05 | 5779.9ms/step | 4.2 samples/s
Step 15280/20005 | Loss: 2.8896 | LR: 6.73e-05 | 5782.0ms/step | 4.2 samples/s
Step 15290/20005 | Loss: 2.8990 | LR: 6.70e-05 | 5781.5ms/step | 4.2 samples/s
Step 15300/20005 | Loss: 2.8788 | LR: 6.68e-05 | 5780.1ms/step | 4.2 samples/s
Step 15310/20005 | Loss: 2.8810 | LR: 6.65e-05 | 5789.8ms/step | 4.1 samples/s
Step 15320/20005 | Loss: 2.9005 | LR: 6.62e-05 | 5792.0ms/step | 4.1 samples/s
Step 15330/20005 | Loss: 2.9062 | LR: 6.60e-05 | 5783.6ms/step | 4.1 samples/s
Step 15340/20005 | Loss: 2.9106 | LR: 6.57e-05 | 5782.4ms/step | 4.2 samples/s
Step 15350/20005 | Loss: 2.8792 | LR: 6.54e-05 | 5782.3ms/step | 4.2 samples/s
Step 15360/20005 | Loss: 2.9125 | LR: 6.52e-05 | 5780.3ms/step | 4.2 samples/s
Step 15370/20005 | Loss: 2.9256 | LR: 6.49e-05 | 5789.7ms/step | 4.1 samples/s
Step 15380/20005 | Loss: 2.9051 | LR: 6.46e-05 | 5783.6ms/step | 4.1 samples/s
Step 15390/20005 | Loss: 2.8943 | LR: 6.44e-05 | 5778.7ms/step | 4.2 samples/s
Step 15400/20005 | Loss: 2.8832 | LR: 6.41e-05 | 5780.4ms/step | 4.2 samples/s
Step 15410/20005 | Loss: 2.9022 | LR: 6.38e-05 | 5781.4ms/step | 4.2 samples/s
Step 15420/20005 | Loss: 2.8879 | LR: 6.36e-05 | 5783.3ms/step | 4.1 samples/s
Step 15430/20005 | Loss: 2.9243 | LR: 6.33e-05 | 5780.7ms/step | 4.2 samples/s
Step 15440/20005 | Loss: 2.9178 | LR: 6.30e-05 | 5776.0ms/step | 4.2 samples/s
Step 15450/20005 | Loss: 2.8974 | LR: 6.28e-05 | 5782.1ms/step | 4.2 samples/s
Step 15460/20005 | Loss: 2.8926 | LR: 6.25e-05 | 5782.2ms/step | 4.2 samples/s
Step 15470/20005 | Loss: 2.9120 | LR: 6.22e-05 | 5781.6ms/step | 4.2 samples/s
Step 15480/20005 | Loss: 2.9269 | LR: 6.20e-05 | 5778.6ms/step | 4.2 samples/s
Step 15490/20005 | Loss: 2.9135 | LR: 6.17e-05 | 5778.6ms/step | 4.2 samples/s
Step 15500/20005 | Loss: 2.8947 | LR: 6.15e-05 | 5776.9ms/step | 4.2 samples/s
Validation at step 15500: Loss = 2.9022, Perplexity = 18.21
Step 15510/20005 | Loss: 2.8729 | LR: 6.12e-05 | 40652.6ms/step | 0.6 samples/s
Step 15520/20005 | Loss: 2.8789 | LR: 6.09e-05 | 5780.1ms/step | 4.2 samples/s
Step 15530/20005 | Loss: 2.8786 | LR: 6.07e-05 | 5783.2ms/step | 4.1 samples/s
Step 15540/20005 | Loss: 2.8819 | LR: 6.04e-05 | 5777.9ms/step | 4.2 samples/s
Step 15550/20005 | Loss: 2.9160 | LR: 6.02e-05 | 5778.6ms/step | 4.2 samples/s
Step 15560/20005 | Loss: 2.8912 | LR: 5.99e-05 | 5779.4ms/step | 4.2 samples/s
Step 15570/20005 | Loss: 2.8978 | LR: 5.96e-05 | 5780.8ms/step | 4.2 samples/s
Step 15580/20005 | Loss: 2.8971 | LR: 5.94e-05 | 5778.5ms/step | 4.2 samples/s
Step 15590/20005 | Loss: 2.8972 | LR: 5.91e-05 | 5781.3ms/step | 4.2 samples/s
Step 15600/20005 | Loss: 2.9194 | LR: 5.89e-05 | 5781.4ms/step | 4.2 samples/s
Step 15610/20005 | Loss: 2.9144 | LR: 5.86e-05 | 5778.8ms/step | 4.2 samples/s
Step 15620/20005 | Loss: 2.9074 | LR: 5.84e-05 | 5780.2ms/step | 4.2 samples/s
Step 15630/20005 | Loss: 2.9177 | LR: 5.81e-05 | 5776.2ms/step | 4.2 samples/s
Step 15640/20005 | Loss: 2.8900 | LR: 5.79e-05 | 5783.8ms/step | 4.1 samples/s
Step 15650/20005 | Loss: 2.9049 | LR: 5.76e-05 | 5852.5ms/step | 4.1 samples/s
Step 15660/20005 | Loss: 2.9134 | LR: 5.73e-05 | 5794.3ms/step | 4.1 samples/s
Step 15670/20005 | Loss: 2.9124 | LR: 5.71e-05 | 5783.2ms/step | 4.1 samples/s
Step 15680/20005 | Loss: 2.9301 | LR: 5.68e-05 | 5778.5ms/step | 4.2 samples/s
Step 15690/20005 | Loss: 2.9052 | LR: 5.66e-05 | 5779.6ms/step | 4.2 samples/s
Step 15700/20005 | Loss: 2.8954 | LR: 5.63e-05 | 5780.3ms/step | 4.2 samples/s
Step 15710/20005 | Loss: 2.9087 | LR: 5.61e-05 | 5778.4ms/step | 4.2 samples/s
Step 15720/20005 | Loss: 2.9080 | LR: 5.58e-05 | 5781.0ms/step | 4.2 samples/s
Step 15730/20005 | Loss: 2.8851 | LR: 5.56e-05 | 5778.4ms/step | 4.2 samples/s
Step 15740/20005 | Loss: 2.8976 | LR: 5.53e-05 | 5780.2ms/step | 4.2 samples/s
Step 15750/20005 | Loss: 2.9046 | LR: 5.51e-05 | 5780.3ms/step | 4.2 samples/s
Validation at step 15750: Loss = 2.9013, Perplexity = 18.20
Step 15760/20005 | Loss: 2.9108 | LR: 5.48e-05 | 40648.3ms/step | 0.6 samples/s
Step 15770/20005 | Loss: 2.8785 | LR: 5.46e-05 | 5779.8ms/step | 4.2 samples/s
Step 15780/20005 | Loss: 2.9181 | LR: 5.43e-05 | 5785.7ms/step | 4.1 samples/s
Step 15790/20005 | Loss: 2.9181 | LR: 5.41e-05 | 5777.2ms/step | 4.2 samples/s
Step 15800/20005 | Loss: 2.9070 | LR: 5.38e-05 | 5777.5ms/step | 4.2 samples/s
Step 15810/20005 | Loss: 2.8784 | LR: 5.36e-05 | 5775.6ms/step | 4.2 samples/s
Step 15820/20005 | Loss: 2.9029 | LR: 5.34e-05 | 5772.5ms/step | 4.2 samples/s
Step 15830/20005 | Loss: 2.9259 | LR: 5.31e-05 | 5778.0ms/step | 4.2 samples/s
Step 15840/20005 | Loss: 2.8848 | LR: 5.29e-05 | 5780.3ms/step | 4.2 samples/s
Step 15850/20005 | Loss: 2.9089 | LR: 5.26e-05 | 5781.4ms/step | 4.2 samples/s
Step 15860/20005 | Loss: 2.8953 | LR: 5.24e-05 | 5796.8ms/step | 4.1 samples/s
Step 15870/20005 | Loss: 2.9075 | LR: 5.21e-05 | 5783.6ms/step | 4.1 samples/s
Step 15880/20005 | Loss: 2.9115 | LR: 5.19e-05 | 5783.9ms/step | 4.1 samples/s
Step 15890/20005 | Loss: 2.9103 | LR: 5.16e-05 | 5782.7ms/step | 4.2 samples/s
Step 15900/20005 | Loss: 2.8736 | LR: 5.14e-05 | 5781.4ms/step | 4.2 samples/s
Step 15910/20005 | Loss: 2.8991 | LR: 5.12e-05 | 5780.1ms/step | 4.2 samples/s
Step 15920/20005 | Loss: 2.9012 | LR: 5.09e-05 | 5782.1ms/step | 4.2 samples/s
Step 15930/20005 | Loss: 2.9219 | LR: 5.07e-05 | 5779.5ms/step | 4.2 samples/s
Step 15940/20005 | Loss: 2.9213 | LR: 5.04e-05 | 5780.6ms/step | 4.2 samples/s
Step 15950/20005 | Loss: 2.9057 | LR: 5.02e-05 | 5780.0ms/step | 4.2 samples/s
Step 15960/20005 | Loss: 2.9065 | LR: 5.00e-05 | 5777.8ms/step | 4.2 samples/s
Step 15970/20005 | Loss: 2.8957 | LR: 4.97e-05 | 5779.5ms/step | 4.2 samples/s
Step 15980/20005 | Loss: 2.9206 | LR: 4.95e-05 | 5777.0ms/step | 4.2 samples/s
Step 15990/20005 | Loss: 2.9125 | LR: 4.93e-05 | 5779.8ms/step | 4.2 samples/s
Step 16000/20005 | Loss: 2.8990 | LR: 4.90e-05 | 5780.1ms/step | 4.2 samples/s
Validation at step 16000: Loss = 2.9006, Perplexity = 18.18
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-16000
Step 16010/20005 | Loss: 2.9085 | LR: 4.88e-05 | 41140.1ms/step | 0.6 samples/s
Step 16020/20005 | Loss: 2.8809 | LR: 4.85e-05 | 5778.0ms/step | 4.2 samples/s
Step 16030/20005 | Loss: 2.8855 | LR: 4.83e-05 | 5778.2ms/step | 4.2 samples/s
Step 16040/20005 | Loss: 2.8932 | LR: 4.81e-05 | 5775.6ms/step | 4.2 samples/s
Step 16050/20005 | Loss: 2.8988 | LR: 4.78e-05 | 5776.4ms/step | 4.2 samples/s
Step 16060/20005 | Loss: 2.8951 | LR: 4.76e-05 | 5779.2ms/step | 4.2 samples/s
Step 16070/20005 | Loss: 2.8992 | LR: 4.74e-05 | 5779.0ms/step | 4.2 samples/s
Step 16080/20005 | Loss: 2.9162 | LR: 4.71e-05 | 5777.5ms/step | 4.2 samples/s
Step 16090/20005 | Loss: 2.8995 | LR: 4.69e-05 | 5778.5ms/step | 4.2 samples/s
Step 16100/20005 | Loss: 2.8864 | LR: 4.67e-05 | 5780.4ms/step | 4.2 samples/s
Step 16110/20005 | Loss: 2.8858 | LR: 4.64e-05 | 5777.8ms/step | 4.2 samples/s
Step 16120/20005 | Loss: 2.9116 | LR: 4.62e-05 | 5822.2ms/step | 4.1 samples/s
Step 16130/20005 | Loss: 2.9075 | LR: 4.60e-05 | 5777.5ms/step | 4.2 samples/s
Step 16140/20005 | Loss: 2.8975 | LR: 4.58e-05 | 5777.0ms/step | 4.2 samples/s
Step 16150/20005 | Loss: 2.8989 | LR: 4.55e-05 | 5779.0ms/step | 4.2 samples/s
Step 16160/20005 | Loss: 2.8865 | LR: 4.53e-05 | 5778.2ms/step | 4.2 samples/s
Step 16170/20005 | Loss: 2.9228 | LR: 4.51e-05 | 5776.5ms/step | 4.2 samples/s
Step 16180/20005 | Loss: 2.9053 | LR: 4.48e-05 | 5780.2ms/step | 4.2 samples/s
Step 16190/20005 | Loss: 2.9066 | LR: 4.46e-05 | 5779.8ms/step | 4.2 samples/s
Step 16200/20005 | Loss: 2.9128 | LR: 4.44e-05 | 5776.2ms/step | 4.2 samples/s
Step 16210/20005 | Loss: 2.9389 | LR: 4.42e-05 | 5780.1ms/step | 4.2 samples/s
Step 16220/20005 | Loss: 2.9139 | LR: 4.39e-05 | 5777.7ms/step | 4.2 samples/s
Step 16230/20005 | Loss: 2.9212 | LR: 4.37e-05 | 5776.3ms/step | 4.2 samples/s
Step 16240/20005 | Loss: 2.9014 | LR: 4.35e-05 | 5778.3ms/step | 4.2 samples/s
Step 16250/20005 | Loss: 2.8943 | LR: 4.33e-05 | 5777.7ms/step | 4.2 samples/s
Validation at step 16250: Loss = 2.9000, Perplexity = 18.17
Step 16260/20005 | Loss: 2.8997 | LR: 4.30e-05 | 40638.3ms/step | 0.6 samples/s
Step 16270/20005 | Loss: 2.9054 | LR: 4.28e-05 | 5778.1ms/step | 4.2 samples/s
Step 16280/20005 | Loss: 2.8825 | LR: 4.26e-05 | 5779.1ms/step | 4.2 samples/s
Step 16290/20005 | Loss: 2.8837 | LR: 4.24e-05 | 5774.6ms/step | 4.2 samples/s
Step 16300/20005 | Loss: 2.8995 | LR: 4.22e-05 | 5778.0ms/step | 4.2 samples/s
Step 16310/20005 | Loss: 2.8935 | LR: 4.19e-05 | 5776.4ms/step | 4.2 samples/s
Step 16320/20005 | Loss: 2.9198 | LR: 4.17e-05 | 5775.9ms/step | 4.2 samples/s
Step 16330/20005 | Loss: 2.9006 | LR: 4.15e-05 | 5777.9ms/step | 4.2 samples/s
Step 16340/20005 | Loss: 2.9083 | LR: 4.13e-05 | 5774.9ms/step | 4.2 samples/s
Step 16350/20005 | Loss: 2.8942 | LR: 4.11e-05 | 5777.0ms/step | 4.2 samples/s
Step 16360/20005 | Loss: 2.8936 | LR: 4.08e-05 | 5779.0ms/step | 4.2 samples/s
Step 16370/20005 | Loss: 2.9022 | LR: 4.06e-05 | 5777.2ms/step | 4.2 samples/s
Step 16380/20005 | Loss: 2.8927 | LR: 4.04e-05 | 5780.4ms/step | 4.2 samples/s
Step 16390/20005 | Loss: 2.8770 | LR: 4.02e-05 | 5781.7ms/step | 4.2 samples/s
Step 16400/20005 | Loss: 2.8976 | LR: 4.00e-05 | 5779.9ms/step | 4.2 samples/s
Step 16410/20005 | Loss: 2.9030 | LR: 3.98e-05 | 5780.7ms/step | 4.2 samples/s
Step 16420/20005 | Loss: 2.9218 | LR: 3.95e-05 | 5779.2ms/step | 4.2 samples/s
Step 16430/20005 | Loss: 2.8749 | LR: 3.93e-05 | 5776.7ms/step | 4.2 samples/s
Step 16440/20005 | Loss: 2.8804 | LR: 3.91e-05 | 5779.9ms/step | 4.2 samples/s
Step 16450/20005 | Loss: 2.9090 | LR: 3.89e-05 | 5777.6ms/step | 4.2 samples/s
Step 16460/20005 | Loss: 2.9021 | LR: 3.87e-05 | 5780.4ms/step | 4.2 samples/s
Step 16470/20005 | Loss: 2.9123 | LR: 3.85e-05 | 5780.9ms/step | 4.2 samples/s
Step 16480/20005 | Loss: 2.8982 | LR: 3.83e-05 | 5782.3ms/step | 4.2 samples/s
Step 16490/20005 | Loss: 2.8728 | LR: 3.81e-05 | 5780.3ms/step | 4.2 samples/s
Step 16500/20005 | Loss: 2.8867 | LR: 3.78e-05 | 5780.3ms/step | 4.2 samples/s
Validation at step 16500: Loss = 2.8995, Perplexity = 18.17
Step 16510/20005 | Loss: 2.8907 | LR: 3.76e-05 | 40705.1ms/step | 0.6 samples/s
Step 16520/20005 | Loss: 2.9044 | LR: 3.74e-05 | 5786.5ms/step | 4.1 samples/s
Step 16530/20005 | Loss: 2.8998 | LR: 3.72e-05 | 5786.1ms/step | 4.1 samples/s
Step 16540/20005 | Loss: 2.8998 | LR: 3.70e-05 | 5784.5ms/step | 4.1 samples/s
Step 16550/20005 | Loss: 2.8856 | LR: 3.68e-05 | 5781.5ms/step | 4.2 samples/s
Step 16560/20005 | Loss: 2.9350 | LR: 3.66e-05 | 5782.0ms/step | 4.2 samples/s
Step 16570/20005 | Loss: 2.8758 | LR: 3.64e-05 | 5774.5ms/step | 4.2 samples/s
Step 16580/20005 | Loss: 2.8952 | LR: 3.62e-05 | 5770.4ms/step | 4.2 samples/s
Step 16590/20005 | Loss: 2.8887 | LR: 3.60e-05 | 5774.1ms/step | 4.2 samples/s
Step 16600/20005 | Loss: 2.8855 | LR: 3.58e-05 | 5823.8ms/step | 4.1 samples/s
Step 16610/20005 | Loss: 2.8999 | LR: 3.56e-05 | 5774.1ms/step | 4.2 samples/s
Step 16620/20005 | Loss: 2.9115 | LR: 3.54e-05 | 5774.5ms/step | 4.2 samples/s
Step 16630/20005 | Loss: 2.8703 | LR: 3.52e-05 | 5775.8ms/step | 4.2 samples/s
Step 16640/20005 | Loss: 2.8808 | LR: 3.49e-05 | 5770.2ms/step | 4.2 samples/s
Step 16650/20005 | Loss: 2.9107 | LR: 3.47e-05 | 5778.6ms/step | 4.2 samples/s
Step 16660/20005 | Loss: 2.8929 | LR: 3.45e-05 | 5773.0ms/step | 4.2 samples/s
Step 16670/20005 | Loss: 2.8996 | LR: 3.43e-05 | 5774.4ms/step | 4.2 samples/s
Step 16680/20005 | Loss: 2.9086 | LR: 3.41e-05 | 5778.2ms/step | 4.2 samples/s
Step 16690/20005 | Loss: 2.8768 | LR: 3.39e-05 | 5777.8ms/step | 4.2 samples/s
Step 16700/20005 | Loss: 2.8947 | LR: 3.37e-05 | 5774.1ms/step | 4.2 samples/s
Step 16710/20005 | Loss: 2.8740 | LR: 3.35e-05 | 5779.8ms/step | 4.2 samples/s
Step 16720/20005 | Loss: 2.9135 | LR: 3.33e-05 | 5776.2ms/step | 4.2 samples/s
Step 16730/20005 | Loss: 2.8715 | LR: 3.31e-05 | 5774.1ms/step | 4.2 samples/s
Step 16740/20005 | Loss: 2.8934 | LR: 3.29e-05 | 5778.0ms/step | 4.2 samples/s
Step 16750/20005 | Loss: 2.8749 | LR: 3.28e-05 | 5782.1ms/step | 4.2 samples/s
Validation at step 16750: Loss = 2.8992, Perplexity = 18.16
Step 16760/20005 | Loss: 2.8938 | LR: 3.26e-05 | 40643.9ms/step | 0.6 samples/s
Step 16770/20005 | Loss: 2.8688 | LR: 3.24e-05 | 5776.6ms/step | 4.2 samples/s
Step 16780/20005 | Loss: 2.9080 | LR: 3.22e-05 | 5775.7ms/step | 4.2 samples/s
Step 16790/20005 | Loss: 2.8714 | LR: 3.20e-05 | 5775.5ms/step | 4.2 samples/s
Step 16800/20005 | Loss: 2.8511 | LR: 3.18e-05 | 5775.7ms/step | 4.2 samples/s
Step 16810/20005 | Loss: 2.9043 | LR: 3.16e-05 | 5772.5ms/step | 4.2 samples/s
Step 16820/20005 | Loss: 2.8912 | LR: 3.14e-05 | 5774.1ms/step | 4.2 samples/s
Step 16830/20005 | Loss: 2.8960 | LR: 3.12e-05 | 5776.4ms/step | 4.2 samples/s
Step 16840/20005 | Loss: 2.9119 | LR: 3.10e-05 | 5776.0ms/step | 4.2 samples/s
Step 16850/20005 | Loss: 2.8842 | LR: 3.08e-05 | 5778.9ms/step | 4.2 samples/s
Step 16860/20005 | Loss: 2.9204 | LR: 3.06e-05 | 5776.7ms/step | 4.2 samples/s
Step 16870/20005 | Loss: 2.8720 | LR: 3.04e-05 | 5780.2ms/step | 4.2 samples/s
Step 16880/20005 | Loss: 2.8599 | LR: 3.02e-05 | 5772.3ms/step | 4.2 samples/s
Step 16890/20005 | Loss: 2.8899 | LR: 3.01e-05 | 5772.1ms/step | 4.2 samples/s
Step 16900/20005 | Loss: 2.9090 | LR: 2.99e-05 | 5773.8ms/step | 4.2 samples/s
Step 16910/20005 | Loss: 2.8797 | LR: 2.97e-05 | 5775.9ms/step | 4.2 samples/s
Step 16920/20005 | Loss: 2.9271 | LR: 2.95e-05 | 5778.7ms/step | 4.2 samples/s
Step 16930/20005 | Loss: 2.8896 | LR: 2.93e-05 | 5779.2ms/step | 4.2 samples/s
Step 16940/20005 | Loss: 2.8850 | LR: 2.91e-05 | 5776.0ms/step | 4.2 samples/s
Step 16950/20005 | Loss: 2.9032 | LR: 2.89e-05 | 5774.5ms/step | 4.2 samples/s
Step 16960/20005 | Loss: 2.8762 | LR: 2.87e-05 | 5775.9ms/step | 4.2 samples/s
Step 16970/20005 | Loss: 2.8927 | LR: 2.86e-05 | 5772.0ms/step | 4.2 samples/s
Step 16980/20005 | Loss: 2.8987 | LR: 2.84e-05 | 5771.8ms/step | 4.2 samples/s
Step 16990/20005 | Loss: 2.9059 | LR: 2.82e-05 | 5773.8ms/step | 4.2 samples/s
Step 17000/20005 | Loss: 2.8615 | LR: 2.80e-05 | 5773.0ms/step | 4.2 samples/s
Validation at step 17000: Loss = 2.8989, Perplexity = 18.15
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-17000
Step 17010/20005 | Loss: 2.9158 | LR: 2.78e-05 | 41070.1ms/step | 0.6 samples/s
Step 17020/20005 | Loss: 2.8962 | LR: 2.76e-05 | 5774.1ms/step | 4.2 samples/s
Step 17030/20005 | Loss: 2.8932 | LR: 2.75e-05 | 5773.3ms/step | 4.2 samples/s
Step 17040/20005 | Loss: 2.9090 | LR: 2.73e-05 | 5770.3ms/step | 4.2 samples/s
Step 17050/20005 | Loss: 2.8777 | LR: 2.71e-05 | 5772.5ms/step | 4.2 samples/s
Step 17060/20005 | Loss: 2.9024 | LR: 2.69e-05 | 5771.6ms/step | 4.2 samples/s
Step 17070/20005 | Loss: 2.8907 | LR: 2.67e-05 | 5775.8ms/step | 4.2 samples/s
Step 17080/20005 | Loss: 2.9026 | LR: 2.66e-05 | 5816.1ms/step | 4.1 samples/s
Step 17090/20005 | Loss: 2.9134 | LR: 2.64e-05 | 5771.9ms/step | 4.2 samples/s
Step 17100/20005 | Loss: 2.8964 | LR: 2.62e-05 | 5770.1ms/step | 4.2 samples/s
Step 17110/20005 | Loss: 2.8978 | LR: 2.60e-05 | 5778.3ms/step | 4.2 samples/s
Step 17120/20005 | Loss: 2.8923 | LR: 2.59e-05 | 5774.2ms/step | 4.2 samples/s
Step 17130/20005 | Loss: 2.8911 | LR: 2.57e-05 | 5774.0ms/step | 4.2 samples/s
Step 17140/20005 | Loss: 2.9045 | LR: 2.55e-05 | 5777.7ms/step | 4.2 samples/s
Step 17150/20005 | Loss: 2.8775 | LR: 2.53e-05 | 5774.5ms/step | 4.2 samples/s
Step 17160/20005 | Loss: 2.9176 | LR: 2.52e-05 | 5772.1ms/step | 4.2 samples/s
Step 17170/20005 | Loss: 2.9071 | LR: 2.50e-05 | 5774.2ms/step | 4.2 samples/s
Step 17180/20005 | Loss: 2.8996 | LR: 2.48e-05 | 5767.6ms/step | 4.2 samples/s
Step 17190/20005 | Loss: 2.9205 | LR: 2.46e-05 | 5774.0ms/step | 4.2 samples/s
Step 17200/20005 | Loss: 2.8999 | LR: 2.45e-05 | 5776.2ms/step | 4.2 samples/s
Step 17210/20005 | Loss: 2.8940 | LR: 2.43e-05 | 5774.8ms/step | 4.2 samples/s
Step 17220/20005 | Loss: 2.8870 | LR: 2.41e-05 | 5774.9ms/step | 4.2 samples/s
Step 17230/20005 | Loss: 2.9007 | LR: 2.40e-05 | 5767.9ms/step | 4.2 samples/s
Step 17240/20005 | Loss: 2.8855 | LR: 2.38e-05 | 5772.6ms/step | 4.2 samples/s
Step 17250/20005 | Loss: 2.9039 | LR: 2.36e-05 | 5771.8ms/step | 4.2 samples/s
Validation at step 17250: Loss = 2.8987, Perplexity = 18.15
Step 17260/20005 | Loss: 2.9294 | LR: 2.34e-05 | 40604.2ms/step | 0.6 samples/s
Step 17270/20005 | Loss: 2.8687 | LR: 2.33e-05 | 5776.1ms/step | 4.2 samples/s
Step 17280/20005 | Loss: 2.8607 | LR: 2.31e-05 | 5778.0ms/step | 4.2 samples/s
Step 17290/20005 | Loss: 2.8639 | LR: 2.29e-05 | 5776.9ms/step | 4.2 samples/s
Step 17300/20005 | Loss: 2.9067 | LR: 2.28e-05 | 5773.0ms/step | 4.2 samples/s
Step 17310/20005 | Loss: 2.8846 | LR: 2.26e-05 | 5776.1ms/step | 4.2 samples/s
Step 17320/20005 | Loss: 2.9013 | LR: 2.24e-05 | 5772.0ms/step | 4.2 samples/s
Step 17330/20005 | Loss: 2.8794 | LR: 2.23e-05 | 5774.1ms/step | 4.2 samples/s
Step 17340/20005 | Loss: 2.9186 | LR: 2.21e-05 | 5772.0ms/step | 4.2 samples/s
Step 17350/20005 | Loss: 2.9112 | LR: 2.20e-05 | 5772.3ms/step | 4.2 samples/s
Step 17360/20005 | Loss: 2.9174 | LR: 2.18e-05 | 5771.9ms/step | 4.2 samples/s
Step 17370/20005 | Loss: 2.9201 | LR: 2.16e-05 | 5773.9ms/step | 4.2 samples/s
Step 17380/20005 | Loss: 2.9030 | LR: 2.15e-05 | 5774.6ms/step | 4.2 samples/s
Step 17390/20005 | Loss: 2.9204 | LR: 2.13e-05 | 5775.3ms/step | 4.2 samples/s
Step 17400/20005 | Loss: 2.8932 | LR: 2.11e-05 | 5772.0ms/step | 4.2 samples/s
Step 17410/20005 | Loss: 2.8758 | LR: 2.10e-05 | 5770.5ms/step | 4.2 samples/s
Step 17420/20005 | Loss: 2.8839 | LR: 2.08e-05 | 5777.8ms/step | 4.2 samples/s
Step 17430/20005 | Loss: 2.9133 | LR: 2.07e-05 | 5774.3ms/step | 4.2 samples/s
Step 17440/20005 | Loss: 2.8763 | LR: 2.05e-05 | 5773.9ms/step | 4.2 samples/s
Step 17450/20005 | Loss: 2.9008 | LR: 2.04e-05 | 5770.0ms/step | 4.2 samples/s
Step 17460/20005 | Loss: 2.8791 | LR: 2.02e-05 | 5777.9ms/step | 4.2 samples/s
Step 17470/20005 | Loss: 2.8855 | LR: 2.00e-05 | 5776.0ms/step | 4.2 samples/s
Step 17480/20005 | Loss: 2.9057 | LR: 1.99e-05 | 5772.2ms/step | 4.2 samples/s
Step 17490/20005 | Loss: 2.9057 | LR: 1.97e-05 | 5775.3ms/step | 4.2 samples/s
Step 17500/20005 | Loss: 2.8854 | LR: 1.96e-05 | 5776.8ms/step | 4.2 samples/s
Validation at step 17500: Loss = 2.8985, Perplexity = 18.15
Step 17510/20005 | Loss: 2.8930 | LR: 1.94e-05 | 40609.5ms/step | 0.6 samples/s
Step 17520/20005 | Loss: 2.9128 | LR: 1.93e-05 | 5780.6ms/step | 4.2 samples/s
Step 17530/20005 | Loss: 2.8773 | LR: 1.91e-05 | 5780.0ms/step | 4.2 samples/s
Step 17540/20005 | Loss: 2.8828 | LR: 1.90e-05 | 5778.1ms/step | 4.2 samples/s
Step 17550/20005 | Loss: 2.8824 | LR: 1.88e-05 | 5780.2ms/step | 4.2 samples/s
Step 17560/20005 | Loss: 2.9061 | LR: 1.87e-05 | 5849.7ms/step | 4.1 samples/s
Step 17570/20005 | Loss: 2.9081 | LR: 1.85e-05 | 5772.1ms/step | 4.2 samples/s
Step 17580/20005 | Loss: 2.9184 | LR: 1.84e-05 | 5774.1ms/step | 4.2 samples/s
Step 17590/20005 | Loss: 2.9017 | LR: 1.82e-05 | 5782.6ms/step | 4.2 samples/s
Step 17600/20005 | Loss: 2.8868 | LR: 1.81e-05 | 5790.0ms/step | 4.1 samples/s
Step 17610/20005 | Loss: 2.8887 | LR: 1.79e-05 | 5783.7ms/step | 4.1 samples/s
Step 17620/20005 | Loss: 2.9259 | LR: 1.78e-05 | 5781.3ms/step | 4.2 samples/s
Step 17630/20005 | Loss: 2.8849 | LR: 1.76e-05 | 5783.8ms/step | 4.1 samples/s
Step 17640/20005 | Loss: 2.8900 | LR: 1.75e-05 | 5784.8ms/step | 4.1 samples/s
Step 17650/20005 | Loss: 2.8846 | LR: 1.73e-05 | 5785.1ms/step | 4.1 samples/s
Step 17660/20005 | Loss: 2.8928 | LR: 1.72e-05 | 5788.7ms/step | 4.1 samples/s
Step 17670/20005 | Loss: 2.9402 | LR: 1.70e-05 | 5786.4ms/step | 4.1 samples/s
Step 17680/20005 | Loss: 2.9025 | LR: 1.69e-05 | 5784.0ms/step | 4.1 samples/s
Step 17690/20005 | Loss: 2.8941 | LR: 1.68e-05 | 5781.7ms/step | 4.2 samples/s
Step 17700/20005 | Loss: 2.9070 | LR: 1.66e-05 | 5789.3ms/step | 4.1 samples/s
Step 17710/20005 | Loss: 2.8938 | LR: 1.65e-05 | 5789.0ms/step | 4.1 samples/s
Step 17720/20005 | Loss: 2.8927 | LR: 1.63e-05 | 5773.3ms/step | 4.2 samples/s
Step 17730/20005 | Loss: 2.8759 | LR: 1.62e-05 | 5770.6ms/step | 4.2 samples/s
Step 17740/20005 | Loss: 2.9216 | LR: 1.60e-05 | 5770.2ms/step | 4.2 samples/s
Step 17750/20005 | Loss: 2.9081 | LR: 1.59e-05 | 5769.9ms/step | 4.2 samples/s
Validation at step 17750: Loss = 2.8984, Perplexity = 18.15
Step 17760/20005 | Loss: 2.8857 | LR: 1.58e-05 | 40614.0ms/step | 0.6 samples/s
Step 17770/20005 | Loss: 2.9177 | LR: 1.56e-05 | 5777.0ms/step | 4.2 samples/s
Step 17780/20005 | Loss: 2.9298 | LR: 1.55e-05 | 5777.5ms/step | 4.2 samples/s
Step 17790/20005 | Loss: 2.9106 | LR: 1.53e-05 | 5781.5ms/step | 4.2 samples/s
Step 17800/20005 | Loss: 2.9097 | LR: 1.52e-05 | 5773.9ms/step | 4.2 samples/s
Step 17810/20005 | Loss: 2.9072 | LR: 1.51e-05 | 5772.4ms/step | 4.2 samples/s
Step 17820/20005 | Loss: 2.8738 | LR: 1.49e-05 | 5777.7ms/step | 4.2 samples/s
Step 17830/20005 | Loss: 2.9146 | LR: 1.48e-05 | 5775.9ms/step | 4.2 samples/s
Step 17840/20005 | Loss: 2.8918 | LR: 1.47e-05 | 5774.6ms/step | 4.2 samples/s
Step 17850/20005 | Loss: 2.8890 | LR: 1.45e-05 | 5775.7ms/step | 4.2 samples/s
Step 17860/20005 | Loss: 2.9097 | LR: 1.44e-05 | 5773.8ms/step | 4.2 samples/s
Step 17870/20005 | Loss: 2.8813 | LR: 1.43e-05 | 5769.5ms/step | 4.2 samples/s
Step 17880/20005 | Loss: 2.9141 | LR: 1.41e-05 | 5774.7ms/step | 4.2 samples/s
Step 17890/20005 | Loss: 2.8736 | LR: 1.40e-05 | 5774.3ms/step | 4.2 samples/s
Step 17900/20005 | Loss: 2.8767 | LR: 1.39e-05 | 5774.6ms/step | 4.2 samples/s
Step 17910/20005 | Loss: 2.9126 | LR: 1.37e-05 | 5775.1ms/step | 4.2 samples/s
Step 17920/20005 | Loss: 2.8820 | LR: 1.36e-05 | 5776.3ms/step | 4.2 samples/s
Step 17930/20005 | Loss: 2.9260 | LR: 1.35e-05 | 5774.9ms/step | 4.2 samples/s
Step 17940/20005 | Loss: 2.9053 | LR: 1.34e-05 | 5774.9ms/step | 4.2 samples/s
Step 17950/20005 | Loss: 2.9036 | LR: 1.32e-05 | 5773.9ms/step | 4.2 samples/s
Step 17960/20005 | Loss: 2.9021 | LR: 1.31e-05 | 5775.5ms/step | 4.2 samples/s
Step 17970/20005 | Loss: 2.9111 | LR: 1.30e-05 | 5775.1ms/step | 4.2 samples/s
Step 17980/20005 | Loss: 2.8640 | LR: 1.29e-05 | 5774.7ms/step | 4.2 samples/s
Step 17990/20005 | Loss: 2.8979 | LR: 1.27e-05 | 5771.0ms/step | 4.2 samples/s
Step 18000/20005 | Loss: 2.8909 | LR: 1.26e-05 | 5772.8ms/step | 4.2 samples/s
Validation at step 18000: Loss = 2.8983, Perplexity = 18.14
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-18000
Step 18010/20005 | Loss: 2.8902 | LR: 1.25e-05 | 41084.3ms/step | 0.6 samples/s
Step 18020/20005 | Loss: 2.8956 | LR: 1.24e-05 | 5774.9ms/step | 4.2 samples/s
Step 18030/20005 | Loss: 2.9045 | LR: 1.22e-05 | 5812.7ms/step | 4.1 samples/s
Step 18040/20005 | Loss: 2.9236 | LR: 1.21e-05 | 5777.6ms/step | 4.2 samples/s
Step 18050/20005 | Loss: 2.9186 | LR: 1.20e-05 | 5773.2ms/step | 4.2 samples/s
Step 18060/20005 | Loss: 2.8921 | LR: 1.19e-05 | 5771.2ms/step | 4.2 samples/s
Step 18070/20005 | Loss: 2.8953 | LR: 1.17e-05 | 5772.1ms/step | 4.2 samples/s
Step 18080/20005 | Loss: 2.9111 | LR: 1.16e-05 | 5773.6ms/step | 4.2 samples/s
Step 18090/20005 | Loss: 2.9054 | LR: 1.15e-05 | 5776.3ms/step | 4.2 samples/s
Step 18100/20005 | Loss: 2.9103 | LR: 1.14e-05 | 5772.4ms/step | 4.2 samples/s
Step 18110/20005 | Loss: 2.8723 | LR: 1.13e-05 | 5775.5ms/step | 4.2 samples/s
Step 18120/20005 | Loss: 2.9001 | LR: 1.11e-05 | 5777.8ms/step | 4.2 samples/s
Step 18130/20005 | Loss: 2.8804 | LR: 1.10e-05 | 5776.6ms/step | 4.2 samples/s
Step 18140/20005 | Loss: 2.9054 | LR: 1.09e-05 | 5771.9ms/step | 4.2 samples/s
Step 18150/20005 | Loss: 2.9073 | LR: 1.08e-05 | 5774.1ms/step | 4.2 samples/s
Step 18160/20005 | Loss: 2.8866 | LR: 1.07e-05 | 5775.8ms/step | 4.2 samples/s
Step 18170/20005 | Loss: 2.9064 | LR: 1.06e-05 | 5775.6ms/step | 4.2 samples/s
Step 18180/20005 | Loss: 2.9225 | LR: 1.05e-05 | 5775.2ms/step | 4.2 samples/s
Step 18190/20005 | Loss: 2.9190 | LR: 1.03e-05 | 5779.2ms/step | 4.2 samples/s
Step 18200/20005 | Loss: 2.8762 | LR: 1.02e-05 | 5774.0ms/step | 4.2 samples/s
Step 18210/20005 | Loss: 2.8836 | LR: 1.01e-05 | 5775.8ms/step | 4.2 samples/s
Step 18220/20005 | Loss: 2.9205 | LR: 1.00e-05 | 5776.6ms/step | 4.2 samples/s
Step 18230/20005 | Loss: 2.9069 | LR: 9.89e-06 | 5776.9ms/step | 4.2 samples/s
Step 18240/20005 | Loss: 2.8895 | LR: 9.78e-06 | 5772.6ms/step | 4.2 samples/s
Step 18250/20005 | Loss: 2.9064 | LR: 9.67e-06 | 5772.8ms/step | 4.2 samples/s
Validation at step 18250: Loss = 2.8983, Perplexity = 18.14
Step 18260/20005 | Loss: 2.9053 | LR: 9.56e-06 | 40628.1ms/step | 0.6 samples/s
Step 18270/20005 | Loss: 2.8868 | LR: 9.46e-06 | 5778.2ms/step | 4.2 samples/s
Step 18280/20005 | Loss: 2.8841 | LR: 9.35e-06 | 5779.7ms/step | 4.2 samples/s
Step 18290/20005 | Loss: 2.9156 | LR: 9.24e-06 | 5776.1ms/step | 4.2 samples/s
Step 18300/20005 | Loss: 2.9039 | LR: 9.13e-06 | 5779.7ms/step | 4.2 samples/s
Step 18310/20005 | Loss: 2.8899 | LR: 9.03e-06 | 5776.4ms/step | 4.2 samples/s
Step 18320/20005 | Loss: 2.9159 | LR: 8.92e-06 | 5777.7ms/step | 4.2 samples/s
Step 18330/20005 | Loss: 2.9122 | LR: 8.82e-06 | 5772.2ms/step | 4.2 samples/s
Step 18340/20005 | Loss: 2.8870 | LR: 8.71e-06 | 5773.9ms/step | 4.2 samples/s
Step 18350/20005 | Loss: 2.9022 | LR: 8.61e-06 | 5777.8ms/step | 4.2 samples/s
Step 18360/20005 | Loss: 2.8743 | LR: 8.51e-06 | 5774.8ms/step | 4.2 samples/s
Step 18370/20005 | Loss: 2.8775 | LR: 8.40e-06 | 5773.3ms/step | 4.2 samples/s
Step 18380/20005 | Loss: 2.8923 | LR: 8.30e-06 | 5774.9ms/step | 4.2 samples/s
Step 18390/20005 | Loss: 2.8576 | LR: 8.20e-06 | 5777.6ms/step | 4.2 samples/s
Step 18400/20005 | Loss: 2.8843 | LR: 8.10e-06 | 5776.0ms/step | 4.2 samples/s
Step 18410/20005 | Loss: 2.9030 | LR: 8.00e-06 | 5776.8ms/step | 4.2 samples/s
Step 18420/20005 | Loss: 2.8968 | LR: 7.90e-06 | 5775.3ms/step | 4.2 samples/s
Step 18430/20005 | Loss: 2.8717 | LR: 7.80e-06 | 5780.0ms/step | 4.2 samples/s
Step 18440/20005 | Loss: 2.9176 | LR: 7.70e-06 | 5775.8ms/step | 4.2 samples/s
Step 18450/20005 | Loss: 2.8941 | LR: 7.61e-06 | 5772.4ms/step | 4.2 samples/s
Step 18460/20005 | Loss: 2.8500 | LR: 7.51e-06 | 5775.9ms/step | 4.2 samples/s
Step 18470/20005 | Loss: 2.9019 | LR: 7.41e-06 | 5774.3ms/step | 4.2 samples/s
Step 18480/20005 | Loss: 2.9006 | LR: 7.32e-06 | 5773.9ms/step | 4.2 samples/s
Step 18490/20005 | Loss: 2.8782 | LR: 7.22e-06 | 5776.3ms/step | 4.2 samples/s
Step 18500/20005 | Loss: 2.9142 | LR: 7.13e-06 | 5773.0ms/step | 4.2 samples/s
Validation at step 18500: Loss = 2.8983, Perplexity = 18.14
Step 18510/20005 | Loss: 2.9017 | LR: 7.03e-06 | 40658.5ms/step | 0.6 samples/s
Step 18520/20005 | Loss: 2.8746 | LR: 6.94e-06 | 5776.6ms/step | 4.2 samples/s
Step 18530/20005 | Loss: 2.9257 | LR: 6.85e-06 | 5777.6ms/step | 4.2 samples/s
Step 18540/20005 | Loss: 2.8841 | LR: 6.75e-06 | 5778.8ms/step | 4.2 samples/s
Step 18550/20005 | Loss: 2.8883 | LR: 6.66e-06 | 5773.7ms/step | 4.2 samples/s
Step 18560/20005 | Loss: 2.8864 | LR: 6.57e-06 | 5773.6ms/step | 4.2 samples/s
Step 18570/20005 | Loss: 2.8755 | LR: 6.48e-06 | 5774.3ms/step | 4.2 samples/s
Step 18580/20005 | Loss: 2.9294 | LR: 6.39e-06 | 5773.6ms/step | 4.2 samples/s
Step 18590/20005 | Loss: 2.8963 | LR: 6.30e-06 | 5773.3ms/step | 4.2 samples/s
Step 18600/20005 | Loss: 2.8904 | LR: 6.21e-06 | 5777.5ms/step | 4.2 samples/s
Step 18610/20005 | Loss: 2.9091 | LR: 6.13e-06 | 5777.9ms/step | 4.2 samples/s
Step 18620/20005 | Loss: 2.9148 | LR: 6.04e-06 | 5777.8ms/step | 4.2 samples/s
Step 18630/20005 | Loss: 2.8931 | LR: 5.95e-06 | 5774.5ms/step | 4.2 samples/s
Step 18640/20005 | Loss: 2.8983 | LR: 5.87e-06 | 5775.4ms/step | 4.2 samples/s
Step 18650/20005 | Loss: 2.9136 | LR: 5.78e-06 | 5772.4ms/step | 4.2 samples/s
Step 18660/20005 | Loss: 2.8781 | LR: 5.70e-06 | 5780.1ms/step | 4.2 samples/s
Step 18670/20005 | Loss: 2.9017 | LR: 5.61e-06 | 5774.1ms/step | 4.2 samples/s
Step 18680/20005 | Loss: 2.8801 | LR: 5.53e-06 | 5779.4ms/step | 4.2 samples/s
Step 18690/20005 | Loss: 2.8899 | LR: 5.45e-06 | 5776.4ms/step | 4.2 samples/s
Step 18700/20005 | Loss: 2.9030 | LR: 5.36e-06 | 5778.2ms/step | 4.2 samples/s
Step 18710/20005 | Loss: 2.8770 | LR: 5.28e-06 | 5777.7ms/step | 4.2 samples/s
Step 18720/20005 | Loss: 2.8651 | LR: 5.20e-06 | 5778.0ms/step | 4.2 samples/s
Step 18730/20005 | Loss: 2.8938 | LR: 5.12e-06 | 5771.9ms/step | 4.2 samples/s
Step 18740/20005 | Loss: 2.9011 | LR: 5.04e-06 | 5777.4ms/step | 4.2 samples/s
Step 18750/20005 | Loss: 2.8885 | LR: 4.96e-06 | 5774.7ms/step | 4.2 samples/s
Validation at step 18750: Loss = 2.8982, Perplexity = 18.14
Step 18760/20005 | Loss: 2.8930 | LR: 4.88e-06 | 40646.8ms/step | 0.6 samples/s
Step 18770/20005 | Loss: 2.8989 | LR: 4.81e-06 | 5777.9ms/step | 4.2 samples/s
Step 18780/20005 | Loss: 2.8853 | LR: 4.73e-06 | 5779.7ms/step | 4.2 samples/s
Step 18790/20005 | Loss: 2.8973 | LR: 4.65e-06 | 5778.5ms/step | 4.2 samples/s
Step 18800/20005 | Loss: 2.9341 | LR: 4.58e-06 | 5777.6ms/step | 4.2 samples/s
Step 18810/20005 | Loss: 2.9092 | LR: 4.50e-06 | 5779.9ms/step | 4.2 samples/s
Step 18820/20005 | Loss: 2.8921 | LR: 4.43e-06 | 5777.9ms/step | 4.2 samples/s
Step 18830/20005 | Loss: 2.9099 | LR: 4.35e-06 | 5776.5ms/step | 4.2 samples/s
Step 18840/20005 | Loss: 2.9202 | LR: 4.28e-06 | 5777.7ms/step | 4.2 samples/s
Step 18850/20005 | Loss: 2.9186 | LR: 4.21e-06 | 5773.8ms/step | 4.2 samples/s
Step 18860/20005 | Loss: 2.8992 | LR: 4.13e-06 | 5768.1ms/step | 4.2 samples/s
Step 18870/20005 | Loss: 2.9059 | LR: 4.06e-06 | 5772.0ms/step | 4.2 samples/s
Step 18880/20005 | Loss: 2.8871 | LR: 3.99e-06 | 5772.3ms/step | 4.2 samples/s
Step 18890/20005 | Loss: 2.8933 | LR: 3.92e-06 | 5766.1ms/step | 4.2 samples/s
Step 18900/20005 | Loss: 2.8916 | LR: 3.85e-06 | 5768.2ms/step | 4.2 samples/s
Step 18910/20005 | Loss: 2.8995 | LR: 3.78e-06 | 5771.4ms/step | 4.2 samples/s
Step 18920/20005 | Loss: 2.8992 | LR: 3.71e-06 | 5772.7ms/step | 4.2 samples/s
Step 18930/20005 | Loss: 2.8685 | LR: 3.64e-06 | 5773.9ms/step | 4.2 samples/s
Step 18940/20005 | Loss: 2.8870 | LR: 3.58e-06 | 5767.2ms/step | 4.2 samples/s
Step 18950/20005 | Loss: 2.9112 | LR: 3.51e-06 | 5774.9ms/step | 4.2 samples/s
Step 18960/20005 | Loss: 2.8901 | LR: 3.44e-06 | 5771.7ms/step | 4.2 samples/s
Step 18970/20005 | Loss: 2.8847 | LR: 3.38e-06 | 5768.4ms/step | 4.2 samples/s
Step 18980/20005 | Loss: 2.8769 | LR: 3.31e-06 | 5773.6ms/step | 4.2 samples/s
Step 18990/20005 | Loss: 2.8876 | LR: 3.25e-06 | 5809.0ms/step | 4.1 samples/s
Step 19000/20005 | Loss: 2.8901 | LR: 3.19e-06 | 5772.9ms/step | 4.2 samples/s
Validation at step 19000: Loss = 2.8982, Perplexity = 18.14
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-19000
Step 19010/20005 | Loss: 2.8988 | LR: 3.12e-06 | 41054.8ms/step | 0.6 samples/s
Step 19020/20005 | Loss: 2.9149 | LR: 3.06e-06 | 5769.6ms/step | 4.2 samples/s
Step 19030/20005 | Loss: 2.9116 | LR: 3.00e-06 | 5772.2ms/step | 4.2 samples/s
Step 19040/20005 | Loss: 2.8763 | LR: 2.94e-06 | 5772.6ms/step | 4.2 samples/s
Step 19050/20005 | Loss: 2.9285 | LR: 2.88e-06 | 5767.8ms/step | 4.2 samples/s
Step 19060/20005 | Loss: 2.8894 | LR: 2.82e-06 | 5774.2ms/step | 4.2 samples/s
Step 19070/20005 | Loss: 2.8974 | LR: 2.76e-06 | 5769.6ms/step | 4.2 samples/s
Step 19080/20005 | Loss: 2.8984 | LR: 2.70e-06 | 5769.9ms/step | 4.2 samples/s
Step 19090/20005 | Loss: 2.9048 | LR: 2.64e-06 | 5767.9ms/step | 4.2 samples/s
Step 19100/20005 | Loss: 2.8982 | LR: 2.58e-06 | 5770.4ms/step | 4.2 samples/s
Step 19110/20005 | Loss: 2.8979 | LR: 2.53e-06 | 5769.8ms/step | 4.2 samples/s
Step 19120/20005 | Loss: 2.9239 | LR: 2.47e-06 | 5766.6ms/step | 4.2 samples/s
Step 19130/20005 | Loss: 2.8830 | LR: 2.42e-06 | 5773.4ms/step | 4.2 samples/s
Step 19140/20005 | Loss: 2.8834 | LR: 2.36e-06 | 5776.1ms/step | 4.2 samples/s
Step 19150/20005 | Loss: 2.9334 | LR: 2.31e-06 | 5774.0ms/step | 4.2 samples/s
Step 19160/20005 | Loss: 2.8826 | LR: 2.25e-06 | 5772.2ms/step | 4.2 samples/s
Step 19170/20005 | Loss: 2.9021 | LR: 2.20e-06 | 5774.3ms/step | 4.2 samples/s
Step 19180/20005 | Loss: 2.8762 | LR: 2.15e-06 | 5775.8ms/step | 4.2 samples/s
Step 19190/20005 | Loss: 2.9307 | LR: 2.10e-06 | 5774.2ms/step | 4.2 samples/s
Step 19200/20005 | Loss: 2.8957 | LR: 2.05e-06 | 5774.2ms/step | 4.2 samples/s
Step 19210/20005 | Loss: 2.9074 | LR: 2.00e-06 | 5778.3ms/step | 4.2 samples/s
Step 19220/20005 | Loss: 2.8947 | LR: 1.95e-06 | 5772.0ms/step | 4.2 samples/s
Step 19230/20005 | Loss: 2.8778 | LR: 1.90e-06 | 5767.3ms/step | 4.2 samples/s
Step 19240/20005 | Loss: 2.8654 | LR: 1.85e-06 | 5771.7ms/step | 4.2 samples/s
Step 19250/20005 | Loss: 2.9028 | LR: 1.80e-06 | 5772.0ms/step | 4.2 samples/s
Validation at step 19250: Loss = 2.8982, Perplexity = 18.14
Step 19260/20005 | Loss: 2.8846 | LR: 1.75e-06 | 40591.2ms/step | 0.6 samples/s
Step 19270/20005 | Loss: 2.9139 | LR: 1.71e-06 | 5771.6ms/step | 4.2 samples/s
Step 19280/20005 | Loss: 2.8764 | LR: 1.66e-06 | 5770.1ms/step | 4.2 samples/s
Step 19290/20005 | Loss: 2.8742 | LR: 1.61e-06 | 5768.0ms/step | 4.2 samples/s
Step 19300/20005 | Loss: 2.9048 | LR: 1.57e-06 | 5765.1ms/step | 4.2 samples/s
Step 19310/20005 | Loss: 2.9011 | LR: 1.53e-06 | 5770.7ms/step | 4.2 samples/s
Step 19320/20005 | Loss: 2.9075 | LR: 1.48e-06 | 5773.9ms/step | 4.2 samples/s
Step 19330/20005 | Loss: 2.8954 | LR: 1.44e-06 | 5770.8ms/step | 4.2 samples/s
Step 19340/20005 | Loss: 2.9023 | LR: 1.40e-06 | 5775.5ms/step | 4.2 samples/s
Step 19350/20005 | Loss: 2.9180 | LR: 1.36e-06 | 5770.0ms/step | 4.2 samples/s
Step 19360/20005 | Loss: 2.9053 | LR: 1.31e-06 | 5764.1ms/step | 4.2 samples/s
Step 19370/20005 | Loss: 2.8971 | LR: 1.27e-06 | 5774.0ms/step | 4.2 samples/s
Step 19380/20005 | Loss: 2.8980 | LR: 1.23e-06 | 5776.0ms/step | 4.2 samples/s
Step 19390/20005 | Loss: 2.8861 | LR: 1.19e-06 | 5772.4ms/step | 4.2 samples/s
Step 19400/20005 | Loss: 2.8896 | LR: 1.16e-06 | 5776.2ms/step | 4.2 samples/s
Step 19410/20005 | Loss: 2.8932 | LR: 1.12e-06 | 5775.7ms/step | 4.2 samples/s
Step 19420/20005 | Loss: 2.9081 | LR: 1.08e-06 | 5781.2ms/step | 4.2 samples/s
Step 19430/20005 | Loss: 2.8806 | LR: 1.04e-06 | 5780.3ms/step | 4.2 samples/s
Step 19440/20005 | Loss: 2.8936 | LR: 1.01e-06 | 5776.6ms/step | 4.2 samples/s
Step 19450/20005 | Loss: 2.8661 | LR: 9.73e-07 | 5774.1ms/step | 4.2 samples/s
Step 19460/20005 | Loss: 2.8703 | LR: 9.38e-07 | 5810.4ms/step | 4.1 samples/s
Step 19470/20005 | Loss: 2.8867 | LR: 9.04e-07 | 5777.3ms/step | 4.2 samples/s
Step 19480/20005 | Loss: 2.9071 | LR: 8.71e-07 | 5780.0ms/step | 4.2 samples/s
Step 19490/20005 | Loss: 2.8720 | LR: 8.38e-07 | 5776.7ms/step | 4.2 samples/s
Step 19500/20005 | Loss: 2.8925 | LR: 8.06e-07 | 5773.0ms/step | 4.2 samples/s
Validation at step 19500: Loss = 2.8982, Perplexity = 18.14
Step 19510/20005 | Loss: 2.8910 | LR: 7.74e-07 | 40643.4ms/step | 0.6 samples/s
Step 19520/20005 | Loss: 2.8851 | LR: 7.43e-07 | 5777.8ms/step | 4.2 samples/s
Step 19530/20005 | Loss: 2.9137 | LR: 7.13e-07 | 5775.9ms/step | 4.2 samples/s
Step 19540/20005 | Loss: 2.8846 | LR: 6.83e-07 | 5778.1ms/step | 4.2 samples/s
Step 19550/20005 | Loss: 2.8789 | LR: 6.54e-07 | 5775.7ms/step | 4.2 samples/s
Step 19560/20005 | Loss: 2.8724 | LR: 6.26e-07 | 5778.9ms/step | 4.2 samples/s
Step 19570/20005 | Loss: 2.8905 | LR: 5.98e-07 | 5779.3ms/step | 4.2 samples/s
Step 19580/20005 | Loss: 2.8814 | LR: 5.71e-07 | 5781.9ms/step | 4.2 samples/s
Step 19590/20005 | Loss: 2.8987 | LR: 5.44e-07 | 5782.3ms/step | 4.2 samples/s
Step 19600/20005 | Loss: 2.9004 | LR: 5.18e-07 | 5777.7ms/step | 4.2 samples/s
Step 19610/20005 | Loss: 2.8897 | LR: 4.93e-07 | 5772.4ms/step | 4.2 samples/s
Step 19620/20005 | Loss: 2.8928 | LR: 4.68e-07 | 5778.4ms/step | 4.2 samples/s
Step 19630/20005 | Loss: 2.9112 | LR: 4.44e-07 | 5777.6ms/step | 4.2 samples/s
Step 19640/20005 | Loss: 2.8856 | LR: 4.21e-07 | 5778.3ms/step | 4.2 samples/s
Step 19650/20005 | Loss: 2.8875 | LR: 3.98e-07 | 5774.1ms/step | 4.2 samples/s
Step 19660/20005 | Loss: 2.8841 | LR: 3.76e-07 | 5775.4ms/step | 4.2 samples/s
Step 19670/20005 | Loss: 2.9158 | LR: 3.55e-07 | 5779.2ms/step | 4.2 samples/s
Step 19680/20005 | Loss: 2.8632 | LR: 3.34e-07 | 5781.1ms/step | 4.2 samples/s
Step 19690/20005 | Loss: 2.8853 | LR: 3.14e-07 | 5777.0ms/step | 4.2 samples/s
Step 19700/20005 | Loss: 2.9027 | LR: 2.94e-07 | 5778.2ms/step | 4.2 samples/s
Step 19710/20005 | Loss: 2.8666 | LR: 2.75e-07 | 5777.2ms/step | 4.2 samples/s
Step 19720/20005 | Loss: 2.8934 | LR: 2.57e-07 | 5780.2ms/step | 4.2 samples/s
Step 19730/20005 | Loss: 2.8917 | LR: 2.39e-07 | 5790.0ms/step | 4.1 samples/s
Step 19740/20005 | Loss: 2.8743 | LR: 2.22e-07 | 5782.0ms/step | 4.2 samples/s
Step 19750/20005 | Loss: 2.8870 | LR: 2.06e-07 | 5779.9ms/step | 4.2 samples/s
Validation at step 19750: Loss = 2.8982, Perplexity = 18.14
Step 19760/20005 | Loss: 2.8843 | LR: 1.90e-07 | 40728.8ms/step | 0.6 samples/s
Step 19770/20005 | Loss: 2.9059 | LR: 1.75e-07 | 5785.2ms/step | 4.1 samples/s
Step 19780/20005 | Loss: 2.9084 | LR: 1.60e-07 | 5778.5ms/step | 4.2 samples/s
Step 19790/20005 | Loss: 2.9043 | LR: 1.46e-07 | 5780.3ms/step | 4.2 samples/s
Step 19800/20005 | Loss: 2.8925 | LR: 1.33e-07 | 5779.9ms/step | 4.2 samples/s
Step 19810/20005 | Loss: 2.8796 | LR: 1.20e-07 | 5779.9ms/step | 4.2 samples/s
Step 19820/20005 | Loss: 2.8851 | LR: 1.08e-07 | 5784.4ms/step | 4.1 samples/s
Step 19830/20005 | Loss: 2.8829 | LR: 9.68e-08 | 5779.9ms/step | 4.2 samples/s
Step 19840/20005 | Loss: 2.9029 | LR: 8.61e-08 | 5781.2ms/step | 4.2 samples/s
Step 19850/20005 | Loss: 2.8849 | LR: 7.59e-08 | 5783.7ms/step | 4.1 samples/s
Step 19860/20005 | Loss: 2.8948 | LR: 6.65e-08 | 5781.6ms/step | 4.2 samples/s
Step 19870/20005 | Loss: 2.8869 | LR: 5.76e-08 | 5781.4ms/step | 4.2 samples/s
Step 19880/20005 | Loss: 2.8835 | LR: 4.94e-08 | 5784.1ms/step | 4.1 samples/s
Step 19890/20005 | Loss: 2.8833 | LR: 4.18e-08 | 5783.1ms/step | 4.2 samples/s
Step 19900/20005 | Loss: 2.8916 | LR: 3.49e-08 | 5779.1ms/step | 4.2 samples/s
Step 19910/20005 | Loss: 2.8811 | LR: 2.85e-08 | 5785.7ms/step | 4.1 samples/s
Step 19920/20005 | Loss: 2.9128 | LR: 2.28e-08 | 5784.0ms/step | 4.1 samples/s
Step 19930/20005 | Loss: 2.8841 | LR: 1.78e-08 | 5784.6ms/step | 4.1 samples/s
Step 19940/20005 | Loss: 2.8744 | LR: 1.34e-08 | 5863.5ms/step | 4.1 samples/s
Step 19950/20005 | Loss: 2.9060 | LR: 9.56e-09 | 5782.3ms/step | 4.2 samples/s
Step 19960/20005 | Loss: 2.9006 | LR: 6.40e-09 | 5777.9ms/step | 4.2 samples/s
Step 19970/20005 | Loss: 2.9035 | LR: 3.87e-09 | 5776.0ms/step | 4.2 samples/s
Step 19980/20005 | Loss: 2.8534 | LR: 1.98e-09 | 5784.0ms/step | 4.1 samples/s
Step 19990/20005 | Loss: 2.9112 | LR: 7.11e-10 | 5782.5ms/step | 4.2 samples/s
Step 20000/20005 | Loss: 2.8839 | LR: 7.90e-11 | 5779.7ms/step | 4.2 samples/s
Validation at step 20000: Loss = 2.8982, Perplexity = 18.14
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-20000
Saved checkpoint to checkpoints/run_20260509_201713/checkpoint-20005
Training complete!